What your lab gets
Preference data
Two or more model outputs ranked, with the reason for the ranking.
Ratings
Outputs scored on the dimensions your guidelines define.
Corrections
Model outputs rewritten so they meet your bar, with the changes marked.
Written answers
Ideal responses written from scratch for supervised fine-tuning.
Kinds of feedback work
Done inside your own tools, to your guidelines.
Pairwise ranking
Pick the better of two outputs and say why.
- chosen
- rejected
- rationale
Multi-output ranking
Order several outputs from best to worst.
- ranking
- notes
Rubric rating
Score each output on the dimensions you care about.
- scores
- dimensions
Rewrite and correct
Fix what the model got wrong and keep what it got right.
- original
- edit
- final
SFT writing
Write the response you want the model to learn from.
- prompt
- ideal answer
Guideline edge cases
Flag cases your guidelines do not cover yet, so they can be updated.
- case
- question
- proposal
Who does the work
Evaluators and writers from a software engineering company, matched to the subject of your prompts.
- Model evaluators who apply your guidelines consistently.
- Engineers for technical and coding prompts.
- Domain experts where judging an answer needs subject knowledge.
Four checks before work counts.
The same process runs on every OceanLab Labs project, and your QA always has the final say.
- 1
Scored trial task
Every person completes a scored trial task before they join your project, so only people who meet the bar work on your data.
- 2
Second reviewer
Submitted work is checked by a second reviewer before it reaches you.
- 3
Lead calibration
A lead runs calibration rounds so everyone applies your guidelines the same way as the work goes on.
- 4
Your QA decides
Your own QA has the final say on what is accepted. Anything you reject comes back to us and is fixed.
Your data stays yours.
NDA for every person
Everyone who works on your project signs an NDA before they see any of your material.
Your tools, your accounts
Work happens only inside your own tools and accounts. Your data does not leave them.
A dedicated team
The people on your work do not work for a competing lab at the same time.
Start small, then scale.
- 1
Tell us what you need
Share the work, the skills it needs, your guidelines or rubric, and your timeline. We match people to those requirements.
- 2
Paid pilot, small team
A small team works on your real tasks first, so you can judge the output before you commit to more.
- 3
Scale up
When the pilot meets your bar, we add people to the same work and keep it running for as long as you need.
Pricing is scoped per project, usually after a short paid pilot. See how it works.
Do you work to our guidelines?
Yes. Your guidelines define what good looks like. A lead runs calibration rounds so everyone applies them the same way.
Can you write answers as well as rate them?
Yes. The team can rank and rate outputs, correct them, or write ideal answers from scratch for supervised fine-tuning.
How do you check quality?
Each person passes a scored trial task before joining, a second reviewer checks submitted work, a lead runs calibration rounds, and your own QA has the final say. Anything you reject is fixed.
How is the work priced?
Pricing is scoped per project, usually after a short paid pilot on your real tasks. There is no public rate card.
How do you keep our data and tasks private?
Everyone on your project signs an NDA, the work happens only inside your own tools and accounts, and the team does not work for a competing lab at the same time.
Related pages.
- OceanLab Labs overview Everything OceanLab Labs does for AI labs, on one page.
- Coding dataCoding tasks, reference solutions, tests and reviews of model-written code.
- RL environmentsEnvironments, tasks and checkers for training agents with reinforcement learning.
- Evals and benchmarksEvaluation sets and benchmarks designed, written and graded to your spec.
- Domain expertsSpecialists who write and check tasks where subject knowledge decides the answer.
- How it worksFrom request to paid pilot to scale: quality, security, pricing and time zones.
Looking to automate your company's workflows with AI instead? See AI automation.
