Coding data written by working software engineers.
OceanLab Labs supplies software engineers who create and review coding data for your models: problems with reference solutions and tests, reviews of model-written code against your rubric, and multi-step tasks in real repositories.
What your lab gets
Tasks with solutions
Coding problems written to your spec, each with a reference solution your pipeline can check against.
Tests and checks
Unit tests and checks that decide whether a solution is correct, written alongside the task.
Code review
Engineers grade and critique model-written code against your rubric and correct it where it fails.
Repository tasks
Multi-step tasks inside real codebases, of the kind coding agents are trained and tested on.
Kinds of coding work
Delivered in the format your pipeline expects, inside your own tools.
Problem and solution
A self-contained problem, a reference solution and the tests that check it.
- prompt
- solution
- tests
Code critique
Rubric scores and a written critique of model output, with a corrected version.
- rubric score
- critique
- fix
Issue to patch
An issue in an existing repository, solved as a patch that passes its tests.
- issue
- repo
- patch
Agent steps
Step-by-step work through a task, the actions taken and the final change.
- steps
- tool use
- diff
Bug finding
Code with a planted or real bug, a description of the fault, and the fix.
- bug
- explanation
- fix
Preference pairs
Two candidate solutions compared, with the better one chosen and the reason given.
- chosen
- rejected
- rationale
Who does the work
Engineers from a software engineering company that has shipped 377 product builds across 19 sectors, not a crowd marketplace.
- Software engineers who build production software as their day job.
- Matched to the languages and frameworks your tasks need. We confirm coverage when we scope the work.
- Reviewers who read code critically, not only write it.
Four checks before work counts.
The same process runs on every OceanLab Labs project, and your QA always has the final say.
- 1
Scored trial task
Every person completes a scored trial task before they join your project, so only people who meet the bar work on your data.
- 2
Second reviewer
Submitted work is checked by a second reviewer before it reaches you.
- 3
Lead calibration
A lead runs calibration rounds so everyone applies your guidelines the same way as the work goes on.
- 4
Your QA decides
Your own QA has the final say on what is accepted. Anything you reject comes back to us and is fixed.
Your data stays yours.
NDA for every person
Everyone who works on your project signs an NDA before they see any of your material.
Your tools, your accounts
Work happens only inside your own tools and accounts. Your data does not leave them.
A dedicated team
The people on your work do not work for a competing lab at the same time.
Start small, then scale.
- 1
Tell us what you need
Share the work, the skills it needs, your guidelines or rubric, and your timeline. We match people to those requirements.
- 2
Paid pilot, small team
A small team works on your real tasks first, so you can judge the output before you commit to more.
- 3
Scale up
When the pilot meets your bar, we add people to the same work and keep it running for as long as you need.
Pricing is scoped per project, usually after a short paid pilot. See how it works.
Can you write tasks in our format?
Yes. You define the task format, the rubric and the bar. The team works to that specification inside your own tools.
Do you also review code written by our models?
Yes. Engineers grade and critique model-written code against your rubric and correct it where it falls short.
How do you check quality?
Each person passes a scored trial task before joining, a second reviewer checks submitted work, a lead runs calibration rounds, and your own QA has the final say. Anything you reject is fixed.
How is the work priced?
Pricing is scoped per project, usually after a short paid pilot on your real tasks. There is no public rate card.
How do you keep our data and tasks private?
Everyone on your project signs an NDA, the work happens only inside your own tools and accounts, and the team does not work for a competing lab at the same time.
Related pages.
- OceanLab Labs overview Everything OceanLab Labs does for AI labs, on one page.
- RL environmentsEnvironments, tasks and checkers for training agents with reinforcement learning.
- Evals and benchmarksEvaluation sets and benchmarks designed, written and graded to your spec.
- Human feedbackRLHF and SFT data: rating, ranking, writing and correcting model outputs.
- Domain expertsSpecialists who write and check tasks where subject knowledge decides the answer.
- How it worksFrom request to paid pilot to scale: quality, security, pricing and time zones.
Looking to automate your company's workflows with AI instead? See AI automation.
