About the Role A technology company focused on advanced content understanding systems is seeking a Knowledge Graph Curator to support the growth, maintenance, and quality of a large-scale knowledge graph. The role involves working with machine learning systems, knowledge…
Data Science Expert
Remote (Global)
$120 – $170/hr
July 11, 2026
Job description
Role Overview
Mercor is partnering with a leading AI research organization to engage experienced Data Science experts to evaluate how effectively AI systems perform real-world data science work.
Rather than producing deliverables yourself, you will define what high-quality data science looks like by creating task-specific grading criteria and evaluating AI-generated and human work using rigorous, evidence-based scoring.
This is a fully remote, hourly contract opportunity for experienced industry data scientists.
Key Responsibilities
Design Evaluation Criteria
- Develop precise, task-specific grading rubrics for real-world data science deliverables, including:
- Data analyses
- Predictive models
- Dashboards
- Experiment reports
- Business recommendations
Evaluate AI Outputs
- Review AI-generated and human work samples against established evaluation criteria.
- Provide detailed written justifications supporting every score.
Ensure Consistent Quality
- Apply objective, evidence-based judgment to ensure evaluations are reproducible, consistent, and defensible.
Improve Evaluation Standards
- Incorporate structured feedback from senior reviewers.
- Continuously refine grading approaches and evaluation quality.
Required Qualifications
5+ years of professional Data Science experience in industry.
Experience working in:
- Business Data Science
- Product Analytics
- Growth Analytics
- Decision Science
- Equivalent data science roles at leading technology companies.
Strong expertise in:
- Experiment design
- A/B testing
- Statistical analysis
- SQL
- Python
- Metric design
- Data visualization
- Executive reporting
Exceptional written communication skills.
Strong analytical thinking with careful attention to detail.
Comfortable working in calibration and peer-review environments.
Previous experience with AI evaluation, RLHF, data annotation, or AI training projects is a strong advantage.
Compensation
- $120–$170 per hour
Work Arrangement
- Independent Contractor
- Fully Remote
- Flexible schedule
- Weekly payments via Stripe or Wise
Application Process
- Submit your resume or technical background.
- Qualified applicants may complete a brief technical assessment or provide additional information.
About Mercor
Mercor connects experienced professionals with leading AI laboratories to help train and evaluate frontier AI systems. Contributors work on high-impact AI research projects while enjoying flexible remote work and competitive compensation.
Apply now
You will be redirected to the company's website to complete your application.
About the Role Mercor is partnering with a leading AI research lab to support a Frontier Code Agents project. This role focuses on evaluating and improving frontier AI coding models through realistic data engineering workflows and structured technical assessments. Contributors…
Job Summary As a Python (ML-Focused) Quality Assurance Lead (QAL), you will oversee quality, consistency, and contributor performance across Python machine learning AI training projects. This role involves reviewing AI-generated Python code, machine learning workflows, model…
Job Summary As a Data Scientist Quality Assurance Lead (QAL), you will oversee quality, consistency, and contributor performance across data science and machine learning AI training projects. This role involves reviewing AI-generated data science content, evaluating trainer and…