Back to remote jobs

Generalists

Mercor

Response Rater Contractor · Full-time
United States $50 – $50/hr July 20, 2026

Job description

Mercor is seeking highly detail-oriented Generalists to support AI training and evaluation projects for a leading AI laboratory. In this fully remote contractor role, you will review, evaluate, and annotate AI-generated outputs while ensuring exceptional accuracy and consistency.

This opportunity is ideal for professionals who excel at following detailed instructions, identifying subtle errors, and maintaining high-quality work across repetitive, large-scale AI evaluation tasks.

Key Responsibilities

Evaluate AI Outputs

  • Review and assess AI-generated responses for quality and accuracy.
  • Annotate and evaluate outputs according to detailed project guidelines.
  • Ensure consistent application of evaluation criteria across all tasks.

Maintain High Quality Standards

  • Identify subtle errors, inconsistencies, and edge cases.
  • Detect ambiguous or unclear outputs that require further review.
  • Maintain exceptional accuracy across high-volume evaluation workflows.

Provide Structured Feedback

  • Deliver clear, objective, and actionable feedback to improve AI models.
  • Flag unclear instructions or ambiguous evaluation cases proactively.
  • Support continuous improvements to AI training datasets.

Follow Evaluation Guidelines

  • Apply complex instructions consistently without deviation.
  • Work independently while maintaining productivity and quality.
  • Contribute to long-term AI training initiatives through reliable evaluations.

Required Qualifications

  • Exceptional attention to detail and commitment to quality.

  • Previous experience as a:

    • AI Rater
    • Reviewer
    • Content Reviewer
    • Human Data Contributor
    • AI Evaluation Specialist
  • Ability to follow detailed guidelines consistently.

  • Strong written communication and reasoning skills.

  • Highly reliable with the ability to maintain accuracy across repetitive tasks.

  • Comfortable working independently in a remote environment.

Preferred Qualifications

  • Experience with:

    • AI data annotation
    • AI evaluation
    • Large-scale human data projects
    • Quality assurance (QA)
    • Operations
    • Trust & Safety
    • Rubric-based evaluations
  • Experience providing calibrated feedback using structured quality standards.

Compensation

  • $50 per hour
  • Weekly payments via Stripe or Wise

Work Arrangement

  • Fully Remote
  • Independent Contractor
  • Flexible schedule
  • Project-based engagement with potential extensions based on business needs and performance
Apply now

You will be redirected to the company's website to complete your application.

Apply now

Stay in the loop.

One email per week, 5 hand-picked roles.