Back to remote jobs

JSON-fluent & Rubrics Expert

Mercor

Evaluation & Quality Contractor Initial sprint (extension possible)
United States $50 – $175/hr July 25, 2026

Job description

Role Overview

Mercor is seeking detail-oriented JSON & Rubrics Experts to support a high-priority AI evaluation project for one of the world's leading AI research organizations. In this fully remote contract role, you will evaluate AI model outputs by applying structured grading rubrics to JSON-formatted responses, ensuring accuracy, consistency, and adherence to detailed evaluation guidelines.

This is an accelerated weekend sprint requiring immediate availability, with the potential for project extensions based on business needs.

Key Responsibilities

Evaluate AI Outputs

  • Review and assess JSON-formatted AI model outputs.
  • Apply detailed grading rubrics to evaluate response quality and correctness.
  • Ensure evaluations are accurate, objective, and consistent.

Interpret Project Documentation

  • Read and understand written task descriptions.
  • Interpret detailed evaluation rubrics and grading criteria.
  • Analyze JSON structures and compare outputs against established standards.

Maintain Quality Standards

  • Complete evaluations without using AI assistance.
  • Deliver work within required turnaround times.
  • Maintain high attention to detail across large volumes of assessments.

Required Qualifications

  • Comfortable reading and interpreting JSON files.
  • Strong attention to detail and consistency.
  • Ability to understand and apply structured grading rubrics.
  • Available to begin immediately.
  • Able to dedicate 20–40 hours per week.
  • Capable of completing assigned work within 12–24-hour turnaround expectations.
  • Access to a desktop or laptop computer.
  • Based in the United States.

Preferred Qualifications

  • Previous experience evaluating AI model outputs.
  • Prior experience working with detailed grading rubrics.
  • Previous Mercor project experience.
  • Experience with structured data review or quality assurance.

Compensation

  • $50–$175 per hour
  • Weekly payments via Stripe or Wise

Work Arrangement

  • Fully Remote
  • Independent Contractor
  • Flexible schedule
  • Immediate start
  • Initial sprint running through July 27–28, with potential project extension
Apply now

You will be redirected to the company's website to complete your application.

Categories

Explore other opportunities in related specializations.

AI & Data Training Paid Research Participant Engineering & Development Data & Analytics Design & Creative Product & Project Management Marketing & Growth Sales & Business Development Customer Support & Success HR & People Operations Finance & Accounting Legal & Compliance Operations & Administration Healthcare & Clinical Science & Research Education & Training Writing & Content Translation & Localization Media & Entertainment Consulting & Strategy Nonprofit, Social Impact & Other Travel & Hospitality Sports & Fitness Fashion, Beauty & Lifestyle

Related jobs

View all

Frequently asked questions

Is Mercor legitimate?
Yes, Mercor is a legitimate company listed on RemoWork. We verify all companies on our platform. However, we always recommend doing your own research before sharing personal information or starting work. Check reviews from other users and visit the company's official website for the most up-to-date information.
How do I get started with Mercor?
To get started, visit Mercor's official website and look for their careers or sign-up page. Create an account, complete your profile with relevant skills and experience, and apply for available positions or projects. Some companies may require you to pass a qualification test or assessment before you can start working.
Does Mercor offer remote work?
Yes, Mercor offers remote work opportunities. Most tasks and positions can be completed from anywhere with a reliable internet connection and a computer or smartphone. Specific location requirements may vary by project or role, so check individual listings for details.
How does Mercor pay its workers?
Payment methods and schedules vary. Common payment options include PayPal, bank transfers, Payoneer, and other digital payment platforms. Visit Mercor's official website or check their payment/FAQ section for specific details about payment methods, minimum thresholds, and payout schedules.
What skills do I need to work with Mercor?
Required skills vary by role and project. Generally, you'll need a reliable internet connection, basic computer literacy, strong attention to detail, and good communication skills. Some positions may require specialized expertise, language proficiency, or technical qualifications. Check individual job listings for specific requirements.
What is Mercor's vetting process like?
Mercor uses an AI-powered vetting process that evaluates candidates through technical assessments, interviews, and skill verification. The platform matches vetted talent with companies looking for specific expertise. The process is designed to be efficient and identify top-tier professionals quickly.
What types of roles does Mercor offer?
Mercor connects professionals with AI engineering, software development, data science, machine learning, and other tech-focused roles. The platform works with startups and enterprises, offering both contract and full-time remote positions across various technical domains.
Is Mercor free for job seekers?
Yes, Mercor is free for job seekers and freelancers. The platform earns revenue from companies hiring through the network. Candidates can create profiles, get vetted, and apply for positions at no cost.
Apply now

Stay in the loop.

One email per week, 5 hand-picked roles.