Mathematics Model Prompt Evaluator

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems while collaborating with top researchers and shaping next-generation AI systems.

Role Overview

We are seeking expert mathematicians to author and verify high-quality, open-ended prompts for AI model evaluation.

In this role, you will craft and review challenging mathematical problems across core subdomains, assess AI reasoning quality, and help establish rigorous evaluation standards for frontier language models.

You will be assigned one of two task types:

Authoring Task

• Create 5 original, open-ended prompts from your assigned subdomain.
• Cover varying difficulty levels (undergraduate, advanced undergraduate, graduate/professional).
• Ensure prompts require human judgment to evaluate AI responses (e.g., proof construction, multi-step reasoning).

Verification Task

• Review 5 authored prompts for clarity, scope alignment, difficulty accuracy, and uniqueness.
• Edit prompts and difficulty ratings where necessary.

Mathematics Subdomains Covered

• Probability & Statistics
• Algebra (including Linear Algebra)
• Ordinary & Partial Differential Equations / Dynamical Systems
• Geometry
• Graph Theory
• Number Theory

Key Responsibilities

• Author clear, unambiguous, open-ended mathematical prompts.
• Ensure prompts elicit evaluable AI responses with meaningful reasoning depth.
• Verify prompts align with assigned subdomains and correct difficulty levels.
• Ensure all prompts within a task are distinct and varied in complexity.
• Apply expert judgment to assess mathematical rigor and reasoning requirements.
• Edit prompts and difficulty assignments where standards are not met.

Ideal Qualifications

• Master’s degree or higher in Mathematics, Applied Mathematics, Statistics, or a related field.
• 2–6 years of professional or research experience in a quantitative field.
• Strong command of graduate-level mathematics, including proof writing and formal reasoning.
• Experience in academic research, math competition design, or quantitative industry roles is a plus.
• Excellent written English with the ability to craft precise technical questions.

Work Details

• Commitment: 10+ hours per week
• Work Type: Asynchronous, fully remote

Contract & Payment Terms

• Independent contractor engagement.
• Flexible schedule; work on your own time.
• Weekly payments via Stripe or Wise based on completed work.
• Projects may be extended, shortened, or concluded early based on performance and business needs.
• No access to confidential or proprietary external data required.

Additional Notes

• We consider all qualified applicants without regard to legally protected characteristics.
• Reasonable accommodations are available upon request.
• Unable to support H1-B or STEM OPT candidates at this time.