About Mercor
Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will collaborate with top researchers and contribute to improving next-generation AI systems in your domain.
Role Overview
Mercor is collaborating with a leading AI lab to contract experienced professionals for an AI model evaluation project.
In this role, you will assess the quality, accuracy, and safety of AI-generated responses across Information Technology (IT) domains. Your work will directly improve the reliability of AI systems used in high-stakes environments where incorrect information can carry significant risks.
Key Responsibilities
- Write realistic prompts that reflect how professionals and users seek domain-specific guidance.
- Evaluate AI-generated responses for factual accuracy, correctness, and practical usefulness.
- Identify fabricated claims, incorrect references, or misleading reasoning.
- Score and rank multiple model responses using structured evaluation rubrics.
- Provide clear, evidence-based written justifications for evaluations.
Ideal Qualifications
- Master’s degree or higher in Computer Science, Information Systems, or a related field.
- Professional experience applying domain expertise in a practical or advisory capacity.
- Familiarity with industry standards, regulations, or best practices.
- Strong written communication and critical reasoning skills.
Work Details
- Commitment: Approximately 20 hours per week
- Work Type: Fully remote and asynchronous
Application Process
- Submit your resume to begin
- Complete a Model Response Evaluation assessment
Contract & Payment Terms
- Independent contractor engagement
- Flexible schedule — work on your own time
- Weekly payments via Stripe or Wise
- Projects may be extended, shortened, or concluded early based on performance and business needs
- No access to confidential or proprietary external data required
Additional Notes
- We consider all qualified applicants without regard to legally protected characteristics
- Reasonable accommodations are available upon request
- Unable to support H1-B or STEM OPT candidates at this time