Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.
Management Consulting Expert
Remote (Global)
$150 – $220/hr
July 14, 2026
Job description
Role Overview
Mercor is partnering with a leading AI research organization to engage experienced management consultants for a project focused on evaluating how well AI systems perform real-world consulting work. Rather than producing consulting deliverables yourself, you will define what excellent consulting looks like by creating evaluation criteria and scoring AI- and human-generated work with rigorous, evidence-based written feedback.
This is a flexible remote contract opportunity for experienced strategy and management consultants who want to apply their consulting expertise to improve frontier AI systems.
Key Responsibilities
Develop Evaluation Frameworks
- Design precise, task-specific grading criteria for consulting deliverables, including:
- Market analyses
- Strategy recommendations
- Financial models
- Client-ready presentation decks
- Implementation roadmaps
Evaluate Consulting Deliverables
- Review AI-generated and human-produced consulting work.
- Score submissions using structured evaluation rubrics.
- Provide detailed written justifications explaining strengths, weaknesses, and overall quality.
Ensure Evaluation Quality
- Apply objective, evidence-based judgment to produce consistent, reproducible evaluations.
- Incorporate structured feedback from senior reviewers and continuously improve evaluation quality.
Required Qualifications
5+ years of professional management consulting experience.
Experience at a top-tier strategy or management consulting firm such as:
- McKinsey & Company
- Bain & Company
- Boston Consulting Group (BCG)
- Or an equivalent consulting firm.
Strong expertise in:
- Structured problem solving
- Market sizing
- Financial modeling
- Business case development
- Executive-level presentation development
- Strategic recommendations
Exceptional written communication skills with the ability to clearly explain why work meets—or fails to meet—professional consulting standards.
Strong attention to detail and the ability to apply evaluation criteria consistently.
Preferred Qualifications
- Experience with AI evaluation, AI training, RLHF, or human-feedback projects.
Application Process
- Submit your resume or relevant professional background.
- Qualified applicants may be invited to complete a brief assessment or provide additional information.
- Selected candidates will proceed through Mercor's onboarding process before beginning project work.
Compensation
- $150–$220 USD per hour
Work Arrangement
- Remote
- Independent contractor
- Flexible schedule
- Project-based engagement
- Weekly payments via Stripe or Wise
Apply now
You will be redirected to the company's website to complete your application.
Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.
Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.
Toloka AI supports frontier model post-training by building domain-specific reinforcement learning environments, tasks, and evaluation frameworks designed by real practitioners.