Help shape the future of AI by evaluating how well AI assistants understand users, use personal context, and deliver helpful conversations.
Task & Evaluation Specialist
Remote (Global)
$60/hr
September 2, 2026
Job description
What this role does Design realistic, difficult shopping-related tasks (e.g. meal planning, restocking, gifting, travel prep) that stress-test AI models, then build the rubrics used to grade how well a model performs them.
Who we're looking for
- Subject-matter expertise in retail, e-commerce, consumer shopping behavior, or a related domain (nutrition, logistics, home/office management, etc.)
- Strong analytical and writing skills — able to define precise, low-variance grading criteria
- Comfortable working with AI outputs and evaluating them critically and objectively
- Prior experience in eval design, QA, tutoring, or curriculum/rubric design is a plus
Core responsibilities
- Author realistic shopping tasks with a clear, concrete end goal
- Build a grading rubric/verifier for each task that two independent reviewers would score consistently
- Review an AI agent's attempt at the task and grade it against the rubric
- Document where the AI succeeded or failed, with clear reasoning
Work style
- Independent, deadline-driven work
- High attention to detail and consistency
- Comfortable with ambiguity — tasks are self-directed with light guidelines
Apply now
Opens verita-ai.com in a new tab. If they ask how you found the role, mention RemoWork.
Turing’s mission is to accelerate superintelligence to drive real economic progress. Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality data, evaluations, and reinforcement learning environments that improve model capabilities in coding,…
Turing’s mission is to accelerate superintelligence to drive real economic progress. Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality data, evaluations, and reinforcement learning environments that improve model capabilities in coding,…
Opportunity Snapshot Pay: $25-45/hour Time per task: ~1.5 hours per run (evidence and scoring included) Project kickoff: Monday, October 12 Onboarding: Today (Fri), Saturday, or Sunday, so you’re ready to start Monday October 10th You’ll need: An active personal popular AI…