About Mercor
Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will collaborate with top researchers and contribute to improving next-generation AI systems in your domain.
Role Overview
Mercor is seeking experienced Software Engineers to evaluate AI-powered CLI coding agents on real-world infrastructure debugging tasks.
In this role, you will work on complex systems issues within containerized environments, using AI tools to diagnose and resolve failures while benchmarking their performance. Your work will directly influence the development of next-generation AI coding systems.
Key Responsibilities
- Solve infrastructure debugging tasks using AI-powered CLI coding agents
- Diagnose broken systems inside Docker containers (databases, networking, pipelines, security)
- Write bash scripts to fix root causes and ensure system stability
- Compare and rank AI agents based on performance and approach
- Analyze outputs and provide structured feedback on AI behavior
Ideal Qualifications
- 3+ years of software engineering experience
- Strong bash/shell scripting skills
- Experience with Docker and containerized environments
- Systems debugging experience (PostgreSQL, MySQL, Redis, nginx, TLS, systemd, logs)
- Familiarity with version control tools (Git, PR workflows)
- Experience with AI coding tools (Copilot, Cursor, Claude, etc.) is a plus
Work Details
- Duration: 1–2 weeks
- Commitment: 15–25 hours per week (flexible up to 40 hours)
- Fully remote and flexible
Compensation
- Pay Range: $75–$80 per hour
Application Process
- Submit your resume
- Complete a short AI interview (~15 minutes)
- Receive follow-up with onboarding details
Contract & Payment Terms
- Independent contractor engagement
- Flexible schedule — work on your own time
- Weekly payments via Stripe or Wise
- Projects may be extended or concluded based on performance
Additional Notes
- Work involves real-world infrastructure debugging scenarios
- Opportunity to influence AI coding tools and developer workflows
- Potential promotion to reviewer roles based on performance