Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
LLM Red Team Specialist — Failure Modes & Edge Cases
United States
$60 – $90/hr
July 25, 2026
Job description
Role Overview
Cincinnatus LLC is seeking experienced LLM Red Team Specialists to help develop next-generation evaluation benchmarks for frontier AI models. In this fully remote W-2 role, you will identify vulnerabilities, edge cases, and failure modes in advanced AI systems by designing adversarial evaluation tasks and analyzing where models produce incorrect or unreliable outputs.
Working alongside researchers at a leading AI lab, you will transform real-world AI weaknesses into rigorous benchmark tasks that improve the evaluation and reliability of frontier language models.
Key Responsibilities
Identify AI Failure Modes
- Probe frontier AI models to uncover vulnerabilities, edge cases, and hidden failure modes.
- Evaluate model behavior across coding, machine learning, and analytical reasoning tasks.
- Identify situations where models produce plausible but incorrect outputs.
Design Adversarial Benchmarks
- Create realistic, multi-step evaluation tasks based on discovered model weaknesses.
- Develop benchmarks that are challenging for AI systems while remaining fair and objectively gradable.
- Strengthen evaluation datasets by incorporating newly identified failure patterns.
Document and Analyze Findings
- Produce clear, reproducible documentation of discovered vulnerabilities.
- Support findings with evidence, experimental results, and reproducible testing steps.
- Analyze model performance and communicate recommendations to researchers.
Collaborate with Research Teams
- Work with benchmark authors to eliminate grading gaps and unintended shortcuts.
- Share insights with AI researchers and fellow specialists to improve benchmark quality.
- Help refine evaluation methodologies for frontier AI systems.
Required Qualifications
- Master's degree or PhD in:
- STEM discipline
- Or equivalent practical experience in a research-intensive field involving coding and data analysis
- At least 1 year of experience in:
- AI Evaluation
- Research Engineering
- Security Research
- Machine Learning Research
- Demonstrated experience with:
- LLM red teaming
- Adversarial testing
- AI vulnerability assessment
- Model failure analysis
- Strong proficiency with:
- Python
- Git
- Strong understanding of:
- Large Language Models (LLMs)
- Model capabilities and limitations
- AI evaluation methodologies
- Excellent written communication skills.
- High attention to detail and ability to work independently.
- Available to work approximately 35 hours per week.
Preferred Qualifications
- Experience with:
- AI model evaluation
- AI training
- Benchmark development
- Task authoring
- Experience identifying security or reliability issues in AI or machine learning systems.
Compensation
- $60–$90 per hour
- Full-time W-2 employment
Work Arrangement
- Fully Remote (United States)
- W-2 Contingent Role
- Approximately 35 hours per week
- Opportunity to work with a leading AI laboratory through Cincinnatus LLC
Apply now
You will be redirected to the company's website to complete your application.
Mercor is seeking bilingual AI Safety Experts fluent in both English and Punjabi to help evaluate and strengthen the safety of frontier AI systems. This role focuses on red teaming AI models by identifying vulnerabilities, testing misuse scenarios, and generating high-quality…
Evaluate the safety, quality, and alignment of frontier AI models by reviewing policy-sensitive content, applying AI safety standards, identifying model failures, and helping improve next-generation AI systems.
Help secure frontier AI models by designing adversarial prompts, identifying jailbreaks and safety vulnerabilities, evaluating AI behavior across high-risk domains, and improving AI alignment through structured red-team assessments.