Red-Teaming Quality Assurance Lead (QAL)

Job Summary

As a Red-Teaming Quality Assurance Lead (QAL), you will oversee quality, consistency, and contributor performance across AI safety and red-teaming projects.

This role involves reviewing AI-generated safety evaluations, adversarial prompts, risk assessments, and trainer/QA outputs while ensuring training data meets rigorous quality, safety, and policy standards.

Your work will directly contribute to improving advanced AI systems by ensuring safety training content is realistic, nuanced, policy-aligned, well-documented, and effective at identifying model vulnerabilities.

Key Responsibilities

Quality Monitoring

• Spot-check red-teaming tasks and QA outputs
• Identify recurring quality issues and evaluation gaps
• Provide actionable written feedback and escalate critical concerns

Safety & Red-Team Review

• Review and evaluate adversarial prompts, model responses, risk classifications, safety analyses, policy interpretations, vulnerability reports, and evaluation reasoning
• Assess work for risk identification, adversarial effectiveness, policy alignment, safety taxonomy accuracy, scenario realism, vulnerability coverage, clarity, and instruction adherence

Trainer & QA Communication

• Communicate guidelines, workflow changes, safety review standards, red-teaming methodologies, and quality expectations

Contributor Support

• Answer questions involving risk categorization, adversarial testing strategies, policy boundaries, severity assessment, edge cases, safety taxonomies, and rubric interpretation

Activation Management

• Follow up with inactive contributors, track engagement and participation, report contributor availability concerns

Documentation & Onboarding

• Create and maintain style guides, documentation, FAQs, examples, calibration tasks, and onboarding materials
• Conduct onboarding and training sessions for contributors

Risk & Safety Review

• Identify and flag weak adversarial designs, unrealistic attack scenarios, policy inconsistencies, poor risk categorization, unsafe outputs, incomplete vulnerability testing, low-quality evaluations, and insufficient documentation

Process Improvement

• Improve QA workflows and review processes
• Identify recurring quality gaps and implement corrective actions

Required Qualifications

• Bachelor's, Master's, or equivalent professional experience in Computer Science, Cybersecurity, AI Safety, Trust & Safety, Public Policy, Psychology, Linguistics, Law, Security Studies, Risk Analysis, or related fields
• Strong English communication skills
• Minimum 3 years of experience in AI Safety, Red Teaming, Cybersecurity, Trust & Safety, Content Policy, Risk Analysis, Adversarial Testing, Model Evaluation, or Content Moderation
• Strong understanding of AI risk categories, adversarial prompting, jailbreak techniques, harmful-content taxonomies, misuse scenarios, policy interpretation, model behavior, and safety evaluation principles
• Ability to identify weak adversarial testing, unrealistic scenarios, policy misinterpretations, unsafe outputs, poor vulnerability coverage, weak risk analysis, and incomplete evaluations

Preferred Qualifications

• Experience with prompt injection testing, social engineering risk analysis, cybersecurity evaluations, content moderation systems, and AI safety research
• Familiarity with privacy risk analysis, fraud prevention, misinformation evaluation, bias analysis, model refusal behavior, and safety benchmark development
• Experience with AI training, data annotation, LLM evaluation, safety evaluations, policy QA, and rubric-based review

Why Join

• Help improve leading AI systems through safety and red-teaming quality assurance
• Flexible remote schedule
• Weekly payments
• Referral rewards and community incentives
• Access to future opportunities through SME Careers' expert network

Selection Process

1. AI Interview
2. Domain-Specific Assessment
3. Recruiter Interview

Important Note

There is currently no active project for this role. Qualified candidates will be added to the expert network and contacted when relevant opportunities become available.