Russian Audio Generalist Evaluator Expert

Mercor is seeking a Russian Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research laboratory.

In this role, you will perform transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. This short-term engagement is ideal for candidates with strong analytical or academic backgrounds who are fluent in Russian and English and are based in the San Francisco Bay Area to support occasional in-person collaboration if required.

Key Responsibilities

Transcribe & Analyze Audio and Video

• Listen to, analyze, and transcribe Russian audio and video content.
• Produce high-quality written outputs in Russian with supporting work in English where required.
• Ensure accuracy, clarity, and adherence to project formatting and style guidelines.
• Capture linguistic nuances such as tone, register, regional vocabulary, dialectal differences, and contemporary Russian usage.

Develop Evaluation Standards

• Create detailed evaluation rubrics and grading guidelines in Russian and English.
• Define quality standards for consumer audio tasks.
• Document linguistic edge cases, grammar considerations, colloquialisms, and regional language variations.

Evaluate AI Model Outputs

• Test language models using Russian prompts.
• Assess outputs for:
  • Accuracy
  • Completeness
  • Fluency
  • Instruction following
• Provide structured feedback to improve model performance.

Quality Assurance

• Participate in QA and review cycles.
• Ensure datasets meet Mercor's quality standards before integration into production benchmarks.
• Collaborate with project leads to resolve ambiguities and improve evaluation processes.

Ideal Qualifications

• Native or near-native fluency in spoken and written Russian.
• Professional proficiency in English.
• Strong writing, editing, analytical, and critical thinking skills.
• Ability to accurately transcribe and evaluate Russian audio content.
• Must be based in the San Francisco Bay Area.
• Available to work 10–20 hours per week.

Preferred Qualifications

• College student or recent graduate.
• Background in linguistics, journalism, translation, localization, humanities, social sciences, or technical disciplines.
• Experience with transcription, annotation, localization, AI evaluation, or research workflows.
• Familiarity with regional variations of Russian and contemporary digital language usage.
• Interest in AI and language technologies.

Application Process

• Complete a short AI-led interview (approximately 15 minutes).
• Qualified candidates will be invited to onboarding and project work.

Compensation

• $50 per hour

Contract & Payment Terms

• Independent contractor engagement
• Fully remote with occasional in-person collaboration if needed
• Flexible schedule
• Weekly payments via Stripe or Wise
• Projects may be extended, shortened, or concluded based on business needs and performance

Important Note

Mercor currently cannot support:

• H1-B candidates
• STEM OPT candidates

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier AI models using human expertise.

Contributors help improve next-generation AI systems through transcription, evaluation, and high-quality multilingual training data.