Japanese Audio Generalist Evaluator Expert

Mercor is seeking a Japanese Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research lab.

This role focuses on transcription, annotation, evaluation, benchmarking, and quality assurance tasks that help train and assess advanced language models.

The position is ideal for candidates with strong analytical, writing, and language skills who can accurately translate spoken audio and video content into structured written outputs in Japanese and English.

Job Responsibilities

Transcribe & Optimize Audio and Video

• Listen to and analyze Japanese audio and video content

• Produce accurate transcriptions following project requirements

• Create high-quality written outputs in Japanese

• Provide supporting work in English when required

• Ensure strict adherence to:

  • Formatting guidelines
  • Style requirements
  • Project instructions

• Capture nuances including:

  • Tone
  • Intent
  • Formal and informal language
  • Honorific language (Keigo)
  • Regional dialects
  • Spoken Japanese variations

Define Evaluation Standards

• Establish expectations for high-quality model responses
• Create evaluation rubrics and grading guidelines in Japanese and English
• Document standards to ensure reviewer consistency
• Identify:
  • Linguistic nuances
  • Grammatical complexities
  • Cultural references
  • Colloquialisms
  • Regional dialect variations
  • Japanese language edge cases

Conduct Model Testing & Evaluation

• Run prompts through AI models

• Evaluate outputs for:

  • Accuracy
  • Completeness
  • Fluency
  • Instruction following

• Provide structured feedback to improve model performance

Support Benchmarking & Quality Assurance

• Participate in review and QA cycles
• Validate tasks, rubrics, and outputs before benchmark inclusion
• Collaborate with project leads to improve task design and resolve ambiguities

Minimum Qualifications

• Strong writing and editing skills

• Strong critical thinking abilities

• Ability to work independently and meet deadlines

• Native or near-native fluency in Japanese

• Professional fluency in English

• Strong familiarity with:

  • Spoken Japanese
  • Regional dialects
  • Keigo (honorific language)
  • Contemporary Japanese language usage

• Ability to accurately transcribe and analyze Japanese audio content

• Availability of 10–20 hours per week

Preferred Qualifications

• College student or recent graduate

• Background in:

  • Linguistics
  • Humanities
  • Social Sciences
  • Journalism
  • Translation
  • Localization
  • Technical disciplines

• Prior experience with:

  • Transcription
  • Annotation
  • Localization
  • Evaluation
  • Research workflows

• Familiarity with regional dialects across Japan

• Interest in AI, language models, and applied research

Application Process

1. Complete a short AI-led interview (approximately 15 minutes)
2. Selected candidates will be onboarded and invited to begin project work

Contract & Payment Terms

• Independent contractor engagement
• Fully remote work
• Flexible schedule
• Weekly payments through Stripe or Wise
• Projects may be extended, shortened, or concluded depending on business needs and performance

Important Note

Mercor currently cannot support:

• H1-B candidates
• STEM OPT candidates

About Mercor

Mercor partners with leading AI labs and enterprises to train and improve frontier AI systems using human expertise.

Contributors work directly on projects that help shape the next generation of AI technologies and language models.