Hindi Audio Generalist Evaluator Expert

Mercor is seeking a Hindi Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research laboratory.

In this role, you will perform transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. This short-term engagement is ideal for candidates with strong analytical or academic backgrounds who are fluent in Hindi and English and are based in the San Francisco Bay Area to support occasional in-person collaboration if required.

Key Responsibilities

Transcribe & Analyze Audio and Video

• Listen to, analyze, and transcribe Hindi audio and video content.
• Produce high-quality written outputs in Hindi with supporting work in English where required.
• Ensure accuracy, clarity, and adherence to project formatting and style guidelines.
• Capture linguistic nuances such as tone, register, regional vocabulary, dialectal differences, and contemporary Hindi usage.

Develop Evaluation Standards

• Create detailed evaluation rubrics and grading guidelines in Hindi and English.
• Define quality standards for consumer audio tasks.
• Document linguistic edge cases, grammar considerations, colloquialisms, code-switching (Hinglish), and dialect variations.

Evaluate AI Model Outputs

• Test language models using Hindi prompts.
• Assess outputs for:
  • Accuracy
  • Completeness
  • Fluency
  • Instruction following
• Provide structured feedback to improve model performance.

Quality Assurance

• Participate in QA and review cycles.
• Ensure datasets meet Mercor's quality standards before integration into production benchmarks.
• Collaborate with project leads to resolve ambiguities and improve evaluation processes.

Ideal Qualifications

• Native or near-native fluency in spoken and written Hindi.
• Professional proficiency in English.
• Strong writing, editing, analytical, and critical thinking skills.
• Ability to accurately transcribe and evaluate Hindi audio content.
• Must be based in the San Francisco Bay Area.
• Available to work 10–20 hours per week.

Preferred Qualifications

• College student or recent graduate.
• Background in linguistics, journalism, translation, localization, humanities, social sciences, or technical disciplines.
• Experience with transcription, annotation, localization, AI evaluation, or research workflows.
• Familiarity with Hinglish and regional Hindi language variation.
• Interest in AI and language technologies.

Application Process

• Complete a short AI-led interview (approximately 15 minutes).
• Qualified candidates will be invited to onboarding and project work.

Compensation

• $50 per hour

Contract & Payment Terms

• Independent contractor engagement
• Fully remote with occasional in-person collaboration if needed
• Flexible schedule
• Weekly payments via Stripe or Wise
• Projects may be extended, shortened, or concluded based on business needs and performance

Important Note

Mercor currently cannot support:

• H1-B candidates
• STEM OPT candidates

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier AI models using human expertise.

Contributors help improve next-generation AI systems through transcription, evaluation, and high-quality multilingual training data.