English (New Zealand) Audio Generalist Evaluator Expert

About the Role

Mercor is seeking an English (New Zealand) Audio Generalist Evaluator Expert to contribute to a high-impact AI audio research project with a leading research laboratory.

In this role, you will work on transcription, annotation, evaluation, and quality assurance tasks that help train and benchmark advanced language models.

This is a structured short-term engagement designed for candidates with strong analytical, academic, linguistic, or research backgrounds who are fluent in New Zealand English.

Responsibilities

Transcribe and Optimize Audio & Video

• Listen to, analyze, and transcribe audio and video content in New Zealand English
• Produce high-quality written outputs in English
• Ensure accuracy, clarity, consistency, and compliance with formatting requirements
• Capture linguistic nuances including tone, intent, formal and informal register, regional expressions, Māori loanwords, and contemporary New Zealand English usage

Define and Document Evaluation Standards

• Establish standards for high-quality AI responses
• Develop evaluation rubrics, grading guidelines, and review standards
• Document linguistic edge cases and language-specific considerations
• Identify grammatical patterns, colloquialisms, cultural references, and regional language variations specific to New Zealand English

Conduct Model Testing and Grading

• Test AI-generated outputs
• Evaluate responses based on accuracy, completeness, fluency, instruction following, and clarity
• Provide structured feedback to improve model performance

Support Benchmarking and Quality Assurance

• Participate in QA reviews, benchmark creation, and evaluation cycles
• Ensure consistency before datasets are integrated into official benchmarks
• Collaborate with project teams to improve task design and resolve ambiguities

Minimum Qualifications

• Strong writing, editing, and critical thinking skills
• Ability to work independently and meet deadlines
• Native or near-native fluency in New Zealand English
• Strong familiarity with New Zealand English vocabulary, regional accents, Māori language influences, and contemporary usage
• Ability to accurately transcribe and analyze English-language audio content
• Availability of 10-20 hours per week
• Must be located in Singapore

Preferred Qualifications

• Background in Linguistics, Humanities, Social Sciences, Journalism, Translation, Localization, or Technical disciplines
• Experience with transcription, annotation, localization, AI evaluation, or research workflows
• Familiarity with differences between New Zealand English, Australian English, and other English varieties
• Interest in Artificial Intelligence, Language Models, or Applied Research

Application Process

1. Complete a short AI-led interview (approximately 15 minutes)
2. Successful candidates will be onboarded and invited to begin project work

Contract and Payment Terms

• Engaged as an independent contractor
• Fully remote — complete on your own schedule
• Payments are weekly via Stripe or Wise

About Mercor

Mercor partners with leading AI laboratories and enterprises to train frontier AI models using human expertise.

Contributors work directly on projects that improve advanced AI systems while collaborating with researchers and domain experts across multiple fields.