Polish Audio Generalist Evaluator Expert

About Mercor

Mercor partners with leading AI labs and enterprises to train frontier models using human expertise. You will work on projects that focus on training and enhancing AI systems while collaborating with top researchers and shaping next-generation AI systems.

Role Overview

Mercor is seeking a Polish Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project with a leading research lab.

In this role, you will work on transcription, annotation, and evaluation tasks that help train and benchmark advanced language models. This is a short-term, structured engagement ideal for candidates with strong analytical or academic backgrounds who are fluent in Polish and English.

Job Responsibilities

Transcribe and Optimize Audio & Video

• Listen to, analyze, and transcribe audio and video content in Polish.
• Produce high-quality written outputs in Polish, with supporting work in English where required.
• Ensure clarity, accuracy, and strict adherence to formatting and stylistic guidelines.
• Capture nuances such as tone, intent, formality, and regional variations.

Define and Document Evaluation Standards

• Establish clear expectations for high-quality responses in general consumer audio contexts.
• Develop evaluation rubrics and grading guidelines in Polish and English.
• Document standards to ensure consistency across reviewers and evaluations.
• Identify linguistic nuances, grammatical complexities, and edge cases specific to Polish.

Conduct Model Testing and Grading

• Run prompts through language models and assess generated outputs.
• Evaluate responses for accuracy, completeness, fluency, and clarity.
• Provide structured feedback to improve model performance.

Support Benchmarking and Quality Assurance

• Participate in QA and review cycles to maintain quality standards.
• Ensure datasets meet requirements before integration into benchmarks.
• Collaborate with project leads to refine task design and resolve ambiguities.

Minimum Qualifications

• Strong writing, editing, and critical thinking skills.
• Ability to work independently and manage time effectively.
• Native or near-native fluency in Polish and professional fluency in English.
• Ability to accurately transcribe and analyze Polish audio content.
• Availability of 10–20 hours per week.

Preferred Qualifications

• College students or recent graduates.
• Background in linguistics, humanities, social sciences, journalism, or technical disciplines.
• Experience with transcription, annotation, localization, evaluation, or research workflows.
• Familiarity with dialectal variations and contemporary usage.
• Interest in AI, language models, or applied research.

Application & Onboarding Process

• Complete a short AI-led interview (approximately 15 minutes).
• Selected candidates will be onboarded and begin project work.

Contract & Payment Terms

• Independent contractor engagement.
• Fully remote with flexible scheduling.
• Weekly payments via Stripe or Wise.
• Projects may be extended, shortened, or concluded early based on performance and needs.
• No access to confidential or proprietary external data required.

Additional Notes

• Location Requirement: United Kingdom only.
• We consider all qualified applicants without regard to legally protected characteristics.
• Reasonable accommodations are available upon request.
• Unable to support H1-B or STEM OPT candidates at this time.