Back to remote jobs

Internet-Native Bilingual Evaluator Expert (Japanese)

Mercor

Bilingual LLM Evaluator Contractor · Part-time
Remote (Global) $30 – $40/hr June 19, 2026

Job description

Mercor is seeking Internet-Native Bilingual Evaluator Experts (Japanese–English) to contribute to advanced AI research focused on multilingual communication, internet culture, and contemporary digital language use.

This role is ideal for individuals who actively participate in both Japanese- and English-language online communities and understand how internet culture, slang, humor, memes, and social norms differ across languages and platforms.

Job Responsibilities

Evaluate Internet-Native Communication

  • Analyze content from:

    • Social media platforms
    • Online forums
    • Video platforms
    • Gaming communities
    • Digital communication channels
  • Assess AI-generated outputs for:

    • Linguistic quality
    • Cultural relevance
    • Authenticity
  • Evaluate:

    • Internet slang
    • Memes
    • Abbreviations
    • Emoji usage
    • Honorific language
    • Code-switching behavior

Develop Evaluation Standards

  • Create evaluation rubrics for bilingual internet-native communication
  • Document linguistic and cultural edge cases specific to Japanese and English online communities
  • Establish standards for:
    • Tone
    • Contextual understanding
    • Appropriateness
    • Cultural accuracy

Conduct Model Testing and Feedback

  • Test AI systems using bilingual prompts and real-world internet scenarios

  • Evaluate model outputs in:

    • Japanese
    • English
  • Provide structured feedback to improve multilingual AI performance

Support Quality Assurance

  • Participate in benchmark development
  • Contribute to QA reviews
  • Ensure consistency and quality across evaluation datasets

Minimum Qualifications

  • Native or near-native fluency in Japanese
  • Professional fluency in English
  • Deep familiarity with Japanese and English internet culture
  • Active participation in online communities

Familiarity With Platforms Such As

  • X (Twitter)
  • YouTube
  • Discord
  • Reddit
  • Twitch
  • Nico Nico
  • Gaming communities
  • Similar online ecosystems

Required Skills

  • Strong writing ability

  • Analytical thinking

  • Critical thinking

  • Understanding of:

    • Internet slang
    • Memes
    • Online etiquette
    • Internet culture
    • Digital communication norms
  • Availability of 10–20 hours per week

Preferred Qualifications

Background in:

  • Linguistics
  • Translation
  • Localization
  • Journalism
  • Social Media
  • Humanities
  • AI Evaluation

Additional experience with:

  • User-generated content evaluation
  • Online community moderation or analysis
  • Internet subcultures
  • Fandom communities
  • Gaming communities
  • Contemporary digital trends

Interest in:

  • Artificial Intelligence
  • Language Models
  • Multilingual Communication Research

Contract & Payment Terms

  • Independent contractor engagement
  • Fully remote work
  • Flexible schedule
  • Weekly payments through Stripe or Wise
  • Projects may be extended, shortened, or concluded based on business needs and performance

Important Note

Mercor currently cannot support:

  • H1-B candidates
  • STEM OPT candidates

About Mercor

Mercor partners with leading AI labs and enterprises to train and improve frontier AI systems using human expertise.

Contributors help shape the future of multilingual AI by improving how models understand internet-native communication across languages and cultures.

Apply now

You will be redirected to the company's website to complete your application.

Apply now

Stay in the loop.

One email per week, 5 hand-picked roles.