AI Quality Analyst - Portuguese (Portugal)

About Turing

Turing is a San Francisco-based research accelerator supporting frontier AI labs and enterprises deploying advanced AI systems.

The company works across:

• AI research acceleration
• High-quality training pipelines
• AI evaluation systems
• Enterprise AI deployment

Turing collaborates with leading AI organizations on projects involving reasoning, multilinguality, personalization, and advanced AI agents.

Job Summary

Turing is seeking an AI Quality Analyst (Portuguese - Portugal) to evaluate personalized AI interactions for Gemini.

This role focuses on assessing how effectively AI models use personal context from:

• Gemini conversations
• Gmail
• Google Search
• YouTube activity

to produce relevant, grounded, and helpful responses.

Contributors will combine:

• Creative prompt design
• Analytical evaluation
• AI quality assessment
• Personalization testing

to improve next-generation AI systems.

Key Responsibilities

• Design and execute multi-turn conversational prompts using personal context and experiences

• Evaluate AI responses for:

  • Grounding accuracy
  • Personalization quality
  • Natural integration
  • Overall helpfulness

• Identify issues such as:

  • Incorrect personalization
  • Hallucinations
  • Weak inferences
  • Overnarration
  • Forced contextual connections

• Conduct side-by-side (SxS) comparisons of AI model responses

• Stack-rank outputs based on:

  • Usefulness
  • Naturalness
  • Relevance
  • User experience quality

• Write structured rationales referencing specific conversation turns

• Verify debug information to confirm proper use of:

  • Chat summaries
  • Personal data sources
  • Context retrieval systems

• Maintain strict data hygiene by removing evaluation conversations after testing

Required Skills & Qualifications

• High proficiency in:

  • Portuguese (Portugal)
  • Written Portuguese
  • Reading comprehension

• Willingness to use a personal Google account with enabled personalization features

• Strong analytical reasoning and AI evaluation skills

• Experience designing:

  • Multi-turn prompts
  • Personalized AI evaluation scenarios

• Ability to identify:

  • Hallucinations
  • Weak personalization
  • Integration failures
  • Subtle response quality differences

• Excellent:

  • Written communication
  • Annotation skills
  • Evaluation rationale writing

• Strong attention to detail

• Ability to work independently in remote environments

• Reliable desktop/laptop setup and internet connection

Preferred Qualifications

• Degree or equivalent experience in:

  • Policy
  • Law
  • Ethics
  • Linguistics
  • Journalism
  • Computer Science
  • Analytical disciplines

• Experience in:

  • Data annotation
  • AI evaluation
  • Content moderation
  • AI quality assurance

Contract Details

• Contractor engagement

• Duration:

  • 3 months

• Commitment:

  • Minimum 4 hours daily
  • Up to 40 hours weekly
  • Minimum 4-hour overlap with PST timezone

• Compensation:

  • $15/hour

Evaluation Process

• Shortlisted candidates receive a Job Interest Form
• Selected applicants complete an assessment within 24 hours
• Successful candidates proceed to pre-onboarding discussions