Python (ML-Focused) Quality Assurance Lead (QAL)

Job Summary

As a Python (ML-Focused) Quality Assurance Lead (QAL), you will oversee quality, consistency, and contributor performance across Python machine learning AI training projects.

This role involves reviewing AI-generated Python code, machine learning workflows, model explanations, and trainer/QA outputs while ensuring training data meets rigorous technical, statistical, and quality standards.

Your work will directly contribute to improving advanced AI systems by ensuring machine learning training content is accurate, reproducible, statistically sound, executable, and clearly explained.

Key Responsibilities

Quality Monitoring

• Spot-check Python and machine learning tasks
• Identify recurring quality issues and methodological gaps
• Provide actionable written feedback and escalate critical concerns

Code & Machine Learning Review

• Review and evaluate:

  • Python code
  • Machine learning pipelines
  • Data preprocessing workflows
  • Model training procedures
  • Evaluation logic
  • Debugging responses
  • Technical explanations

• Assess work for:

  • Code correctness
  • Statistical validity
  • Machine learning methodology
  • Reproducibility
  • Model evaluation quality
  • Data leakage risks
  • Maintainability
  • Instruction adherence

Trainer & QA Communication

• Communicate updates regarding:
  • Guidelines
  • Workflow changes
  • Python-specific standards
  • Machine learning review standards
  • Quality expectations

Contributor Support

• Answer questions involving:
  • Python syntax
  • Package usage
  • Data leakage
  • Validation strategies
  • Metrics selection
  • Statistical assumptions
  • Reproducibility
  • Notebooks
  • Rubric interpretation

Activation Management

• Follow up with inactive contributors
• Track engagement and participation
• Report contributor availability concerns

Documentation & Onboarding

• Create and maintain:

  • Style guides
  • Documentation
  • FAQs
  • Examples
  • Calibration tasks
  • Onboarding materials

• Conduct onboarding and training sessions for contributors

Risk & Quality Review

• Identify and flag:
  • Data leakage
  • Flawed methodologies
  • Invalid assumptions
  • Wrong metrics
  • Non-reproducible workflows
  • Hallucinated APIs
  • Misleading conclusions
  • Statistically invalid recommendations

Process Improvement

• Improve QA workflows and review processes
• Identify recurring quality gaps and implement corrective actions

Required Qualifications

• Bachelor's, Master's, or PhD in:
  • Computer Science
  • Machine Learning
  • Data Science
  • Statistics
  • Mathematics
  • Engineering
  • Related quantitative fields

or equivalent professional experience.

• Strong English communication skills

• Minimum 3 years of experience in:

  • Python development
  • Machine learning
  • Data science
  • ML engineering
  • Model evaluation
  • Research engineering
  • Technical review
  • ML education

• Strong understanding of:

  • Python data structures
  • Functions
  • Classes
  • Iterators
  • Comprehensions
  • Exception handling
  • Virtual environments
  • Package management
  • Testing
  • Debugging

• Strong understanding of:

  • Supervised learning
  • Unsupervised learning
  • Feature engineering
  • Train/test splits
  • Cross-validation
  • Model selection
  • Data leakage
  • Regression
  • Classification
  • Clustering
  • Evaluation metrics
  • Bias and variance
  • Regularization
  • Reproducibility

• Ability to identify:

  • Flawed methodologies
  • Wrong metrics
  • Data leakage
  • Invalid assumptions
  • Hallucinated APIs
  • Misleading conclusions
  • Non-reproducible code

Preferred Qualifications

• Experience with:

  • NumPy
  • pandas
  • scikit-learn
  • PyTorch
  • TensorFlow
  • Keras
  • XGBoost
  • LightGBM
  • Hugging Face

• Familiarity with:

  • Jupyter
  • MLflow
  • SQL
  • Docker
  • GitHub
  • CI/CD systems
  • Cloud ML platforms

• Experience leading:

  • Data scientists
  • ML engineers
  • Trainers
  • Reviewers
  • QA teams

• Experience with:

  • AI training
  • Data annotation
  • LLM evaluation
  • ML QA
  • Rubric-based technical review

Why Join

• Help improve leading AI systems through machine learning quality assurance
• Flexible remote schedule
• Weekly payments
• Referral rewards and community incentives
• Access to future opportunities through SME Careers' expert network

Selection Process

1. AI Interview
2. Domain-Specific Assessment
3. Recruiter Interview

Important Note

There is currently no active project for this role. Qualified candidates will be added to the expert network and contacted when relevant opportunities become available.