Remote contract opportunity for experienced software engineers to design LLM prompts, evaluate AI model outputs, and contribute to rubric-based software engineering assessment projects for frontier AI systems.
Coding Expert (US & International)
Job description
HumanSignal is seeking Software Engineering Experts to contribute to advanced AI training, code evaluation, and annotation projects focused on improving AI performance in software engineering and technical problem-solving.
This role involves reviewing, annotating, validating, and evaluating code across multiple programming languages, frameworks, and software engineering domains.
Contributors will work on projects involving:
- Code correctness evaluation
- Algorithm analysis
- Software architecture
- AI-generated code validation
- Bug detection
- Security analysis
- Performance optimization
About HumanSignal
HumanSignal builds infrastructure and tooling for AI training, evaluation, and data curation workflows.
Its open-source platform, Label Studio, is widely used for:
- Data labeling
- AI evaluation
- Multi-modal annotation
- Agent evaluation workflows
The platform supports:
- Foundation model training
- Agent evaluation
- Continuous feedback pipelines
- AI system validation
HumanSignal supports hundreds of millions of labeled samples and is used by AI teams globally.
Responsibilities
Review and evaluate code for:
- Correctness
- Efficiency
- Best practices
- Maintainability
Annotate:
- Code logic
- Algorithms
- Architectural decisions
- Implementation approaches
Validate AI-generated code across multiple programming languages and frameworks
Identify:
- Bugs
- Security vulnerabilities
- Performance bottlenecks
- Structural issues
Provide detailed technical feedback on:
- Code quality
- Documentation
- Software architecture
- Engineering trade-offs
Work on problems involving:
- Algorithms
- Data structures
- System design
- Software architecture
Required Qualifications
Education
Master’s degree required in:
- Computer Science
- Software Engineering
- Computer Engineering
- Computational Science
- Information Science
- Related technical discipline
PhD strongly preferred
Technical Qualifications
Deep understanding of:
- Software engineering principles
- System architecture
- Development methodologies
Proficiency in multiple programming languages
Strong:
- Analytical thinking
- Problem-solving abilities
- Attention to detail
Ability to clearly explain technical reasoning and trade-offs
Experience with:
- Version control systems
- Testing frameworks
- Developer tooling
- Software development workflows
Ability to work independently and meet project deadlines
Work Details
- Fully remote project-based work
- Flexible scheduling
- Variable hours depending on project availability
- Ongoing training and calibration provided
- Quality-focused review environment with continuous feedback
Ideal Candidate Profile
Ideal candidates may include:
- Active software engineers
- Researchers
- Professors
- Technical experts
- Recent CS or engineering graduates
Strong GitHub profiles, technical portfolios, research backgrounds, or industry software engineering experience are highly valued.
Application Process
Applicants should be prepared to:
- Submit a CV or resume
- Provide proof of advanced degree qualifications
- Complete a coding assessment
- Participate in a technical qualification review
You will be redirected to the company's website to complete your application.