Senior AI Interaction Evaluator (Codex / Claude Code) Contract | $50-200/hr | 10+ hrs/week | Project-based Roles open on a rolling basis - apply to join the talent bench and we’ll reach out when one matches.
Shopify & SQL Expert
Remote (Global)
$60 – $60/hr
July 21, 2026
Job description
Role Overview
Mercor is seeking a Shopify & SQL Expert to help improve next-generation AI systems by evaluating how Shopify's Sidekick AI assistant responds to merchant analytics and customer segmentation questions. In this fully remote hourly contract role, you will analyze real merchant conversations, identify SQL and filtering errors, and provide structured feedback that improves AI performance.
This opportunity is ideal for professionals with strong SQL skills and hands-on experience operating Shopify stores or working with Shopify analytics.
Key Responsibilities
Evaluate AI Responses
- Review multi-turn conversations between merchants and Shopify Sidekick.
- Analyze AI-generated responses alongside tool-call results and returned data.
- Identify where AI-generated SQL queries or segmentation filters are incorrect.
Analyze SQL Queries
- Determine the correct SQL query or segmentation filter.
- Evaluate queries involving WHERE and HAVING clauses, GROUP BY, ORDER BY and LIMIT, joins, and aggregations.
- Explain why an existing query or filter produced incorrect results.
Categorize Errors
- Classify failures using a detailed evaluation rubric.
- Identify issues such as incorrect metrics, wrong dimensions, invalid filters, unsupported reports, and incorrect segmentation logic.
Provide Structured Feedback
- Write concise, objective explanations describing what went wrong.
- Help improve AI reasoning for Shopify analytics and merchant support tasks.
- Maintain consistent evaluation quality across assigned conversations.
Required Qualifications
- Strong SQL skills with the ability to read and interpret queries, write SQL queries, debug incorrect queries, and explain query behaviour.
- Strong understanding of WHERE vs HAVING, GROUP BY, ORDER BY, LIMIT, joins, and aggregate functions.
Preferred Qualifications
- Hands-on experience operating a Shopify store.
- Experience using Shopify Sidekick.
- Familiarity with Shopify Analytics, ShopifyQL, Shopify data models, customer segmentation, sales and payments data, fulfillment reporting, and e-commerce analytics.
- Experience in e-commerce operations or analytics.
Compensation
- $60 per hour
- Weekly payments via Stripe or Wise
Work Arrangement
- Fully remote
- Independent contractor
- Approximately 5 to 10 hours per week
- Flexible schedule
- Project-based engagement with potential extensions based on business needs and performance
Apply now
You will be redirected to the company's website to complete your application.
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.