AI Resume Builder
JOIN NOW Post Job
Mercor
AI

Data Scientist | Fully Remote

Mercor · · $208k - $249k

Actively hiring Posted 7 months ago

About The Job
Mercor
connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include
Benchmark
,
General Catalyst
,
Peter Thiel
,
Adam D'Angelo
,
Larry Summers
, and
Jack Dorsey
.

Position:
AI Task Evaluation & Statistical Analysis Specialist

Type:
Contract
Compensation:
$100–$120/hour
Location:
Remote
Role Responsibilities

  • Conduct comprehensive statistical failure analysis to identify patterns in AI agent failures across task components such as prompts, rubrics, and templates.
  • Perform root cause analysis to determine if failures are due to task design, rubric clarity, file complexity, or agent limitations.
  • Analyze performance variations across finance sub-domains, file types, and task categories to enhance understanding of AI model performance.
  • Create dashboards and reports to highlight failure clusters, edge cases, and improvement opportunities.
  • Recommend improvements to task design, rubric structure, and evaluation criteria based on statistical findings.
  • Present insights to data labeling experts and technical teams to foster collaboration and drive improvements.

Qualifications
Must-Have

  • Statistical Expertise: Strong foundation in statistical analysis, hypothesis testing, and pattern recognition.
  • Programming: Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis.
  • Data Analysis: Experience with exploratory data analysis and creating actionable insights from complex datasets.
  • AI/ML Familiarity: Understanding of LLM evaluation methods and quality metrics.
  • Tools: Comfortable working with Excel, data visualization tools (Tableau/Looker), and SQL.

Preferred

  • Experience with AI/ML model evaluation or quality assurance.
  • Background in finance or willingness to learn finance domain concepts.
  • Experience with multi-dimensional failure analysis.
  • Familiarity with benchmark datasets and evaluation frameworks.
  • 2-4 years of relevant experience.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
,

Tags & focus areas

Used for matching and alerts on DevFound
Parttime Remote Ai Data Science Generative Ai
Common Questions

Frequently asked questions

Quick answers about how DevFound's AI matching, resumes, and referrals work.

DevFound's AI Copilot ingests your profile, goals, and live job data to deliver curated matches in seconds. Every match includes a resume variant, suggested referrals, and interview prep so you can act immediately. The more feedback you provide, the sharper the Copilot becomes.

AI-led job searches shrink the hours spent sifting through boards and formatting resumes. DevFound pairs automation with your personal outreach, so you reserve energy for interviews and negotiation. Traditional networking still matters, but AI gives you a lift before you even send a message.

Modern AI roles expect comfort with production-grade code, data fluency, and practical ML tooling. The strongest candidates pair deep technical chops with storytelling—translating model impact to product, GTM, and exec partners. Continuous learning keeps you ahead as stacks evolve.

DevFound rewards active seekers. Keep your profile fresh, respond to match quality prompts, and enable alerts so you never miss a role. The AI prioritizes companies and teams that align with your feedback, accelerating both introductions and interview invites.

High-density tech hubs continue to host the deepest AI talent pools, yet distributed teams are catching up fast. Use DevFound filters to hone in on onsite, hybrid, or fully remote roles and watch openings expand across time zones.

DevFound aggregates thousands of remote AI openings and flags the nuances—core hours, async culture, and visa needs—up front. The Copilot also recommends how to position your distributed work experience so hiring managers know you can thrive on a remote team.

Still have questions?

Contact Support