AI Engineer
Actively Hiring
Full-time Posted 9 months ago
Role overview
- check_circle Designing and implementing evaluation frameworks to measure LLM and agent performance across reasoning, accuracy, multi-turn dialogue, and tool usage
- check_circle Creating and open-sourcing benchmarks for evaluating LLM output on investment-research-specific tasks such as synthesis quality and citation grounding
- check_circle Building prompt refinement systems that learn from production signals and human feedback to improve reliability and performance
- check_circle Developing and maintaining agentic tooling including research assistants, deep research flows, and voice agents
- check_circle Integrating external APIs, search, speech-to-text, and text-to-speech technologies into production systems
- check_circle Prototyping lightweight voice agent frameworks with strong evaluation around latency, error recovery, and conversational flow
- check_circle Collaborating closely with research and product teams to productionize new prompting, retrieval, and multi-agent orchestration techniques
- check_circle Contributing meaningfully to product direction, prioritization, and long-term technical strategy
Benefits
- check_circle Gym membership
- check_circle In-office cook
- check_circle Summers working remotely by the beach
About the company
- check_circle Is based in NYC (in-person, 5 days per week)
- check_circle Has 5+ years of professional experience, with recent, hands-on work with LLMs
- check_circle Has strong opinions and enjoys contributing to product and architectural decisions
- check_circle Communicates clearly and is comfortable in a client-facing environment
- check_circle Can explain complex AI concepts to non-technical stakeholders and turn ideas into testable experiments
- check_circle Has built LLM systems end to end in a product-focused organization, from data and logging to evaluation and prompt optimization
- check_circle Has a strong bias to action and experience delivering complex projects with senior stakeholders
- check_circle Is excited to help grow a team and shape engineering culture
- check_circle Deep, recent experience working with LLMs and agentic systems is required
- check_circle Strong software engineering mindset rather than a purely research-focused background
- check_circle Either a software engineer who has transitioned into LLM systems, or an ML engineer who has spent the last few years heavily focused on LLMs
- check_circle Experience forming clear views on improving LLM output, reliability, and evaluation
- check_circle Leadership potential, with the opportunity to grow into a Head of AI role over time
Tags & Focus Areas
Fulltime Remote Ai Ai Engineer
About Mentor Talent Acquisition
We are on the lookout for an AI Engineer for a profitable, bootstrapped company building cutting-edge LLM-powered tools that transform how private market investors conduct research. Their platform is already trusted by several of the world’s top private equity and consulting firms and supports investment decisions involving billions of dollars annually.
Industry Fulltime