Applied AI/ML Engineer

📋 Job Overview

As a founding AI Engineer at Overtone, you will build and own the AI systems that power our core product experience, from conversational intelligence to matchmaking insight generation. You'll design prompting, evaluation, and model infrastructure that makes our AI reliable, trustworthy, and continuously improving. This is a hands-on, early-stage role: you'll run experiments, ship infrastructure, and build the tools that give the team real visibility into model behavior and quality.

🏢 About Overtone

Overtone, led by the founder and former CEO of Hinge, is building a new kind of dating service – one that cuts through the noise and helps daters find someone who truly resonates. We’re building a team who are excited to work together in person in our NYC office.

🎯 The Role

You will build and own the AI systems that power our core product experience, from conversational intelligence to matchmaking insight generation. You'll design prompting, evaluation, and model infrastructure that makes our AI reliable, trustworthy, and continuously improving. This is a hands-on, early-stage role: you'll run experiments, ship infrastructure, and build the tools that give the team real visibility into model behavior and quality.

✅ Key Responsibilities

• Design and maintain the prompts, structured outputs, and orchestration systems that power Overtone's AI-driven features.
• Build evaluation infrastructure to measure AI quality at scale using tools such as Braintrust, Langfuse, TensorZero, and Promptfoo.
• Run structured experiments across models, prompts, and configurations to optimize quality, cost, and latency.
• Build internal tools (dashboards, admin interfaces, debugging workflows) that give the team visibility into system behavior and model performance.
• Apply data science and lightweight modeling to improve match quality, identifying when traditional ML models or ranking systems outperform prompt-based approaches.
• Own the model serving layer: deploying models, managing inference infrastructure (API providers and self-hosted), model versioning, and cost/latency optimization in production.
• Identify opportunities to fine-tune open-source models using techniques such as SFT, DPO, and RLHF for cost and latency improvements.
• Design and maintain vector databases and memory architectures that enable personalization and context-aware experiences.
• Ensure systems are fair, unbiased, and auditable, building evaluation pipelines and human-in-the-loop processes where appropriate.
• Work closely with product, engineering, and research to bring AI capabilities into real user-facing features with scalability and reliability in mind.

🎁 Benefits

Salary of $200k to $300k. This position will receive meaningful stock options with considerable upside potential. We provide standard benefits such as health insurance, dental insurance, etc.

🛂 Visa & Eligibility

No visa or eligibility information provided.