Member of Technical Staff, Evals Platform

📋 Job Overview

Sage Care is a fast-growing, early-stage healthcare startup founded by exceptional leaders from Apple, Uber, Carbon Health and backed by top-tier venture capital (General Catalyst, Chelsea Clinton). With a strong customer pipeline, Sage Care is transforming healthcare by simplifying care navigation. Our platform makes it easier for patients to find the right doctor and helps providers focus on those who need them most through harnessing the latest AI innovations. Building on our successful collaborations with health systems across the U.S., we have expanded internationally to the MENA region. We are now partnering with health systems there to deploy our AI-powered care navigation platform.

🏢 About Sage Care

Sage Care is a fast-growing, early-stage healthcare startup founded by exceptional leaders from Apple, Uber, Carbon Health and backed by top-tier venture capital (General Catalyst, Chelsea Clinton). With a strong customer pipeline, Sage Care is transforming healthcare by simplifying care navigation. Our platform makes it easier for patients to find the right doctor and helps providers focus on those who need them most through harnessing the latest AI innovations. Building on our successful collaborations with health systems across the U.S., we have expanded internationally to the MENA region. We are now partnering with health systems there to deploy our AI-powered care navigation platform.

🎯 The Role

We’re hiring a Software Engineer to build the evaluations platform behind our AI agents. You’ll define how we measure quality, build the systems that make those measurements trustworthy, and turn production failures into improvements. You’ll work closely with engineering, research, product, and clinical stakeholders. We welcome senior and staff-level engineers; scope will reflect your experience. Healthcare experience is optional.

✅ Key Responsibilities

• Define metrics and curate datasets that capture quality, task completion, and safety across patient workflows.
• Build the evaluation platform, harnesses, and grading infrastructure to measure agent performance at scale.
• Develop verifiers and LLM-based evaluators for agent actions and outputs, calibrated against expert assessments.
• Analyze production conversations and execution traces, and turn failure modes into regression tests.
• Own the feedback loop from evaluation results to model, prompt, and product improvements.
• Establish evaluation gates that guide whether agent changes ship.
• Build reusable tools that let engineers and domain experts contribute evaluations and act on results.

📌 Required Qualifications

• 5+ years of software or ML engineering experience shipping production systems.
• You’ve built evaluation platforms or frameworks that other engineers depend on.
• You can design datasets, metrics, and evaluators—and explain why their results are trustworthy.
• Strong backend engineering skills and the ability to own ambiguous problems end to end.

⭐ Desirable Experience

• Experience with voice AI, agent simulation, or post-training.
• Healthcare experience.

🎁 Benefits

Base Salary $200K – $250K • Offers Equity. Compensation is determined by level of experience.