Senior Applied ML Engineer | Kimchi (Model Router)

📋 Job Overview

Cast AI is seeking an Applied Machine Learning Engineer to join our Kimchi team, responsible for the Auto Router that decides which model serves each request. This is an applied-ML role focused on training, evaluating, and shipping models that make real-time decisions for model routing.

🏢 About Cast AI

Made by Cast AI — a Miami-headquartered, globally distributed team across 34 countries that hit unicorn status in January 2026 with a $50B+ backed investment pushing its valuation past $1 billion. We help companies run LLMs on their own Kubernetes clusters and run our own coding harness on any model you pick.

🎯 The Role

Every request gets a decision: which model serves it. This is an applied-ML seat, not an infrastructure one. You will own the model inside the router: the training data, the objective, the evaluation, and the iteration loop that ships a better router every cycle.

✅ Key Responsibilities

• Own the routing model end to end: features, objective, training, calibration, and release cadence.
• Build the evaluation that gates every change: production-parity benchmarks, router-vs-router comparisons, holdouts by task family, error bars on pass rates.
• Turn fleet data into training signal: per-request outcomes from production, benchmark rollouts, and override feedback.
• Model the objective the user actually experiences: task success × token cost × wall-clock speed under the real concurrency regime.
• Run the online loop: shadow, A/B, full rollout, drift monitoring, retrain.
• Push the frontier where it pays: calibrated decisions, abstention under uncertainty, speed-aware routing.
• Partner with platform engineers on instrumentation: the features and labels recorded per request.

📌 Required Qualifications

• You've spent 5+ years in applied ML or ML research, and you've trained, evaluated, and shipped models that make real-time decisions.
• Strong Python and SQL at scale: you can pull, clean, and interrogate millions of request-level records without waiting on anyone.
• You know the difference between a model that got better and an eval that got easier.
• You have real opinions about what LLM benchmarks measure.
• You know enough serving reality (TTFT, decode speed, batching, prefix caching, concurrency) to model the objective honestly.
• You instrument before you optimize, error bars and all.
• A wide mandate excites you more than it unsettles you.

⭐ Desirable Experience

• You've worked on LLM routing specifically: RouterBench-class evals, cost/quality/latency Pareto fronts.
• Calibration, abstention, or conformal methods in production decision systems.
• You've run online A/B loops with drift monitoring on a model that touches real money.

🎁 Benefits

• Competitive salary (depending on the level of experience).
• Enjoy a flexible, remote-first global environment.
• Collaborate with a global team of cloud experts and innovators.
• Equity options.
• Get quick feedback with a fast-paced workflow.
• Spend 10% of your work time on personal projects or self-improvement.
• Learning budget for professional and personal development.
• Annual hackathon to spark new ideas and strengthen team bonds.
• Team-building budget and company events.
• Equipment budget to ensure you have everything you need.
• Extra days off to help maintain a healthy work-life balance.

🛂 Visa & Eligibility

Cast AI does not provide any form of visa sponsorship/work permit.