Senior Machine Learning Engineer, Jockey Core

📋 Job Overview

TwelveLabs builds the intelligence layer to change how machines understand video. Our multimodal AI models understand video across sight, sound, and motion, powering production-scale AI workloads across media, entertainment, sports, security, and government. We have raised over $210 million from top investors and are a global company with offices in San Francisco, Seoul, New York, and London. We believe diverse experiences make our products stronger and are looking for driven individuals to build technology that understands the world's complexity.

🏢 About Twelve Labs

TwelveLabs builds the intelligence layer to change that. Our multimodal AI models understand video the way humans do — across sight, sound, and motion — and power production-scale AI workloads across media, entertainment, sports, security, and government. We have raised more than $210 million from NEA, Radical Ventures, Amazon, NVIDIA, Snowflake, Databricks, Index Ventures, NAVER Ventures, Korea Investment Partners, Quadrille Capital, Red Bull Ventures, and AI pioneers including Fei-Fei Li, Silvio Savarese, and Alexandr Wang. We are a global company, headquartered in San Francisco with offices in Seoul, New York, and London, and employees around the world. We believe the differences in our cultural, educational, and life experiences make our products stronger. Building technology that understands the world in all its complexity requires people who see it from every angle. We are looking for individuals who are driven by hard problems and want their work to matter. Come build it with us!

🎯 The Role

This role leads serving engineering for Jockey Core — from engine selection through production scale-out on Blackwell. You will build benchmarks and load tests that replay the agent's real traffic, apply inference optimization to hit cost and latency targets, build cost models and drive production hardening, collaborate to ensure model-efficiency gains land as real serving wins, and explore AI-assisted development tools to improve productivity.

✅ Key Responsibilities

• Lead serving engineering for Jockey Core — from engine selection through production scale-out on Blackwell.
• Build benchmarks and load tests that replay the agent's real traffic (short prompts, frequent round-trips, long tool-output contexts), measuring TTFT and inter-token latency separately.
• Apply inference optimization — quantization, batching/scheduling, disaggregated prefill/decode, speculative decoding — to hit cost and latency targets.
• Build cost models and drive production hardening — autoscaling, capacity planning, observability, failover — through rollback-safe rollouts.
• Collaborate so model-efficiency gains land as real serving wins, and set the serving technical bar through design review.
• Explore and adopt AI-assisted development tools (Claude, Gemini, GPT) to improve productivity.

📌 Required Qualifications

• Significant experience serving and optimizing large-scale LLM inference in production (vLLM, TensorRT-LLM, SGLang, or similar), across techniques like batching/scheduling, quantization, disaggregated prefill/decode, and speculative decoding.
• Experience designing and operating large-scale distributed systems in high-performance GPU environments.
• A track record of driving ambiguous technical decisions with measured latency/throughput/cost tradeoffs.

⭐ Desirable Experience

• Deep expertise in multimodal systems with high instruction-following capability and complex, hierarchically structured outputs.
• Experience with training infrastructure from pre-training to RL, temporal segmentation and structured metadata extraction, large-scale inference and serving systems, data-curation and evaluation pipelines.
• Experience using advanced compute resources, including NVIDIA B300s, to accelerate the research-to-production cycle.

🎁 Benefits

Competitive

🛂 Visa & Eligibility

Not specified