Machine Learning Engineer

📋 Job Overview

Help train the software engineers of the future. Cosine is building autonomous software-engineering systems for organisations where reliability, security and control matter most. Our Lumen Enterprise models are designed to work across real codebases. They need to understand unfamiliar architectures, make changes across multiple files, use tools, run tests and produce software that engineers can maintain and trust. We’re looking for a Machine Learning Engineer to help train and improve these models.

🏢 About Cosine

Cosine is building autonomous software-engineering systems for organisations where reliability, security and control matter most. Our Lumen Enterprise models are designed to work across real codebases, understanding unfamiliar architectures, making changes across multiple files, using tools, running tests and producing maintainable software.

🎯 The Role

You’ll work closely with research, infrastructure and product to answer questions such as: What should Lumen learn next? Which data and environments will teach it most effectively? How should we reward correct, maintainable and useful software? How do we train models to work across large repositories and long contexts? How do we know when a model is genuinely better for engineers? The models you train will write code, use development tools, run tests and receive feedback from real software-engineering environments. Your work will directly influence how Lumen reasons about code, navigates unfamiliar systems and improves through experience.

✅ Key Responsibilities

• Work across the full training lifecycle, running pre-training, supervised fine-tuning and reinforcement-learning experiments.
• Optimize training methods at scale on frontier models.
• Diagnose and fix performance bottlenecks and communication overhead in distributed training environments.
• Train models on long-context, multi-file and repository-level tasks.
• Develop RL and RLVR loops using signals from execution, tests, linters, static analysis and other engineering feedback.
• Investigate model failure modes such as poor planning, brittle fixes, reward hacking and regressions.
• Collaborate with infrastructure, platform and data teams.

📌 Required Qualifications

• Strong experience training deep-learning models in a production or research environment.
• Deep practical experience with PyTorch or a comparable framework.
• Experience with distributed training and multi-GPU systems (FSDP, Deepspeed, ...).
• Strong software-engineering fundamentals and a high bar for correctness and maintainability.
• Clear communication and the ability to explain technical trade-offs.
• Experience with large language models is important. You should have worked meaningfully on at least one of, pre-training, SFT, preference optimisation, RLHF or RLVR.

⭐ Desirable Experience

• Experience with mixture-of-experts (MoE) architectures.
• Experience with CUDA.
• Familiarity with vLLM, SGLang or other high-performance inference systems.
• Experience running jobs on SLURM, Kubernetes or managed GPU platforms.
• Experience with code-model benchmarks such as SWE-bench, HumanEval or internal repository-level evaluations.
• Contributions to open-source model-training, RL or evaluation tools.

🎁 Benefits

Salary and Equity options available for discussion.

🛂 Visa & Eligibility

No visa or eligibility information provided.