Founding ML Engineer

📋 Job Overview

Base Compute is an AI inference lab. Our mission is to bring AGI on device. We believe in a world where everyone has access to intelligence: fast, private and always available on your device. We’re building the infrastructure for the next generation of on-device AI, from silicon-level optimizations to distributed inference systems. We’re working on hard problems at the intersection of inference efficiency, model intelligence and autonomous research.

🏢 About Base Compute

Application About Us Base Compute is an AI inference lab. Our mission is to bring AGI on device. We believe in a world where everyone has access to intelligence: fast, private and always available on your device. We’re building the infrastructure for the next generation of on-device AI, from silicon-level optimizations to distributed inference systems. We’re working on hard problems at the intersection of inference efficiency, model intelligence and autonomous research.

🎯 The Role

We’re looking for a Founding ML Engineer to work at the frontier of on-device AI. This role is for someone who lives at the intersection of systems engineering and machine learning, turning state-of-the-art research into hyper-optimized, production-ready infrastructure. You’ll have significant ownership over our entire inference stack and direct influence on the technical bets the company makes.

✅ Key Responsibilities

• Inference engine development: Building and scaling our custom inference engine, handling everything from weight loading and KV-cache management to efficient request scheduling
• Cross-platform silicon optimization: Writing and tuning custom kernels and leveraging hardware-specific instructions to squeeze maximum performance out of diverse architectures, including Apple Silicon, NVIDIA, AMD, Snapdragon, and other edge platforms
• Systems architecture: Developing robust, low-latency serving runtimes in C++ to manage model routing, continuous batching, and novel decoding strategies under strict thermal and memory constraints
• Performance profiling: Identifying and eliminating bottlenecks across the entire stack, from memory bandwidth ceilings to kernel interleaving

📌 Required Qualifications

• 3+ years of experience in ML engineering or systems programming (Rust, C/C++), with a strong track record of building performance-critical software
• Expertise in GPU programming and hardware optimization across various platforms (CUDA, ROCm, Metal, Triton, or similar)
• Solid understanding of modern LLM architectures, including parsing formats and implementing optimization techniques (quantization, speculative decoding, etc.)
• A strong sense of ownership and autonomy: the ability to take ambiguous architectural challenges and drive them from research translation directly into production-ready infrastructure
• Good communication: the ability to explain complex architectural decisions simply, give honest feedback and document systems cleanly

⭐ Desirable Experience

• Familiarity with ML compilers (torch.compile, custom operators)
• Experience with low-precision inference (INT8/FP8/FP4)
• Knowledge of Edge LLMOps

🎁 Benefits

Founding team equity and strong base salary
Direct influence on technical direction: your ideas will shape the roadmap
Work on genuinely hard problems that haven't been solved yet
Small team, fast iteration, low bureaucracy

🛂 Visa & Eligibility

The team is based in Melbourne and Berlin and works in-person from the office most days. We require strong written and spoken English, since the team collaborates across time zones.