Inference Infrastructure Engineer, Serving

📋 Job Overview

We are a well-funded, early-stage AI lab focused on building the next generation of frontier multimodal AI models. Founded by former DeepMind researchers, including Andrew Dai, who was previously a leader on Gemini. Our team currently consists of 20 world-class scientists and engineers. We recently raised $55M in seed funding from Striker Ventures, Menlo Ventures, Altimeter Capital, and NVIDIA. We are tackling some of the hardest problems in artificial intelligence, and we are growing fast.

🏢 About Elorian AI

Elorian AI is an early-stage AI lab focused on building the next generation of frontier multimodal AI models. Founded by former DeepMind researchers, including Andrew Dai, who was previously a leader on Gemini. Our team currently consists of 20 world-class scientists and engineers. We recently raised $55M in seed funding from Striker Ventures, Menlo Ventures, Altimeter Capital, and NVIDIA.

🎯 The Role

We're looking for an infrastructure engineer to design, optimize, and scale the systems that serve our large multimodal models. Your work will make inference faster, more cost-effective, and more reliable, so our teams can focus on advancing model capabilities rather than managing bottlenecks. Our focus is on performant, efficient inference, both to power real-world applications and to accelerate research. This role owns the infrastructure that ensures every deployment and evaluation runs smoothly at scale for our visual foundation models.

✅ Key Responsibilities

• Build low-latency, high-throughput inference serving systems for our large multimodal models
• Design and implement techniques that improve latency, throughput, and efficiency, including quantization, batching, speculative decoding, and KV cache management
• Optimize our codebase and GPU fleet to fully utilize hardware FLOPs, bandwidth, and memory
• Implement multi-GPU and multi-node model parallelism for serving (tensor or pipeline parallel)
• Build autoscaling and load balancing for production ML services
• Establish standards for reliability, observability, and reproducibility across the inference stack
• Collaborate with researchers to enable high-performance inference for novel architectures

📌 Required Qualifications

• 3+ years of experience building low-latency, high-throughput inference serving systems for large models
• Strong knowledge of inference optimization techniques (quantization, batching, speculative decoding, KV cache management)
• Hands-on experience with serving frameworks such as vLLM, TensorRT-LLM, Triton, or SGLang
• Experience with multi-GPU/multi-node model parallelism for serving (tensor or pipeline parallel)
• Strong systems programming skills; C++/CUDA a plus alongside Python
• Experience with autoscaling and load balancing for production ML services
• A track record of GPU cost optimization at scale

⭐ Desirable Experience

• Experience serving multimodal (vision + language) models
• Contributions to open-source ML or systems infrastructure projects (e.g., vLLM, SGLang, TensorRT-LLM, Triton)
• A bias for action and comfort working across stacks and teams in an early-stage environment

🎁 Benefits

Compensation: Depending on background, skills, and experience, the expected annual base salary range for this position is $200,000 - $400,000 USD, plus equity and benefits. We offer health, dental, and vision benefits, unlimited PTO, paid parental leave, and relocation support as needed.

🛂 Visa & Eligibility

Visa sponsorship: We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.