The next-generation AI developer platform unifying the development and deployment of AI for the world.
Job Overview
Modular is seeking an AI Serving Platform Engineer to join our Cloud Inference team. We're on a mission to revolutionize AI infrastructure by systematically rebuilding the AI software stack from the ground up. This is a unique opportunity to work with industry leaders and experts building cutting-edge, modular infrastructure that simplifies AI development and deployment.
Key Responsibilities
Work with Product and partner engineering teams to design and ship new inference server features
Collaborate with our kernels and genAI teams to achieve SOTA performance at the serving layer
Help design and develop helm charts and cloud services for scaled LLM inference (intelligent routing, distributed kvcache management, disaggregated inference, etc.)
Required Qualifications
Experience working on high scale ML inference infrastructure (traditional AI or genAI)
Experience with kubernetes
Familiarity with HuggingFace API and workflows for using community models
Ability to create durable, reusable software tools and libraries that are leveraged across teams and functions
Experience in machine learning technologies and use cases
Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture
Strongly identifies with our core company cultural values
Preferred Qualifications
Experience building ML models in PyTorch
Familiarity with modern C++
Some experience with and interest in Mojo, our AI programming language
About Modular
At Modular, we're on a mission to revolutionize AI infrastructure by systematically rebuilding the AI software stack from the ground up. Our team, made up of industry leaders and experts, is building cutting-edge, modular infrastructure that simplifies AI development and deployment. By rethinking the complexities of AI systems, we're empowering everyone to unlock AI's full potential and tackle some of the world's most pressing challenges.
Benefits
Amazing Team. We are a progressive and agile team with some of the industry's best engineering and product leaders.
World-class Benefits. Premier insurance plans, up to 5% 401k matching, flexible paid time off, and more.
Please let Modular know that you found this role at devopsprojectshq.com as a way to support us, so we can keep providing you with awesome DevOps jobs.
Never miss a job
Join 2,000+ DevOps developers getting weekly alerts for remote and US/EU roles, Kubernetes, AWS, Terraform, filtered for your stack.
🔒 Need an IP to whitelist?
Get a dedicated static EU outbound IP for Banks, payments, EHRs, APIs, AI.
