Systems Generalist, GPT Infrastructure

Job Overview

We are seeking an experienced systems generalist who can work comfortably across the stack to help build an automated inference optimization platform. Given a workload, target hardware profile, compiler and runtime context, and a trusted verifier, the system runs durable optimization campaigns that generate, compile, execute, grade, and improve candidate kernels, runtime configurations, and serving-stack changes.

You will design both the OpenAI-hosted control plane and the partner-side software that evaluates candidates on real accelerator hardware. The product must keep long-running workflows reliable, make performance results reproducible, and maintain clear trust boundaries around sensitive model and hardware information.

This is a deeply cross-stack role, combining strong software engineering fundamentals with systems thinking and performance intuition. You will work closely with research, inference engineering, infrastructure, security, product, and strategic partners to turn a powerful research workflow into a scalable product.

Key Responsibilities

• Design, build, and operate durable APIs and control-plane services for multi-hour or multi-day optimization campaigns, including scheduling, retries, budgets, checkpoints, artifact lineage, and observability.


• Build secure partner-side runner and grader software that can compile, execute, verify, and benchmark candidate artifacts on third-party accelerator hardware.


• Integrate hardware profiles, ISA and toolchain context, compilers, runtimes, and inference-serving engines into a repeatable optimization workflow.


• Turn research prototypes into reliable product surfaces with clear contracts, debuggable failure modes, reproducible outputs, and excellent developer ergonomics.


• Develop correctness and performance evaluation systems spanning latency, throughput, memory use, utilization, and cost efficiency.


• Build artifact, provenance, and qualification workflows that make optimized kernels, binaries, configurations, and reports safe to review and deploy.


• Collaborate with Research, Inference Engineering, Infrastructure, Security, Product, and Strategic Partnerships to deliver production-ready solutions.


• Drive technical architecture and execution across ambiguous, cross-functional initiatives that connect OpenAI systems with partner environments.

Basic Qualifications

8+ years of professional software engineering experience building large-scale distributed systems, infrastructure platforms, or cloud services, or equivalent depth of experience.

Strong programming skills in one or more of C++, Python, Go, or Rust.

Experience designing and operating highly available backend systems, APIs, job orchestration systems, or durable workflows for production workloads.

Strong understanding of distributed systems, Linux, networking, storage,