Senior DevOps & Infrastructure Engineer

Job Overview

We are seeking a Senior DevOps & Infrastructure Engineer to join our team at HUD. In this role, you will be responsible for owning and evolving our infrastructure to ensure it is extremely performant, reliable, operationally excellent, and secure and compliant.

Key Responsibilities

• Own our AWS + EKS-based sandbox platform that runs Dockerized workloads for customers and internal teams.
• Optimize sandbox lifecycle end-to-end: provisioning, scheduling, image pulls, startup, execution, teardown, and caching.
• Design for massive parallelism while maintaining reliability, fairness, and predictable performance.
• Evolve our cluster architecture: node groups, autoscaling strategies, spot/on-demand mixes, scheduling policies, and workload isolation.
• Build safe-by-default patterns: quotas, resource limits, network policies, pod security, secrets management, and guardrails.
• Improve cluster resiliency and operational ergonomics (upgrades, rollouts, disaster recovery, fail-safes).
• Address infrastructure bottlenecks as we scale.
• Improve developer experience for internal teams: safer deploys, better CI/CD, smoother local/dev workflows, faster iteration.
• Provide architectural input and raise the infra maturity of the team via docs, patterns, and coaching.
• Interface with our backend/workers (Railway), frontend (Vercel/Next.js), and data (Supabase/Postgres) to ensure the whole system is cohesive.
• Establish “infra product metrics” and instrument everything: P50/P95/P99 sandbox startup times, queue times, job success rates, noisy-neighbor rates, image pull latencies, cluster saturation, and cost-per-run.
• Build benchmarking harnesses for sandboxes and workloads to track regressions and validate improvements.
• Treat efficiency as a first-class metric: optimize utilization without sacrificing latency or reliability.
• Implement gold-standard observability across logs/metrics/traces with actionable dashboards and alerting tied to SLOs.
• Create runbooks, incident processes, and postmortem culture that meaningfully improves the system each time.

Required Qualifications

• Deep AWS experience, including operating production systems at scale (networking, IAM, compute, storage, observability, cost).
• Strong Kubernetes/EKS experience: cluster design, workload isolation, autoscaling (cluster + pod), upgrades, reliability practices.
• Excellent Docker + container runtime knowledge: image optimization, build pipelines, caching strategies, and runtime security considerations.
• Systems-level competence: Linux fundamentals, networking, performance debugging, resource contention, concurrency basics.
• Infrastructure automation: strong ability to implement infrastructure as code (Terraform/CDK/CloudFormation) and repeatable environments.
• Observability expertise: metrics/logging/tracing design, SLOs/SLIs, alerting that avoids noise and catches real issues.
• Security + compliance mindset: experience working in SOC 2-aligned environments; ability to implement least privilege, auditability, and operational controls.
• Strong engineering communication: can write clear docs, propose designs, and upskill the team.

About HUD

At HUD, we’re building the future of how companies and individuals train and evaluate AI. We believe that in the near future, most post-training data used to align and improve LLMs will flow through HUD. We build a platform and developer tools that let teams create post-training data through RL environments and run reinforcement fine-tuning (RFT) reliably, reproducibly, and at scale. We’re trusted by foundation labs, Fortune 500s, and fast-growing startups. We’re also a high-caliber team: former founders, published ML researchers, Olympiad medalists, and engineers who have built products with real adoption. We run lean, move fast, and hold an extremely high bar.