Join an AI/ML team building tools to identify and address failures in production AI agents. This role owns the intelligence behind failure detection, evaluation, and proposed fixes, helping make agents more reliable for real-world use.
🏢 About [Company]
Clera is a company focused on AI/ML engineering, working on tools to improve the reliability of AI agents in production.
🎯 The Role
This role involves detecting subtle agent failures, turning user behavior into evidence of potential failures, building evaluation methods, verifying proposed fixes, improving cost efficiency, and reviewing production traces regularly.
✅ Key Responsibilities
Detect subtle agent failures, including omissions, incorrect responses, and patterns across large volumes of traces.
Turn user behavior such as rephrasing, abandonment, and retries into evidence of potential failures.
Build evaluation methods that measure detection quality, including in situations without ground-truth labels.
Verify that proposed fixes reproduce and resolve failures, and avoid recommending changes when evidence is weak.
Improve the cost efficiency of detection by choosing appropriate models and methods for each task.
Review production traces regularly to identify problems and guide product improvements.
📌 Required Qualifications
Demonstrated AI/ML engineering experience, with 0 to 10 years of experience considered.
Experience shipping products that people use and working as an individual-contributor engineer.
Hands-on experience with LLMs in production, including evaluations, LLM-as-judge, embeddings, and deciding when to use smaller models or no model.
Experience running AI agents in production and handling their failures.
Strong research judgment and the ability to distinguish meaningful improvements from noise, including when direct ground truth is unavailable.
🎁 Benefits
Base salary range: $120,000 to $200,000 annually.
🛂 Visa & Eligibility
Visa sponsorship is not available. The role requires authorization to work in the United States without visa sponsorship.
