Senior Machine Learning Engineer

About Ambience Healthcare

Ambience is building the AI intelligence platform that restores humanity to healthcare. We deliver real-time, coding-aware clinical documentation and workflow support across ambulatory, emergency, and inpatient settings at top North American health systems. Ranked #1 for Improving Clinician Experience by KLAS Research, recognized by Fast Company, Inc., and LinkedIn as a top startup, backed by Oak HC/FT, Andreessen Horowitz, OpenAI Startup Fund, and Kleiner Perkins.

The Role

As a Senior Machine Learning Engineer, you will own complex projects end-to-end—from diagnosing production failures and designing evaluations to building, deploying, and iterating on model and agentic systems. This highly hands-on role involves significant technical ownership, working closely with clinicians, product managers, and engineers to translate cutting-edge research into reliable, production-grade AI systems.

Key Responsibilities

• Build Trustworthy AI Evaluation Systems: Design and own evaluation pipelines for LLM and agentic systems, combining automated graders, regression testing, production feedback, and human evaluation to measure real product quality.
• Improve Production Model Behavior: Diagnose high-impact failure modes and test improvements across prompting, retrieval, context, routing, data, fine-tuning, and other model and system interventions.
• Build Agentic AI Systems: Develop production systems involving tool use, retrieval, context and state management, routing, orchestration, tracing, and failure recovery.
• Build Data and Improvement Flywheels: Turn production failures and user feedback into better datasets, evaluations, and model behavior through active learning and systematic iteration.
• Stay at the Cutting Edge: Distill insights from recent research in LLMs, agents, NLP, and speech to inform product strategy and engineering decisions.

Work Arrangement

This is a hybrid role requiring three days per week onsite at our San Francisco office.