Staff ML Engineer, Frontier AI

About Ambience Healthcare

Ambience is building the AI intelligence platform that restores humanity to healthcare. We deliver real-time, coding-aware clinical documentation and workflow support across ambulatory, emergency, and inpatient settings at leading North American health systems. Ranked #1 for Improving Clinician Experience by KLAS Research, recognized by Fast Company as a Next Big Thing in Tech, and backed by Oak HC/FT, a16z, OpenAI Startup Fund, and Kleiner Perkins.

The Role

As a Staff Machine Learning Engineer, you will set technical direction for AI systems powering our clinical products. This is a highly hands-on role with broad technical influence, working closely with clinicians, product managers, researchers, and engineers to translate cutting-edge research into production-grade AI systems.

Work Arrangement: Hybrid—three days per week onsite at our San Francisco office.

Key Responsibilities

• Define AI Quality and Evaluation Strategy: Establish measurement frameworks for production AI quality across LLM and agentic systems, including automated graders, regression testing, human evaluation, failure taxonomies, and offline-to-online validation.
• Drive Production Model Improvement: Identify high-impact failure modes and lead improvements across prompting, retrieval, context, routing, data, and post-training. Design experiments to prove improvements hold in production.
• Advance Post-Training Capabilities: Shape use of supervised fine-tuning, preference optimization, reinforcement learning, distillation, and synthetic data to improve model behavior.
• Architect Agentic AI Systems: Set technical direction for production systems involving tool use, retrieval, context and state management, routing, orchestration, tracing, and failure handling.

What We're Looking For

We seek engineers with deep expertise in machine learning systems, production AI, and the ability to influence technical strategy across teams. Experience with LLMs, agentic systems, evaluation methodologies, and post-training techniques is essential.