Research Scientist, Open Benchmarks Red Team
💰 $80,000 – $130,000/yrMarket estimate · not provided by the employer
At a glance
- Who can apply
- United States
- Work arrangement
- Hybrid · New York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote)
- Employment type
- Full-time
- Salary
- $80,000 – $130,000/yr · market estimate
Checked against the employer's source on November 21, 2026. Details come from the employer's posting; confirm them on the application page.
Advertisement
Put your brand in front of 300,000+ monthly visitors worldwide
Job Description
Snorkel AI is a data and evaluation company working with frontier AI labs and enterprises. This role supports its Open Benchmarks Red Team program, which finds and repairs weaknesses in frontier AI benchmarks.
- Identify and reproduce benchmark problems such as reward hacking, grader failures, task ambiguity, shortcuts, and environment vulnerabilities.
- Propose next steps for validated benchmark issues and implement selected fixes together with benchmark authors.
- Verify that proposed fixes work without breaking valid solutions to benchmark tasks.
- Reuse existing tools where possible and build new checks for recurring benchmark weaknesses.
- Review findings with benchmark authors before publication and support official improved benchmark releases.
- Contribute to technical reports, open-source tools, publications, and new benchmark evaluation approaches.
Summary of Snorkel AI's posting by JobsRemoteList. Read the full details and apply on the employer's site.
Requirements
- Strong research or engineering background in machine learning, LLMs, AI agents, and evaluation.
- Strong Python skills, including building, running, and debugging AI agents or evaluation systems.
- Ability to find weaknesses in evaluation methods, develop new approaches, and rigorously test them.
- Experience designing rigorous experiments, analyzing model behavior, and investigating unexpected failures.
- Familiarity with LLM or agent benchmarks, evaluation frameworks, or automated grading systems.
- Experience independently driving projects from problem identification through implementation and validation.
- Strong communication and collaboration skills.
- Preferred: experience with adversarial testing, red teaming, reward hacking, or specification gaming.
- Preferred: experience developing or evaluating benchmarks for coding agents, computer use, or professional workflows.
- Preferred: familiarity with RL environments, agent trajectories, verifiers, and evaluation infrastructure.
- Preferred: open-source contributions or publications in AI evaluation, robustness, or agentic systems.
- Preferred: PhD in Computer Science, Machine Learning, or a related field, or equivalent research or industry experience.
Benefits
- Salary range stated as $200,000 to $375,000 USD, with actual pay depending on skills, experience, and geographic location.
Advertisement
Put your brand in front of 300,000+ monthly visitors worldwide
Related Jobs
Accounting Manager, General Ledger & Accounting Operations
Snorkel AI
Senior Software Engineer
Snorkel AI
Engineering Manager, Data Delivery
Snorkel AI
Staff Applied AI Engineer - Pre-Sales
Snorkel AI
More openings for this role: 30 remote Research Scientist jobs
Find more than a single listing
Discover more jobs
Join now and find your dream job or a higher-paying opportunity with better employer visibility, stronger internal discovery and easier access to worldwide hiring signals.
Discover hidden jobs
Surface roles you may miss on noisier boards and move through cleaner employer links and better intent pages.
Head start against the competition
See fresh roles, related pages, salary context and company signals faster so you can apply earlier and better.
10,000+ more jobs than narrower job boards
Explore remote, hybrid and location-based opportunities across worldwide markets instead of relying on one narrow inventory.
Worldwide opportunity coverage
Use region pages, country-eligibility pages and skill pages built for global job seekers, not just one market.
Social proof
Trusted by professionals
“I discovered several remote-friendly companies that I had not seen elsewhere.”
Catalina Rojas
Data Engineer · 🇨🇱 Chile
“Good organization and useful information for an international job search.”
Ama Mensah
Product Manager · 🇬🇭 Ghana
“The alerts helped me keep track of new remote opportunities.”
Camila Quispe
Customer Success Manager · 🇵🇪 Peru