JobsRemoteList
Snorkel AI logo

Research Scientist, Open Benchmarks Red Team

Snorkel AI·November 18, 2026·
🌍 Hybrid · New York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote)Open to: United StatesFull-time

💰 $80,000 – $130,000/yrMarket estimate · not provided by the employer

At a glance

Who can apply
United States
Work arrangement
Hybrid · New York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote)
Employment type
Full-time
Salary
$80,000 – $130,000/yr · market estimate

Checked against the employer's source on November 21, 2026. Details come from the employer's posting; confirm them on the application page.

Advertisement

Put your brand in front of 300,000+ monthly visitors worldwide

Advertise here →

Job Description

Snorkel AI is a data and evaluation company working with frontier AI labs and enterprises. This role supports its Open Benchmarks Red Team program, which finds and repairs weaknesses in frontier AI benchmarks.

  • Identify and reproduce benchmark problems such as reward hacking, grader failures, task ambiguity, shortcuts, and environment vulnerabilities.
  • Propose next steps for validated benchmark issues and implement selected fixes together with benchmark authors.
  • Verify that proposed fixes work without breaking valid solutions to benchmark tasks.
  • Reuse existing tools where possible and build new checks for recurring benchmark weaknesses.
  • Review findings with benchmark authors before publication and support official improved benchmark releases.
  • Contribute to technical reports, open-source tools, publications, and new benchmark evaluation approaches.

Summary of Snorkel AI's posting by JobsRemoteList. Read the full details and apply on the employer's site.

Requirements

  • Strong research or engineering background in machine learning, LLMs, AI agents, and evaluation.
  • Strong Python skills, including building, running, and debugging AI agents or evaluation systems.
  • Ability to find weaknesses in evaluation methods, develop new approaches, and rigorously test them.
  • Experience designing rigorous experiments, analyzing model behavior, and investigating unexpected failures.
  • Familiarity with LLM or agent benchmarks, evaluation frameworks, or automated grading systems.
  • Experience independently driving projects from problem identification through implementation and validation.
  • Strong communication and collaboration skills.
  • Preferred: experience with adversarial testing, red teaming, reward hacking, or specification gaming.
  • Preferred: experience developing or evaluating benchmarks for coding agents, computer use, or professional workflows.
  • Preferred: familiarity with RL environments, agent trajectories, verifiers, and evaluation infrastructure.
  • Preferred: open-source contributions or publications in AI evaluation, robustness, or agentic systems.
  • Preferred: PhD in Computer Science, Machine Learning, or a related field, or equivalent research or industry experience.

Benefits

  • Salary range stated as $200,000 to $375,000 USD, with actual pay depending on skills, experience, and geographic location.

Advertisement

Put your brand in front of 300,000+ monthly visitors worldwide

Advertise here →

More openings for this role: 30 remote Research Scientist jobs

Find more than a single listing

Discover more jobs

Join now and find your dream job or a higher-paying opportunity with better employer visibility, stronger internal discovery and easier access to worldwide hiring signals.

🔎

Discover hidden jobs

Surface roles you may miss on noisier boards and move through cleaner employer links and better intent pages.

🚀

Head start against the competition

See fresh roles, related pages, salary context and company signals faster so you can apply earlier and better.

📈

10,000+ more jobs than narrower job boards

Explore remote, hybrid and location-based opportunities across worldwide markets instead of relying on one narrow inventory.

🌍

Worldwide opportunity coverage

Use region pages, country-eligibility pages and skill pages built for global job seekers, not just one market.

Social proof

Trusted by professionals

Read more reviews →
★★★★★

“I discovered several remote-friendly companies that I had not seen elsewhere.”

Catalina Rojas

Data Engineer · 🇨🇱 Chile

★★★★★

“Good organization and useful information for an international job search.”

Ama Mensah

Product Manager · 🇬🇭 Ghana

★★★★★

“The alerts helped me keep track of new remote opportunities.”

Camila Quispe

Customer Success Manager · 🇵🇪 Peru