Brian Goodrich (former)

Brian Goodrich

Former Technical Staff

Brian was on the Model Interaction team, and had a particular interest in understanding patterns of agentic capabilities. Before joining METR, he wrote software for an energy efficiency startup, and practiced forecasting.

By Brian Goodrich

HCAST: Human-Calibrated Autonomy Software Tasks

March 17, 2025

Sharing details about HCAST (Human-Calibrated Autonomy Software Tasks), a benchmark we’ve been developing to measure the abilities of frontier AI systems to complete diverse software tasks autonomously.

Evaluating frontier AI R&D capabilities of language model agents against human experts

November 22, 2024

We’re releasing RE-Bench, a new benchmark for measuring the performance of humans and frontier model agents on ML research engineering tasks. We also share data from 71 human expert attempts and results for Anthropic’s Claude 3.5 Sonnet and OpenAI’s o1-preview.