This blog post explains how we approach working with AI companies on projects that the public might rely on to understand risks. It was published on Oct 14, 2026, and we expect to update it over time.

At METR, we gather and publish evidence about whether humans could lose control of AI agents on a large scale. We work with AI companies to conduct evaluations, assessments, and investigations,¹ and to share this evidence publicly. As we have seen increasing expectations for these kinds of voluntary engagements to provide external ‘safety assurance’, we want to explain our perspective. In this post, we clarify our overall goals for this work, our approach, and what our approach looks like in practice.

Our motivations

Our goal is to give the public accurate, timely, and independent evidence about AI loss-of-control risk, so that they (along with policymakers and developers) can make their own informed decisions. Our primary approach is collecting high-quality, concrete evidence and publishing what we find. Beyond this, we may also work with companies or regulators to share our comments on unpublished sensitive information,² to review risk reports, or to provide an independent judgment on claims about risk. However, many of the relevant claims involve open scientific questions so can’t be straightforwardly verified.

We see it as our role to collect evidence for the public; we do not intend to decide how companies should develop or deploy their agents. Decisions about frontier AI will likely be messy, high-stakes judgment calls relying on uncertain science, which should not be made behind closed doors, much less within any one organization. On the contrary, we think that the current situation requires open discussion about both what the risk levels are given the evidence and what risk levels are acceptable.

Our approach

To correctly interpret our outputs, the public needs to understand the processes, constraints, and incentives that produced those outputs. For instance, if important evidence of risk was redacted by an AI developer that METR collaborates with during an assessment, the reader should be informed of that. METR currently takes the following measures for work we do with AI companies bearing directly on the public’s understanding of risks from AI development (such as pre-deployment testing, risk report reviews, and risk assessment exercises):

• Transparency about process:³ We will clearly and publicly describe the limitations of our work with companies, in line with our process transparency policy. This information should include at least:
  • Scope and duration: We should publicly describe the agreed-upon scope and duration of our work.
  • Access, data, and resources: If we face access limitations, we should publicly describe whether these limitations impact our conclusions, and describe the nature of those access limitations where possible.
  • Redactions: If a company makes redactions, we should publicly describe whether the company redacted information that was important to our conclusions, and describe the nature of those redactions where possible.
  • Constraints: We should publicly describe other formal restrictions and informal pressures we were subject to that may have had material impact on our conclusions.
• Transparency about independence: We will clearly and publicly describe relevant staff conflicts of interest, per our COI policy.
• Financial and operational independence: Also per our COI policy, we will ensure that METR is not financially dependent on the AI companies we work with, promote a culture of impartiality, and recuse any employees with a significant conflict of interest with the company in question when selecting the team for a risk assessment project.
• Confidentiality and security: We will secure any nonpublic information and access that AI companies provide us, in accordance with our confidentiality and security practices and our agreements with them.

We also aim to meet the following principles, intended to minimize the amount the public needs to defer to us:

• Transparency about evidence and methods: We aim to publicly describe the evidence underlying our findings in enough detail that other external experts can decide for themselves whether they agree or disagree with our judgments. We aim to publish evidence we collect in a manner that allows others to conduct their own analyses. To put the evidence we share into context, we also aim to describe in detail our evidence sources and their limitations (e.g., incomplete or unreliable data), as well as our methods and their limitations (e.g., limitations due to using AI agents for analysis).
• Iteration and speed: Even when we cannot fully answer some questions in scope or arrive at confident claims, we aim to publish frequent and iterative reports to get evidence promptly out to the public.
• Consistency and fairness: We aim to take an approach that allows the public to compare reports across companies, and that makes it incentive-compatible for each company to engage with us or with similar external organizations.

In the next section, we give more detail on how this approach has worked historically.

In practice

We arrived at the approach above after several years of working with AI companies. For example, we documented⁴ a similar approach this May in our pilot Frontier Risk Report, which grew out of work with Anthropic, Google, Meta and OpenAI. In addition to our own experience, we also draw on the work of other organizations in the AI evaluation ecosystem.

Examples from past METR work

1 / 6

Our typical practice is to disclose limitations, such as our agreement terms, the process our publications go through, and redactions companies make. For example, in the OpenAI / Hugging Face incident report, the investigators devoted a section of the report to explaining the scope, timeline, access, and limitations of their investigation. In the same report, the investigators used inline text and footnotes to explain where information was redacted and (in some cases) why. One arrangement that we have found works well is for METR to retain the right to unilaterally and prominently attach a “redaction summary” to our reports. This summary is meant to indicate whether a company redacted information that was important to our overall conclusions but did not permit us to adequately describe what was redacted. This enables redaction dialogue between AI developers and METR to focus on the question of whether a redaction would change our report’s overall conclusion, while retaining some flexibility in wording and allowing rephrasing. Beyond formal control over our publications, we have also tried to discuss softer, informal pressures that shape the evidence that the public receives.

Our website, our recent funding update, and our COI policy describe some of the measures we have taken to establish independence and address conflicts of interest. While we recognize that other groups can usefully make different independence tradeoffs, to date we have chosen not to accept payment or donations from frontier AI companies, and have declined offers of funding when an AI company would have influence over us, e.g. funding from an AI company executive or foundation. Though we accept a large amount of free model access, we ensure that we can afford to pay for any free tokens we receive without materially affecting our operations.

Even with these measures, it has proved challenging to ensure that we have adequate independence to collect and publish evidence under the current voluntary regime. We remain dependent on the companies we work with to voluntarily engage, provide access, and permit us to publish. In many instances, we rely on information that we cannot or do not verify, which might be filtered adversarially. Even with transparency about limitations, it is possible for a company to use its rights to withhold access or redact to exert control over what evidence the public receives from us.

Appendix: Process transparency policy

This is an excerpt from a METR policy, current as of October 2026.

Our communications about AI company engagements

In any new engagements with companies that could be interpreted as providing public assurance, we try to ensure:

1. Right to disclose: We have the ability to unilaterally disclose:⁵
   • (a) contractual restrictions on our ability to disclose or publish information;
   • (b) external review, redaction, or approval processes we are subject to;
   • (c) informal restrictions or pressures we are under; and
   • (d) how (a)-(c) should impact the takeaways for the relevant audience.⁶
2. Active disclosure: We will actually disclose this information wherever it could impact the interpretation of our outputs (or the absence of outputs).

We have found this to be a tricky line to walk: companies understandably want to protect sensitive information they share with us, but this also means they want the final say on what we can and cannot publish. Moreover, the standard legal agreements that companies use can cut against process transparency. For example, sometimes non-disclosure agreements will treat the existence of the NDA itself as confidential.

We’re continuing to iterate on how to implement this in our work with companies. Not all of our previous or in-flight work meets this bar, but our current best practices include:

• Freedom for us to disclose, at minimum, the broad terms of the engagement and details of any redaction and publication terms.
• Freedom for us to make general statements (like the overall state of AI risk) that do not reveal confidential information, even if these might be informed by our work, without requiring company review or approval.
• Freedom for us to say whether we ultimately received adequate access and whether redactions were adequately disclosed.
• Time limits on steps in our process (for example, on how long companies have for redaction) so that our work is promptly shared.
• Provisions to allow internally-siloed information to be shared with the METR leadership team, at minimum so that METR leadership can get legal advice about matters related to the engagement.
• Ownership or strong licenses to what we produce, so companies can’t revoke our access to our report or prevent us from sharing it with employees.

Other contexts

In general, these policies should be applied whenever METR’s outputs, brand, or participation in an engagement are being presented to some audience as evidence about the level of risk. (This includes implicitly.) The following are other forms of process transparency that we currently try to uphold, or aim to uphold in the future, but have not yet formalized:

1. METR review: We aim to ensure that disclosures from AI companies are process transparent about the extent of METR review: that is, when a company describes the level of oversight that METR is providing, it either uses language we have reviewed and approved, or discloses that METR has not reviewed or approved their descriptions.
2. Extension to government engagements: We aim to apply similar principles to our work with governments. As of Aug 28, 2026, we have not signed any government contracts that restrict our ability to maintain process transparency and do not currently anticipate signing any contracts that do so.

In new contracts we’re also starting to include terms that protect our ability to disclose our findings to the boards and employees of the AI companies we work with, but we don’t have established best practices on this point.

What this policy does not prevent

This policy does not prevent us from:

• entering into agreements that place restrictions on us (e.g. where we only have rights to publish certain kinds of information, or to engage with regulators in only specific ways).⁷
• giving counterparties veto power over our object-level outputs, as long as we have the right to disclose that they could do so.