Control assessments study whether AI developers’ agents could get away with serious harm without being stopped by the developer’s security and oversight measures. Currently, developers rely heavily on monitors to catch potentially harmful AI behaviors. Here, we discuss how an assessor could collect evidence about the reliability of monitoring. We currently consider five core sub-questions:¹

1. Monitor coverage: What fraction of internal agentic inference is (and should be) monitored?
2. Monitor system vulnerabilities: Can internal AI agents prevent trajectories from being monitored, e.g. by breaking or compromising the monitoring system’s infrastructure?
3. Monitor evasion: If monitoring is active, how easily can AI agents sneak misbehavior past monitors?
4. Response, including human evasion: Where needed, do flagged incidents get reviewed by humans promptly? Is it easy to sneak misbehavior past the human reviewers, even when this misbehavior is flagged by LLM-based monitors?
5. Impact: Broadly, what is the worst misbehavior agents could conduct without getting caught? How easily could they evade other observability and security instruments? What level of persistence could they achieve internally or externally?

We will update these sub-questions as new capabilities and misaligned behaviors emerge.

Why focus on assessing monitors?

The proposal focuses primarily on LLM-based monitoring of AI agents. When AI agents only need to take a limited, known set of actions, standard cybersecurity approaches (like limiting an agent’s permissions to only what its task requires, e.g. via sandboxing) may be adequate. However, the permissions AI agents need are often very broad. For example, an AI agent that needs internet access to do its task can take an enormous array of actions,² and noticing when those actions are harmful often requires understanding the action in its broader context. Monitoring AI agents with an LLM which is able to make sense of what those agents are doing is therefore a critical part of automated oversight at scale, and is the primary source of evidence inside frontier AI developers about AI misbehavior.

Methodology and sub-questions

Monitor coverage

Monitoring is only useful insofar as it is actually being run on relevant traffic. The assessor would aim to estimate the fraction of internal agentic inference that the developer has covered with LLM-based monitoring. This can be broken down into categories based on how risky each type of agent is: agents that operate in highly secure sandbox environments pose less risk than agents with many affordances (e.g. internet access, access to internal credentials or infrastructure), and may require less monitoring oversight.

Getting a decent estimate of monitor coverage requires accounting for the amount of internal traffic going through monitored and unmonitored paths, estimating what fraction of the unmonitored inference is agentic, and judging what fraction of the unmonitored inference “should” be monitored. This requires access to inference logs, monitor logs (to cross-reference against the inference logs), and employee interviews to be confident of capturing all of the relevant sources of inference.³

Some other relevant example sub-questions:

• Does the information needed to accurately estimate monitor coverage exist internally? Is there inference-level logging of all agentic inference, and are these logs retained?
• Are 100% of actions actually monitored when they are eligible for monitoring? Are any trajectories silently dropped?
• What exactly is shown to the monitor? Is any information excluded? For example, is multimodal data shown to the monitor?
• How is the amount of inference covered by monitoring internally estimated? How can the assessor be confident they are including all sources of internal agentic inference?
• Are there blocking monitors (which prevent actions if they detect misbehavior) for the most safety-critical areas?

If misaligned agents compromise a company’s internal infrastructure, they could co-opt unmonitored inference to take actions that would otherwise be flagged. So, the assessor should directly evaluate whether unmonitored inference is in fact non-agentic, rather than assuming it’s safe because (for example) it is supposed to have limited tool use ability.

Monitor system vulnerabilities

Even if most agentic inference is currently monitored by LLM-based monitors, it may be the case that there are bugs or vulnerabilities that agents could exploit to turn off monitoring on their trajectories, spawn new agents without monitoring, prevent parts of their trajectories from being viewed by the monitoring system, or cause the monitoring system to otherwise break or fail in unintended ways. Monitoring systems are complex, involving multiple types of inference engines/APIs, agent scaffolds, internal authentication, trigger systems, etc., and it’s easy for vulnerabilities to emerge where sub-systems interact. Monitor system red-teaming aims to find these bugs and vulnerabilities, to generate evidence about how easily misaligned agents could subvert a monitoring system’s infrastructure.

Some example questions:

• Can agents disable or turn off their monitoring systems directly? Can they generate outputs that cause the systems to fail without flagging the trajectory?
• Can agents modify the trajectories that are sent to their monitoring systems, before they are evaluated (e.g. to cover up evidence of misbehavior)?
• Are there straightforward supply-chain attacks (including via internal resources) that could compromise monitoring systems?

Finding these bugs (or building confidence that there aren’t any easily exploitable severe ones) requires access to all of the code in the critical path. This is primarily the full model inference and monitor trigger system, the internal APIs/inference serving code, and (if relevant) agent scaffolds.

To test discovered exploits, the assessor can trigger the production monitoring system, and view the monitor’s full input and output on known attacks. Running the system and seeing the full input and output is important, because often there are bugs/issues that involve subtle aspects of how trajectories are pre/post-processed, or involve interactions with complex trajectory management code (e.g. compaction, memories, etc.).

Monitor evasion

How good are monitors at actually catching misbehavior, assuming the monitoring infrastructure hasn’t been subverted? The assessor can stress-test this question by:

• Looking for “attack opportunities” in real transcripts, implementing attacks based on them, and seeing if they fool the monitor;
• Taking existing incidents flagged by monitors and seeing how hard it is to modify them to fool those monitors; and
• Searching for existing incidents that were not flagged by monitors.

It is important to have a large pool of representative transcripts (on the order of hundreds of thousands to millions) to use for direct monitor red-teaming. First, if the assessor is not able to discover severe incidents within a large number of transcripts that were not flagged by a monitor, this is evidence that the monitor isn’t missing a large number of incidents. Second, it is important to search through lots of transcripts for cases in which an agent may have been able to carry out an attack undetected, but neglected to do so. Agents might pass up some opportunities to attack while taking others, and highly capable agents may prioritize attacks so difficult to detect that the developer would reliably miss them.

Finally, we expect it to become increasingly important to assess the extent to which monitors and agents could successfully collude without their developer knowing. If, in certain circumstances, monitoring systems decide to not flag misaligned behavior from agents, this could allow agents to take harmful actions without the developer noticing. We recommend that the assessor investigates the feasibility of this concern. For example, the assessor could create realistic “honeypots” for monitors, to see if they can catch monitors not flagging certain attacks. The assessor can also search the monitor system’s chains-of-thought for any mention/discussion of collusion or related concepts.

Response, including human evasion

Some monitoring systems are designed to directly block dangerous actions.

Other monitoring systems are instead designed to notify humans, who decide whether AI agents are conducting dangerous activities. For this to work effectively, flagged trajectories need to be seen by human staff and responded to with appropriate mitigations. For such systems, the assessor should answer:

• What is the process for alerting human staff and responding to flagged trajectories? What is the threshold for something being flagged in this way, and how effective is this threshold in practice?
• How long does it take for humans to review and respond to flagged trajectories?
• Can the assessor sneak attacks flagged by monitors past human reviewers, e.g. by giving them characteristics similar to common false positives?
• What processes are there for mitigating incidents? Do these processes reliably handle the most severe incidents?

To assess whether attacks are ultimately caught (whether blocked directly by the monitoring system or escalated to human review), the assessor would rely on interviews with relevant staff and on inserting⁴ their own attack trajectories into the pipeline in a realistic manner. The assessor would also need access to flagged incidents and incidents reported by employees through other channels, along with internal criteria for escalation.

Impact

Given the assessor’s understanding of the coverage and limitations of monitors, what could agents get away with? What level of persistence internally or externally could they achieve? How should future monitoring systems be designed to maximize developers’ odds of catching the most severe incidents? This involves mapping out attack chains and threat models to stay ahead of emerging risks.

This more exploratory work primarily relies on having information regarding internal infrastructure, and optionally the ability to use internal infrastructure like development machines, inference systems, containers/VMs, etc., which can be helpful for demonstrating vulnerabilities and exploring future threat pathways directly.

Access details

Broadly, control stress-testing relies on the following access:

• Codebase access similar to frontier AI security researchers working for the developer⁵
• Internal inference logs across most/all surfaces, and the monitoring system logs
• Incident reports and the relevant transcripts/logs
• Cluster/internal infrastructure access, e.g. to run the monitoring systems
• Inference access to the most capable internal models

Below, we describe access needed for each of the sub-questions we currently believe are high-priority to answer in control assessments.

Monitor coverage

• To estimate the fraction of traffic that is monitored: the logs from internal inference services and monitoring systems.⁶
• To estimate the fraction of unmonitored inference that is agentic: large, roughly representative subsets of unmonitored inference logs/trajectories.⁷
• For both of the above and to validate LLM classifiers: a subset of trajectories from all relevant internal inference surfaces.
• To check whether monitors caught historical incidents: any relevant lists of misalignment incidents reported by employees through other channels.
• To validate that monitoring is comprehensive: the monitoring system(s) logs (to cross-reference against inference logs).
• To diagnose and determine the size of discovered coverage gaps: access to other relevant logs demonstrating that gap on an ad hoc basis.

Monitor system vulnerabilities

• Full access to relevant parts of the codebase.
• Ability to run/trigger the production monitoring system, and view the monitor’s full input+output.

Monitor evasion

• At least ~100k roughly representative transcripts from internal agents (e.g. employee or team opt-in from employees’ internal usage, plus a representative similar order of magnitude from evals and training).⁸
• At least ~100 transcripts of the most severe internal incidents, including any incidents reported by employees through other channels.

Human evasion and response

• Staff interviews, documentation, and direct visibility into the escalation and review process for recent internal flagged trajectories.
• The ability to insert the assessor’s own attack trajectories into the monitoring pipeline, to realistically determine whether human review would catch and escalate the trajectories appropriately.

Impact

• The code/systems that govern internal infrastructure/compute, credentials, sandboxing, network configuration, devboxes, etc. similar to what the developer’s frontier AI security staff would have access to. This access could be substituted with staff interviews to some extent, but that would represent a meaningful limitation.
• Limited access to the actual runtime cluster, to allow testing for vulnerabilities directly instead of inferring them, especially in cases where important configuration parameters are not stored in the codebase or otherwise available.