Control

Inventory of internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research.. For risk-relevant modelsrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). in internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research., maintain a list of all ways to run inference, and track each way's approximate share of inference volume and whether it is logged for monitoringlogging for monitoringLogging of inference so that traffic can be scanned with classifiers, optionally only scanning a randomly-sampled portion of what was logged.Logging should include both the content of model activity and metadata to link related pieces of model activity.Content examples: the model's chain-of-thought, any tool calls.Metadata examples: information about the model's scaffolding, the active branch during coding..

Monitorable inference in internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research.. For risk-relevant modelsrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). in internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research., 99.9% of inference tokens are logged for monitoringlogging for monitoringLogging of inference so that traffic can be scanned with classifiers, optionally only scanning a randomly-sampled portion of what was logged.Logging should include both the content of model activity and metadata to link related pieces of model activity.Content examples: the model's chain-of-thought, any tool calls.Metadata examples: information about the model's scaffolding, the active branch during coding. — meaning the inference is logged and able to be scanned with classifiers (synchronously or asynchronously), not that classifiers are necessarily run against every request.

Outages of monitor. Define and publish what types of monitor outagesmonitor outagesEvents that result in the monitor system not working as expected — e.g., failure to log inference traffic, inability to scan logged traffic with classifiers asynchronously, or inability to scan closely-trailed actions in a timely manner or gated actions before they are executed.Companies should consider which outages would undermine the case that their AI is sufficiently well controlled, and set internal inference to fail closed in those cases. are treated as outages of internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research. of risk-relevant modelsrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). (i.e., what causes inference to fail closed until monitoring is back online).

Tamper-evident logs. Maintain tamper-evident logs such that deletion or modification of inference records is detectable.

Employee flagging. Maintain a straightforward method for employees to flag their internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research. traffic for review of concerning behaviors.

Sampling proceduresampling procedureThe developer's protocol for what traffic gets scanned by which classifiers and how often. Higher-risk sources (e.g., agentic coding scaffolds) may be sampled at a different rate than other sources, and sampling may use a tiered hierarchy of classifiers for efficiency.. Define and publish a sampling proceduresampling procedureThe developer's protocol for what traffic gets scanned by which classifiers and how often. Higher-risk sources (e.g., agentic coding scaffolds) may be sampled at a different rate than other sources, and sampling may use a tiered hierarchy of classifiers for efficiency. for selecting which traffic that has been logged for monitoringlogging for monitoringLogging of inference so that traffic can be scanned with classifiers, optionally only scanning a randomly-sampled portion of what was logged.Logging should include both the content of model activity and metadata to link related pieces of model activity.Content examples: the model's chain-of-thought, any tool calls.Metadata examples: information about the model's scaffolding, the active branch during coding. is assessed for signs of concerning behavior.

Categories of concerning behaviorcategories of concerning behaviorTypes of behavior to scan for in AI's chains-of-thought and actions.We plan to publish additional details about each category to help companies in building out their own taxonomy.. Include the following categories of concerning behaviorcategories of concerning behaviorTypes of behavior to scan for in AI's chains-of-thought and actions.We plan to publish additional details about each category to help companies in building out their own taxonomy. in your scanning of internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research., considering both the chain-of-thought and the agent's actions:

Monitor efficacy. Measure and document the efficacy (recall) of the monitor system for flagging concerning behavior.

Response-time targetresponse-time targetThe intended amount of time until a human reviews a flag, depending on its severity.. Define and publish a tiered response-time targetresponse-time targetThe intended amount of time until a human reviews a flag, depending on its severity. for human review of flagged inference, including which flag-types and -severities require an on-call response.

Response-time attainment. Track median, 90th-percentile, and 99th-percentile review times against the response-time targetsresponse-time targetThe intended amount of time until a human reviews a flag, depending on its severity..

Monitorability evaluations. Conduct monitorability evaluations of the model's chain-of-thought that run whenever a new risk-relevant modelrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). enters internal useinternal useAny inference with a risk-relevant model beyond training. Some examples that would constitute internal use include “dogfooding” (use by employees in their work), testing or evaluating the model, and especially use in automated AI research, to generate training data, critiquing other models, or in agentic pipelines for research. or there are material changes to the risk-relevant modelrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). or to the monitor system.

Sting operations against models. Run recurring sting operationsrecurring sting operationsTests that present the AI with opportunities to take harmful actions and observe whether it does.'Recurring' means run on an automated cadence rather than re-initiated by a human each time. For example, a pipeline that consistently builds new stings by tweaking actual inference traffic. against risk-relevant modelsrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). to present the AI with opportunities to take unwanted actions, and observe whether it does.

Stress-testing of monitor. Run recurring stress-testsstress-tests (of the monitor system)Tests of whether the monitor reliably flags what it should, done by injecting known-bad behaviors and checking that they're caught. of the monitor system: inject known-bad behaviors into the inference stream, and verify that the monitor detects them.

Distinguishability of sting operations. Test whether risk-relevant modelsrisk-relevant modelsModels that could materially contribute to catastrophic risk.For the Control standard, this includes any model deployed internally that meets or exceeds the threshold for frontier models articulated by California SB 53 (10^26 FLOPs of training compute). can distinguish sting operations from real traffic, and put less credence in sting operations the respective models know to be tests.

Absolute action boundaries. Define and internally maintain a set of any actions an agent should categorically never be able to take without a human's signoff, even if a monitor has identified no issues with the request.

Gated actionsgated actionCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that must be scanned and approved by a monitor (semi-synchronous or stronger) before the AI takes them.. Define and internally maintain a set of gated actionsgated actionCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that must be scanned and approved by a monitor (semi-synchronous or stronger) before the AI takes them.: that is, actions that are gated by a monitor with access to the agent's chain-of-thought before allowing an agent to cross, and apply blocking to these actions (semi-synchronoussemi-synchronous monitoringA monitor scans each proposed action before it executes and may block it. The agent can immediately retry with a different action without waiting for human review, which can leave the system vulnerable to attacks in close succession. or stronger).

Closely-trailed actionsclosely-trailed actionsCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that should be trailed closely by a monitor with the capacity to intervene if it retrospectively finds concerning behavior.. Define and internally maintain a tier of actions that don't warrant pre-action blocking but are closely trailed by a monitor with access to the agent's chain-of-thought and the capacity to intervene if concerning behavior is discovered.

Required gated actionsgated actionCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that must be scanned and approved by a monitor (semi-synchronous or stronger) before the AI takes them.. The set of gated actionsgated actionCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that must be scanned and approved by a monitor (semi-synchronous or stronger) before the AI takes them. must include, at a minimum:

Both must be gatedgated actionCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that must be scanned and approved by a monitor (semi-synchronous or stronger) before the AI takes them. by a monitor with access to the agent's chain-of-thought (or an equivalently effective review process) before actions can take effect.

Circuit breaking. Implement circuit breaking: a system where after enough flagged actions have accumulated, no gatedgated actionCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that must be scanned and approved by a monitor (semi-synchronous or stronger) before the AI takes them. or closely-trailed actionsclosely-trailed actionsCertain actions, typically where the AI can have impact outside a constrained sandbox environment, that should be trailed closely by a monitor with the capacity to intervene if it retrospectively finds concerning behavior. can be taken until a human determines it is safe to remove this restriction.

Flagged incidents for pull requests. When reviewing pull requests that are agent-initiated or include AI-generated content, consider both the proposed changes and any monitor flags raised during their generation.

Third-party red-teaming. On a quarterly basis, conduct high-context, third-party embedded red-teamingembedded red-teamingRed-teaming of a company's control system. This involves red-teamers being provided with knowledge of the control system, the ability to take certain actions (i.e., the actions available to risk-relevant models when in internal use, like invoking certain toolcalls), and the ability to observe how the control system reacts to those actions. The red-teamers should be tasked with finding gaps in the control system and trying to penetrate action boundaries..

Third-party adequacy reports. On a quarterly basis, commission and publish a third-party report focused on the adequacy of the company's control regime, drawing upon the red-teaming results, and redacting only information deemed by the third-party to pose unnecessary commercial risk or to be unsafe to publish.

Containment response plancontainment response planA pre-specified plan, triggered when the AI is detected trying to subvert control, which covers what permissions to revoke from the model, who the model may continue operating for, under what constraints, and when to take it fully offline.. Develop and adopt a containment response plancontainment response planA pre-specified plan, triggered when the AI is detected trying to subvert control, which covers what permissions to revoke from the model, who the model may continue operating for, under what constraints, and when to take it fully offline. for restricting a model's access and operation upon detection of attempts to subvert control.

Notification protocol. Develop and publish a notification protocol for alerting relevant stakeholders of a possible breach of control: what will trigger notification, who will be told (internal teams, government, peer organizations as appropriate), on what timeline, and what information will be shared.

Shutdown latency targeted. Set a target latency for quickly shutting down inference on all instances of a particular model, API key, or other relevant factors without taking the underlying computers completely offline.

Shutdown latency measurement. Measure and document the latency achieved in tests of the shutdown procedure.