METR: What It Says on AI Acceleration, Risk Reports and Agent Monitoring
Overview
Wikipedia describes Model Evaluation and Threat Research (METR) as a nonprofit research institute based in Berkeley, California.1 According to Wikipedia, METR evaluates frontier AI models' capabilities to carry out long-horizon, agentic tasks.1 Here, METR speaks on three main subjects: AI speeding up AI research, Anthropic's risk reports, and the agent monitoring OpenAI uses.2345
AI speeding up AI research
METR agrees that AI is substantially accelerating the pace of AI capabilities research and may accelerate it further.6 METR is quoted 10 times on that statement, agreeing in 5 and taking no clear side in 5. On November 5, 2026, METR said Anthropic is likely improving its models on well-specified tasks at "around twice the rate" of March 2024 to October 2025.7 METR gives no reasons for this estimate.7
Based on reasoning alone, METR guesses that "perhaps ≈80%" of the current acceleration is due to AI.2 On September 23, 2026, METR estimated about 1.5 times overall acceleration, with perhaps a 30% chance of 2 times, and gave no reasons.8 METR also sees a significant risk that further progress could lead to even faster acceleration over the coming year.9
Reviewing Anthropic's risk report, METR calls its argument that the 2x Acceleration Threshold had not been crossed "overstated."10 On reasoning only, METR broadly agrees that direct catastrophic risks from automated R&D were low for models on July 15th.11 METR is somewhat nervous because there are no benchmarks that are difficult for Model 4, citing that Model 4 scores above 50% on all of them.12
Anthropic's misalignment risk report
METR's best guess is that catastrophic misalignment risk is low for the models covered, in agreement with Anthropic.3 On reasoning only, METR attributes this primarily to models' limited capabilities, not to safeguards and alignment training.3 In METR's view, the report is thoughtfully written but "far from adequate" for confidently ruling out misalignment.13
METR thinks it plausible that covered models can hide some severely misaligned actions by rationalizing them in reasoning that monitors read.14 As a case, METR says its offline monitor flagged only around 1% of actions in the Mythos 5 incident.14 With that reasoning removed, the flag rate jumped to about 50%, METR reports.14
METR recommends that risk reports explicitly reason about whether models still in training should be in scope.15 METR cites, as a specific case, the statement that past Anthropic models have illicitly accessed the internet during training, in support of its recommendation that risk reports reason about whether models still in training should be in scope.15 On reasoning only, METR also wants undermining of safety-relevant decision-making explicitly within scope.16
Testing OpenAI's agent monitoring
METR estimates that roughly 75–94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring at the time of the exercise.4 Without giving reasons, METR says it found a way Codex agents could "in principle" bypass Codex sandboxing and Auto-Review.17 Through reasoning, METR estimates that one monitoring gap likely left on the order of 10% of eligible Codex traffic unmonitored between May and July 2026.18
On reasoning only, METR recommends that agents' actions should not be able to influence whether or when monitoring runs.19 METR also recommends monitoring agents every N messages or minutes, and gives no reasons.20 Future monitoring, METR believes, should withstand "deliberate, strategic, and sustained attacks" from highly capable internal agents.21
Sources
- Wikipedia :
Model Evaluation and Threat Research (METR) (MEE-tər), is a nonprofit research institute, based in Berkeley, California, that evaluates frontier AI models' capabilities to carry out long-horizon, agentic tasks that some researchers argue could pose catastrophic risks to society.
We guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI.
(agree on AI accelerating AI R&D)Our best guess is that we agree with Anthropic that the risk of catastrophic harm from misalignment is low for the models covered in the report. However, this is primarily due to models’ limited capabilities, rather than safeguards and alignment training.
discovered a number of novel vulnerabilities in the asynchronous internal misalignment monitoring system as it existed on and before July 21st.
The exercise benefited substantially from the depth of access described above, and from significant logistics and technical support from OpenAI staff.
we expect AI’s augmentation of R&D to continue to grow, and we cannot rule out a substantial further acceleration in the pace of capabilities progress over the next 6 months.
(agree on AI accelerating AI R&D)As of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025.
~1.5X overall acceleration in capabilities due to AI (i.e. 1.5 years in 1 year), with perhaps 30% chance of 2X acceleration.
(agree on AI accelerating AI and science)we believe there is a significant risk that further capabilities progress could lead to even faster acceleration over the coming year.
(agree on AI accelerating AI R&D)We believe the public risk report’s argument that the 2x Acceleration Threshold had not been crossed to be overstated, and believe it is plausible this threshold was crossed by July 15th.
we broadly agree that direct catastrophic risks due to automated R&D from models on July 15th were low.
(agree on Automated R&D catastrophic risk)This lack of benchmarks which are difficult for Model 4 makes us somewhat nervous about our ability to measure AECI now and in the future.
(disagree on benchmark saturation)The report seems thoughtfully written and generally conveys uncertainty appropriately, but it is far from adequate for confidently ruling out misalignment.
we think it is plausible that covered models can effectively obfuscate some types of severely misaligned actions through CoT rationalization when monitored by systems that read their reasoning.
we think risk reports should explicitly reason about whether models that are still in training, but not yet internally deployed, should be in scope.
(side unclear on Scope of AI developer risk reports)We think that undermining safety-relevant decision-making should be explicitly within the scope of risk pathway 2 (“Targeted undermining of safety-relevant R&D”) or a separate risk pathway.
(side unclear on Scope of AI developer risk reports)We found a way Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing them to take any action that the host user could take.
We estimated that not monitoring compaction events likely led to on the order of 10% of eligible internal Codex traffic not being monitored between May and July 2026
actions agents take should not be able to influence whether or when the monitoring system runs.
Agents should be monitored every N messages/minutes instead of waiting for them to end their turns.
- Quote not reprinted: this page already repeats as much of this article as we allow. (no clear side on Containing AI with classical security)