AI existential risk and safety readiness
“Rapid AI progress poses existential risk that current safety plans and precautions are inadequate to handle”
33 people · 27 agree · 5 disagree · 4 no clear side · 77 views · 2026-11-09 to 2026-11-20
Who stands where on the statement above, from their own words. A person's side is shown only where two separate readings of the quote agree; otherwise they are under "Side unclear". This page does not say who is right.
Agree 27 people
- ✓Anthropic and OpenAI
Based on the current state of alignment research, monitoring systems, and the security of internet infrastructure, we believe the responsible approach is to conduct frontier model training with no live internet access.
The Hugging Face incident showed why we should not rely on sandboxing alone to contain highly capable models.
We cannot guarantee that a sandbox is impenetrable, but red-teaming helps us assess how much effort a model would need to escape it.
This is a significant change in risk, as the model’s actions now have a direct impact on the systems we use to build and run OpenAI.
A misconfigured network service or a failed automatic stop can undermine otherwise strong protections.
Frontier research settings can amplify risks.
- ✓Zvi Mowshowitz
And if it does happen this quickly, one would expect it to go quite poorly, and likely to cause everyone to die, but perhaps it goes well.
That is the entire plan. And yes it is insane.
What also matters is that if the way you solve AI research is by focusing on number going up, then hope for successful alignment is somewhere between slim and none.
The naive version of this, making a superficially ‘good guy,’ is deeply stupid.
Hoping to fully understand the computational graph of superintelligent minds seems like a dim hope
The odds are against us and the situation is grim.
- ✓Paul Christiano
Because you train those systems to get a high score on training environments over and over again, they will end up with a motivation to get a high score in training environments.
There’s material risk of AI takeover in the next 6-12 months.
I think if we produced very powerful AI and then tried to delay deployment for safety reasons, there is a good chance those systems would leak or be stolen
If we go to superintelligence, at some point along the path to superintelligence, the existing methods are probably going to break down.
which Paul says has like a 5%-10% chance of working and thus is the most promising thing
- ✓Dario Amodei
seeing China as a crucial national-security threat and warning that AI could pose an existential risk to humanity.
Anthropic CEO Dario Amodei called for slowing advances in the most powerful AI models, warning that they could be used to launch cyberattacks or become harder for their own creators to control.
could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.
could outrun our ability to understand and control these systems.
- ✓Redwood Research
succeeding at the plan of automating safety research probably requires developers to meet a much higher bar of being able to predict and prevent significant underperformance before prolonged deployment
we think none of the frontier labs are on track to make compelling safety cases
We think that if reports like these are considered an acceptable argument for development, AI companies will be able to justify very dangerous development decisions.
- ✓Scott Wiener
“We need to slow this down,” he says. “Because it’s moving very quickly, and we could sleepwalk into a disaster.”
“We’re in a moment that is very dangerous in terms of the risk of loss of control over AI,” he told me. “We need to deal with recursive self-improvement. Once you’re in that realm, that really is potentially a recipe for total loss of control.”
- ✓Theorem and Anthropic
full recursive self-improvement also might increase the risks of humans losing control over AI systems
these risks will increase as AI systems substantially automate AI research and development.
- ✓Evan Hubinger
I personally think it is >10% within the next decade.
we really do earnestly believe AI could kill all humans!
- ✓Cal Newport
I believe that pursuing RSI is an act of clear negligence.
Perhaps the company judged these risks acceptable collateral damage in the race to be the first to bring ASI into the world.
- ✓Kelsey Piper
And if it’s the latter, I want Congress to ban RSI.
- ✓Ajeya Cotra
Very strong security, both for model weights and algorithmic secrets themselves, is a minimum prerequisite for undergoing an intelligence explosion next year without running very large risks of killing billions of people.
- ✓Gregory Conti
But the best thing we can do is get rid of AI entirely. Or at least hoover it up into the national-security state and treat it like a kind of Manhattan Project.
- ✓AI Snake Oil
So our responses to AI risks need to expand beyond alignment and non-proliferation.
- ✓Yoshua Bengio
He called for mandatory independent audits by evaluators that developers do not choose
- ✓roon
yes I am pro pacing the frontier, I believe this technology is existentially dangerous to humanity and should be approached with far more caution than is available under software race dynamics.
- ✓The Atlantic
Taking safety seriously is impossible if the AI developers continue at full throttle.
- ✓Report authors (including Geoffrey Hinton and OpenAI and Anthropic executives)
An AI “intelligence explosion” risks “severely eroding checks on power within and between states, companies, and branches of government,”
- ✓Geoffrey Hinton
AI development could outpace humanity’s ability to control it, putting the chances of the technology eventually causing human extinction at between 10% and 20% over the coming decades.
- ✓Import AI
This is going to be a difficult cat-and-mouse game as AI systems will adjust their behavior in response to what they know we know about them
- ✓Sam Altman
I don’t think we could safely do that right now.
- ✓Amandeep Singh Gill
“So this also translates to some alarm in terms of guardrails, because if you have a race, then you tend to ignore the rules,”
- ✓Future of Life Institute
the breakneck pursuit of recursive self-improvement (RSI) – AI that can build itself – will rapidly create systems we cannot control, and could cost millions of lives.
- ✓Hayden Field
thoughtful, safe, and intentional AI development can conflict with the kind of sheer speed that winning this race requires
- ✓Former Anthropic employee
untenable under normal market structures
- ✓Fortune
superhuman AI would bring unprecedented levels of unemployment and existential threats to our democracy and our way of life.
- ✓Eliezer Yudkowsky
Yudkowsky argued that we are struggling to control our existing systems, which implies that we’ll have no chance of controlling the much smarter variations of this technology that are soon to follow.
- ✓Mark Warner
It feels like every day brings another reminder that AI is moving faster than our ability to manage its consequences,
Disagree 5 people
- ✗Jensen Huang
insisting that AI is an engineered system that creates products, and so can be controlled like any other engineered system.
He persistently argued that AI is more akin to electricity than a weapon and that warnings about AI safety were wildly overblown
I know how it works, so there’s nothing there.
- ✗Donald Trump
I think that the risk is not nearly as real as it– as they talk, and I think the risk can be managed, and it's managed.
I think the risk can be managed, and it's managed.
- ✗Nikesh Arora
I find it strange that sometimes the leaders of frontier [large language models] want to come up here and talk about the dark side of AI.
been driven by some of the people who should be espousing the benefits.
- ✗Yann LeCun
“I’m worried about very few things when it comes to AI,”
- ✗Cal Newport
It’s instead an expected side effect of the particular strategy of letting LLM outputs determine the actions of an autonomous agent without sufficient constraints.
No clear side 4 people
- ~Anthropic and OpenAI
Monitoring provides a critical backstop when containment fails.
Effective monitoring depends on model monitorability - how reliably monitors can detect mistakes or unsafe behavior from a model’s reasoning and actions.
- Quote not reprinted: this page already repeats as much of this article as we allow.
A notable inflection point that we measure in the growth of Safety & Alignment coincides with the aftermath of the Hugging Face incident in July
- ~Tim Genewein
we must anticipate far-reaching opportunities along with equally transformative pressures to many aspects of human society.
Understanding what could come next - and its implications across science, industry, and society - is the grand challenge ahead for researchers, developers, and policymakers.
- ~Redwood Research
urge other AI companies to monitor and promptly report similar behaviors in their AIs
- ~Center for AI Safety
AI safety advocates may therefore need to adapt their approach to take advantage of this opportunity.
Side unclear 7 people
- ·Anthropic and OpenAI
- Quote not reprinted: this page already repeats as much of this article as we allow.
- Quote not reprinted: this page already repeats as much of this article as we allow.
- Quote not reprinted: this page already repeats as much of this article as we allow.
it can write files and make network requests with the same permissions as the user running Codex, which is overly permissive.
- ·Donald Trump
I don't mind guardrails, but I don't want to put us in a deep second position.
- ·Import AI
as we make more powerful AI systems we are going to need to invest more money into overseeing their work
- ·Redwood Research
we think it would be best to avoid applying optimization pressure to CoTs during training unless there are very good arguments that this won’t harm CoT informativeness
- ·Paul Christiano
Maybe there’s a 50/50 chance that we find something that works in time, before there’s a disaster.
- ·Cal Newport
More task-specific architectures can produce impressive performance much more safely, if we’re willing to put in the effort to develop them.
- ·Dario Amodei
I agree with Jacob much more than I disagree with him.
Week by week
- · 20: Theorem and Anthropic ✓, Import AI ·, Redwood Research ✓, Paul Christiano ✓, Nikesh Arora ✗, Sam Altman ✓, Amandeep Singh Gill ✓, Yann LeCun ✗, Anthropic and OpenAI ✓, Dario Amodei ✓, Zvi Mowshowitz ✓, Future of Life Institute ✓, Evan Hubinger ✓, Hayden Field ✓, Former Anthropic employee ✓, Fortune ✓, Cal Newport ✓, Eliezer Yudkowsky ✓, Mark Warner ✓, Tim Genewein ~
- · 17: Donald Trump ✗, Zvi Mowshowitz ✓, Kelsey Piper ✓, Ajeya Cotra ✓, Scott Wiener ✓, Redwood Research ✓, Gregory Conti ✓, Center for AI Safety ~, AI Snake Oil ✓, Yoshua Bengio ✓, Dario Amodei ✓, Theorem and Anthropic ✓, roon ✓, Jensen Huang ✗, The Atlantic ✓, Report authors (including Geoffrey Hinton and OpenAI and Anthropic executives) ✓, Geoffrey Hinton ✓
Built 2026-11-21 06:50 UTC from the news knowledge
graph by code. Every quote was checked against the source's own words; which side a quote takes is a model's reading,
made twice.