Safety standard for shipping AI
“Labs should not ship frontier AI models unless they are proven safe”
6 people · 5 agree · 3 disagree · 3 no clear side · 18 views · 2026-09-23 to 2026-10-08
Who stands where on the statement above, from their own words. A person's side is shown only where two separate readings of the quote agree; otherwise they are under "Side unclear". This page does not say who is right.
Agree 5 people
- ✓Anthropic and OpenAI
structured safety documentation should be required before continuing any frontier reinforcement learning training run
Ideally, such documentation would rise to the level of “safety cases”
The safety case should be reviewed by members of senior leadership, who should each have the ability to veto the run
Safety features such as monitoring and auto-pausing should fail closed
- ✓David Robinson
frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning
AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science
the time for trial and error is over. Achieving something much closer to perfection the first time is essential because iteration after a mistake may not be possible.
Companies do not have anything close to certainty that good scores on their alignment tests actually mean a good model: Models might detect when they are being tested, and behave differently when they’re deployed.
- ✓Sam Altman
we should not train models that we cannot make an extremely strong case that we will be able to keep under human control
- ✓Jensen Huang
then I think the answer is that we have to shut the labs down. Because the cost to humanity, the damage is too great.
- ✓Ryan Lowe
Ryan Lowe calls for nuclear-style layered systems safety at the labs
Disagree 3 people
- ✗Jensen Huang
The fact of the matter is, there are plenty of incentives for them to do it right. So I have to disagree with your premise that somehow somebody’s pushing them to do this. Nobody’s pushing them to do this.
- ✗Zvi Mowshowitz
The Huang Rule is actually way too harsh.
- ✗Sam Altman
“the world should accept some bad things happening” for the technology’s benefits
No clear side 3 people
- ~Anthropic and OpenAI
acknowledging the challenges of making them as rigorous for AI models as for aviation or nuclear power, due to the emergent complexity at each new level of AI capability
- ~Sam Altman
It’s certainly incorrect to talk about this only as an engineering problem.
- ~Zvi Mowshowitz
Either Kimi models are capable enough to provide uplift with bioweapons and assassinations, or they’re not. There is no ‘it is capable enough but no one can jailbreak it.’
Side unclear 1 person
- ·Jensen Huang
I wouldn’t be surprised if the amount of compute necessary to develop these models increased by a factor of 10, because the evaluation is so rigorous.
Week by week
- · 3: Ryan Lowe ✓, Sam Altman ✗, Zvi Mowshowitz ~
- · 3: Anthropic and OpenAI ✓, Sam Altman ~, David Robinson ✓
- · 3: Sam Altman ✓, Jensen Huang ✓, Zvi Mowshowitz ✗
Built 2026-11-21 06:50 UTC from the news knowledge
graph by code. Every quote was checked against the source's own words; which side a quote takes is a model's reading,
made twice.