P(doom) Gamified: AI Risk Begins to Have a Price

Let's lay out some numbers first. In public materials, Jan Leike's P(doom) estimate is 46%, and Paul Christiano's is 40%. In other public interviews, someone pushed the path to 90%, and another gave a near-extreme expression of 99.999999%. On the other side, some frame the possibility of AI killing humans at 10% to 20%.

These numbers usually drift around in the AI safety circle like an emotional thermometer. Now they've been turned into P(doom), a DOOM 64-style parody. That line in the screenshot, SAFETY OVERRIDES OFFLINE, feels like pressing the industry's most sensitive button right in front of players.

I don't think this should be treated just as a joke. When a concept starts being made into games and memes, it often means it has moved beyond papers and conferences into public narrative.

AI safety used to be internal jargon for a few researchers. P(doom) is a probability, but it also carries anxiety and stance. Now it becomes an interface, an operable object, reflecting that risk is starting to have a price.

Analogous to finance, tail risks are hard to predict but can be priced. Brokers don't just ask about crash probability; they ask about trigger conditions, liquidity, and position variables. The problem with P(doom) is the same. A single number has no trading value; conditional statements do.

Don't just tell me your P(doom), tell me your conditions.

This sentence sounds very analyst-like. The difference between 46% and 90% falls on the premises. Whether models can act autonomously, bypass tool restrictions, have audit logs and red-teaming—all change the valuation.

A paper discusses why experts disagree greatly on existential risks, and one conclusion is heartbreaking. Risk perception correlates with familiarity with alignment literature. Many people argue from different coordinates; some haven't even seen the map yet. This is particularly dangerous for capital markets. Primary markets are more prone to listening to stories; secondary markets look at delivery. If the safety narrative is just emotion, valuation repair logic will be repeatedly interrupted by policy, accidents, or model releases.

That 2016 CMU work already had AI agents playing Doom via vision. Back then, it only processed pixels, accessing the game engine via API. Today, models can write code, call tools, and control browsers. Game boundaries are just cheaper experimental fields. Recently, I used Claude Code to build research report workflows, used AI review for compliance initial screening, and tried DAMO LiON for retrieval. The more I use it, the more I feel that safety is an interface in the delivery phase. The stronger the model, the more it needs boundaries.

The competitive landscape is changing too. Top model companies previously competed on parameters and chips, and inference costs. Next, they will compete on who can package risk into enterprise contracts. Cloud vendors, chip manufacturers, model factories, regulators, and third-party auditors will form a middle layer. Invisible watermarks, red-teaming, model cards, log retention, and content tracing—these sound like compliance costs but might actually become barriers to entry. The more expensive compute gets, the more expensive error tolerance becomes. Enterprises won't accept unauditable systems just because chips are cheap.

When I wrote about NVIDIA acquiring Hugging Face the other day, I mentioned that model distribution rights are starting to be priced. Following this logic, safety rights will also be priced. Whoever can prove a model is auditable, rollback-able, and accountable will secure long-term budgets from enterprise clients.

Turning P(doom) into a game, beneath the satirical shell, is risk pricing. It pulls AI existential risk from forum arguments into an interactive public space. Players don't need to believe in the apocalypse to see that once model capabilities enter production environments, risk lands on costs.

Looking ahead, I don't think P(doom) will become a regulatory metric. It's too subjective, too easy to take sides. More likely, it will be broken down into a set of procurable, auditable, and measurable engineering metrics, such as red-team coverage, tool call permissions, jailbreak interception rates, log completeness, and incident response times. Over the next two years, AI safety will first land in enterprise procurement terms, then enter compliance products, and finally feed back into model architecture. Turning safety into delivery capability might be easier to monetize than shouting about the apocalypse.

---

📌 This article is compiled from Hacker News. Original source: https://p-doom.transitivebullsh.it

Copyright belongs to the original authors. This is a compilation and independent analysis based on public reports.