{"id":22495,"date":"2026-07-19T14:11:14","date_gmt":"2026-07-19T14:11:14","guid":{"rendered":"https:\/\/scannn.com\/demis-hassabis-on-the-new-coming-age\/"},"modified":"2026-07-19T14:11:14","modified_gmt":"2026-07-19T14:11:14","slug":"demis-hassabis-on-the-new-coming-age","status":"publish","type":"post","link":"https:\/\/scannn.com\/lv\/demis-hassabis-on-the-new-coming-age\/","title":{"rendered":"Demis Hassabis on the New Coming Age"},"content":{"rendered":"\n
\n

Google CEO Demis Hassabis offered us a first rate second rate essay, <\/span>A Framework for Frontier AI and the Dawning of a New Age<\/a>. I\u2019ll go over that essay and various responses to it in Part 1.<\/span><\/p>\n

Part 2 of this post then covers Alex Turner\u2019s resignation, and his story about how he tried and failed to prevent Google from signing up to allow the Department of War to use its models for essentially whatever the government wants, including autonomous weapons. <\/p>\n

Demis Hassabis sold DeepMind to Google on condition that something like this would not happen. Yet here it is, happening. A cautionary tale. <\/p>\n

I will cover Kimi K3 tomorrow. I am hoping to know more by then. Please do share any reactions or info about it in the comments here. <\/p>\n

\n
<\/p>\n
<\/picture><\/div>\n

<\/a><\/figure>\n<\/div>\n

He saying we are standing in the foothills of the singularity. <\/p>\n

His ask is a Frontier AI Standards Body within the US Government, similar to FINRA, that would govern \u2018frontier labs,\u2019 defined as any company that produces a frontier model based on various technical benchmarks. Evaluations would be updated regularly, and vulnerabilities would be addressed, both before and after release. <\/p>\n

He is excellent about stating that this is <\/span>big, really big, no bigger than that, it be big<\/a>.<\/span><\/p>\n

\n

Demis Hassabis: I\u2019ve spent my whole life working on AGI because I\u2019ve always had a deep conviction that, if built and deployed responsibly, it would prove to be one of the most beneficial and transformative technologies ever invented. AGI cannot be compared to standard technological breakthroughs, not even ones as consequential as the internet or mobile – it is much more akin to the discovery of electricity or fire. If you stop to think about it, we\u2019ve essentially found a way to make sand think. It\u2019s miraculous.<\/p>\n

The magnitude of this technology\u2019s impact will be unprecedented, perhaps 10x of the Industrial Revolution at 10x the speed. It will help us solve some of the biggest problems society faces from accelerating drug discovery to developing new clean energy sources to creating novel advanced materials. We could even reach a point where resources are no longer the limiting factor for human progress, leading to an amazing new era of abundance.<\/p>\n<\/blockquote>\n

There is definitely a \u2018don\u2019t say the thing\u2019 aspect of this, where he won\u2019t name what the downside risks actually are. When Demis says \u2018experts disagree\u2019 he is rather avoidant about the way in which they disagree here. <\/p>\n

\n

Nate Soares<\/a> (MIRI): I\u2019m glad Demis acknowledges that this is a \u201cpivotal moment in human history\u201d during an \u201cextremely intense\u201d race. I\u2019m disappointed that his proposed solution is a \u201cstandards body\u201d to evaluate whether models are dangerous, with no plan for what to do once they are.<\/span><\/p>\n

I\u2019m glad he acknowledges that \u201cexperts disagree.\u201d I\u2019m annoyed that he glosses past how the disagreement is about whether there\u2019s a ~5% or \u226550% chance of total catastrophe. We\u2019ve gotta do better.<\/p>\n

Aaron Scher<\/a>: Glad to see AI CEOs speaking publicly about their views on AGI. I think Demis is wrong about his policy prescription: it\u2019s far too little too late. When he says the experts disagree, he means that some think 5% this tech kills literally everybody, some at 40%, some at 90%.<\/span><\/p>\n<\/blockquote>\n

Clearly this is strategic, but if you don\u2019t already know, or are looking to not realize, it is very easy to come away thinking that Demis does mean the effect on jobs, even though when he says \u2018safely\u2019 he very much does not (primarily) mean that.<\/p>\n

\n

Demis Hassabis: \u2026 On the horizon, we will need robust safeguards to maintain control of increasingly agentic, recursively self-improving systems – and tackle unknown issues that will only become clearer over time.<\/p>\n

\u2026 I\u2019ve always believed in the power of human ingenuity and creativity to solve any problem. I\u2019m confident that mitigating the technical risks related to AI is a challenge we can collectively address, but only if we give ourselves the time and space to get this next crucial step right. Currently, as a field and as a wider society, we aren\u2019t doing that.<\/p>\n<\/blockquote>\n

He makes clear part of this is about giving us options, including for a slowdown.<\/p>\n

\n

The strength of this approach is it would be technically focused, while at the same time supporting innovation and incentivising responsible behaviour. It is designed to keep up with the field\u2019s acceleration and adapt to the biggest risks as they are identified, and could be ratcheted up if the seriousness of the situation demands, including coordinating a slowdown in development among the Frontier Labs if deemed necessary.<\/p>\n<\/blockquote>\n

Demis keeps it short, not offering many details. To the extent that he has laid out a proposal, it seems to be a good one. It is definitely an improvement on the margin.<\/p>\n

\n

Jack Clark<\/a> (Anthropic): At this point, everyone at the frontier of AI agrees that third-parties should test out AI systems and use these to develop standards to feed into policy – excellent to see @demishassabis laying out a framework to do this!<\/span><\/p>\n

Samuel Hammond<\/a>: It is striking to see leadership at Google, Anthropic, OpenAI and Microsoft all fairly independently sounding warning alarms about an imminent technological acceleration.<\/span><\/p>\n<\/blockquote>\n

Thus I file this post and its ask, as high praise, under \u2018the least you could do.\u2019 <\/p>\n

I <\/span>agree with Peter Wildeford<\/a> that while better than nothing FINRA is not a great model here, with heightened risk of regulatory capture, and not a substitute for full government action. You need an SEC to your FINRA. That doesn\u2019t mean don\u2019t make the FINRA. It does mean you still need the SEC. <\/span><\/p>\n

Would such a (at least partly) voluntary regime, only for models intended for release, and without a related binding intentional agreement, be sufficient to solve the problem? No, again it\u2019s just way better than doing nothing, as Peter Wildeford and many others noted. <\/p>\n

You do not need to believe, as Aaron Scher and Connor Leahy do below, that only a full halt would be sufficient here, to know we have a long way to go. Demis\u2019s statements here, if you know what they actually mean, imply a level of danger and urgency that is not reflected in the proposal. <\/p>\n

\n

Eli Tyre<\/a>: > Initially, Frontier Labs would voluntarily share models with the Standards Body for review up to 30 days before release.<\/span><\/p>\n

Is this proposal only intended to address risks from models that companies plan to release? If a company develops a frontier model and never releases it, only deploying it internally to develop even more powerful AI capabilities, are they thereby exempt from this oversight scheme?<\/span><\/p>\n

Connor Leahy<\/a>: While @demishassabis is right that we need urgent action to address risks as we approach AGI (and superintelligence, I\u2019d add), the correct response to the threats is not a \u2018self-regulatory organization\u2019.<\/span><\/p>\n

We need to prohibit superintelligence, not give industry regulatory power.<\/span><\/p>\n

Aaron Scher<\/a>: \u2026 The extinction threat, the \u201conly a few short years\u201d, the \u201c10x the Industrial Revolution\u201d\u2014these aren\u2019t indicators that point to \u201clet\u2019s evaluate models to understand their capabilities and have voluntary safety standards\u201d. We need to back off, we need to halt the creation of ASI.<\/span><\/p>\n

Point 2: I agree with the attached quote that we need more time. But I think Demis\u2019s optimism is a vibe, not a trustworthy basis for predictions. <\/span>Rob Miles says it best in this video<\/a>, if an asteroid we\u2019re headed earth\u2019s way 200 years ago, we\u2019d just die <\/span><\/p>\n

Point 3: As others have pointed out, it\u2019s not clear that this proposal would reduce risks from internal deployment (it seems to focus on public deployment and pre-deployment testing), but internal deployment is where much of the risk is.<\/p>\n

Point 4: I don\u2019t think the proposed body could actually enact, verify, and enforce a slowdown; there\u2019s ambiguity about what\u2019s voluntary. Again, I think we need a long-term international treaty and to actually back off, not just to slow down a little.<\/p>\n<\/blockquote>\n

Timothy Lee dissents on the theory that we have already discovered<\/a> more than 10% of available technological value from superior intelligence and information by 1945, as weighted by living standards. I don\u2019t think that is the only way to measure impact. If it is, then it depends on if you mean at a given point in time or over a lifetime, and how \u2018still human\u2019 you are restricting the experience to being. <\/span><\/p>\n

In terms of moment-to-moment lived experience as a basically human entity, measured the way we would for the quality per year in a QALY measurement, if you assume a mostly fixed lifespan and path of aging, I think it is likely we are more than 10% of the way to optimal. However, I would take a cure for aging and disease, <\/span>plus the internet<\/a> (and computers and phones) which people underestimate bigly, plus not having to work a job like those in 1945, over everything that came before, by a very wide margin, let alone full transhumanist experiences. Something about heaven and Earth and your philosophy, even if things stay relatively \u2018normal\u2019 afterwards. <\/span><\/p>\n

Whereas if we\u2019re just looking at pure GDP numbers? That seems super doable.<\/p>\n

\n

<\/span>Noah Smith<\/a>: The Second Industrial Revolution lasted roughly 100 years — 1870 to 1970. In that time, living standards in the U.S. quintupled.<\/span><\/p>\n

10x the Industrial Revolution in 1\/10th the time would be a 50x increase in living standards in 10 years — a 48% annual GDP growth rate.<\/p>\n<\/blockquote>\n

The stupid dissent, espoused here by Coinbase CEO Brian Armstrong<\/a>, is \u2018oh AI is like software so we don\u2019t even need an SRO, existing laws and incentives are fine.\u2019 <\/span><\/p>\n

DeepMind also released <\/span>this three page statement on their approach to bioresilience<\/a>, which can serve as an example test of their ability to self-govern. <\/span><\/p>\n

\n

The program has three pillars:<\/span>
\u25cf Prevent threat actors from misusing our models.<\/span>
\u25cf Detect new outbreaks quickly.<\/span>
\u25cf Respond decisively to these outbreaks.<\/span><\/p>\n<\/blockquote>\n

They are using a variety of methods to prevent misuse via defense-in-depth, of which they are (at least somewhat appropriately) vague. <\/p>\n

At least one of these four must be true:<\/p>\n

    \n
  1. \n

    Their methods don\u2019t work.<\/p>\n<\/li>\n

  2. \n

    Their methods only work because Gemini is not good enough.<\/p>\n<\/li>\n

  3. \n

    Anthropic is going nuts with its classifiers for no reason.<\/p>\n<\/li>\n

  4. \n

    Anthropic\u2019s countermeasures are very incompetent in an expensive way.<\/p>\n<\/li>\n<\/ol>\n

    The second pillar, detecting outbreaks, is something the world should be doing anyway, but it mostly isn\u2019t. DeepMind is looking for ways to help, but not on the level of \u2018we are taking this responsibility on ourselves.\u2019 That\u2019s not an entirely reasonable thing to ask even of Google, but I\u2019d look into it. <\/p>\n

    The third pillar is again important, a place where the world is not prepared, and not exactly Google\u2019s department. They are trying to help somewhat via use of AI, but most of what must be done at this capability level is ordinary prevention and preparedness work that our government has decided not to do. <\/p>\n

    Thus, Google suggests that policymakers get off their collective asses and do the second and third things via various new laws and actions. Without looking into their proposed details I am confident we should at least do these things. <\/p>\n

    They also call for a federal framework to judge risk-benefit tradeoffs in bio for frontier models, which would presumably be a complement to Demis\u2019s proposal. Good.<\/p>\n

    After heroic efforts to stop Google from signing an \u2018all lawful use\u2019 deal with the Department of War, <\/span>Alex Turner resigns from Google DeepMind<\/a> because they broke their promise and sold AI to the military with no restrictions against killer robots or mass spying. Lots of people did not raise their voice and justified it because they said it would not happen. Then it happened. So far only Alex Turner is quitting in response.<\/span><\/p>\n

    One thing I want to be clear on here is that, even though Jeff Dean failed to use his full leverage on this issue, Jeff Dean went farther than most anyone else. I don\u2019t want anyone to focus their blame on Jeff Dean for ultimately not opposing this more strongly, as opposed to all those who did nothing at all. <\/p>\n

    I do think Demis Hassabis is blameworthy here. He continues to insist that DeepMind\u2019s principles are unchanged and unviolated. This is not the case. <\/p>\n

    If Hassabis was overruled by Google CEO Sundar Pichai or otherwise had no choice, perhaps due to other leverage brought against Google, that happens and Demis Hassabis is not a king. I would have accepted that, although it would have showed that his strategy of steering Google was failing. Even then, it does not excuse pretending that nothing happened. <\/p>\n

    Andreas Kirsch speaks plainly and offers a similar read<\/a>. <\/span>Andreas works at DeepMind, and has made clear<\/a> that all the safeguards promised in 2014 have now been erased. More than 600 colleagues signed an open letter asking Google not to sign this contract, and then Google went ahead and still signed the contract. As a lawyer pointed out and as should be clear to anyone who follows the issue, <\/span>Google\u2019s contract terms here are completely ineffective and the government can do whatever it wants<\/a>.<\/span><\/p>\n

    \n

    Andreas Kirsch<\/a>: None of this is new. DeepMind\u2019s governance record:<\/span><\/p>\n

    2014: independent ethics board \u2014 reportedly a condition of the sale<\/span>
    2015: one informal meeting; then effectively abandoned<\/span>
    2018: AI Principles exclude weapons & surveillance<\/span>
    2025: exclusions dropped<\/span><\/p>\n

    2026: Pentagon contract <\/p>\n

    Demis Hassabis then made a different bet: \u201cTake the energy that was going into the trustless negotiation and put it into creating real trust.\u201d<\/p>\n

    And: \u201cSafety isn\u2019t about governance structures.\u201d<\/p>\n

    Trust instead of governance. The Pentagon contract is the litmus test of that bet.<\/p>\n<\/blockquote>\n

    You can read the <\/span>Twitter thread version<\/a>, or <\/span>the 25-page full version<\/a> of what happened. The whole post is telling, but long post is long. I will quote from the thread. <\/span><\/p>\n

    \n

    Alex Turner<\/a>: What hurt most was watching AI ethics leaders do nothing. When the Pentagon threatened Anthropic, I was at a conference held by the International Association for Safe and Ethical AI. I lobbied IASEAI leaders to act.<\/span><\/p>\n

    What hurt most was watching AI ethics leaders do nothing. <\/span><\/p>\n

    When the Pentagon threatened Anthropic, I was at a conference held by the International Association for Safe and Ethical AI. I lobbied IASEAI leaders to act.<\/span><\/p>\n

    On stage, IASEAI leadership announced a member vote on making a statement supporting Anthropic. ~Unanimous show of hands supported it. But the vote vanished. IASEAI went silent, ignoring two months of opportunities to sway Google.<\/p>\n

    Inside Google: I organized a petition to Chief Scientist Jeff Dean asking him to fight. Over 250 GDM employees signed. I also asked Jeff to sign an amicus brief backing Anthropic. He did, which was awesome.<\/p>\n

    \u2026 I proposed that Jeff head the potential Review Body, but he didn\u2019t champion the proposal. In the end, I don\u2019t think he used his leverage. I think he could have stopped the deal but didn\u2019t. As a last attempt, I directly messaged Google DeepMind\u2019s CEO, Demis Hassabis. He told me to send my proposal to two senior policy staff. They let the proposal wilt unattended until Google signed the deal.<\/p>\n

    In 2018, Demis, Jeff, & DeepMind itself pledged to neither \u201cparticipate in nor support the development [..] or use of lethal autonomous weapons.\u201d<\/span><\/p>\n

    Google signed, yet these pledgers remain. I see three honest paths: explain, renounce the pledge, or quit. Silence isn\u2019t one of them.<\/span><\/p>\n

    Here\u2019s a path Demis has chosen: deny anything has changed. He claims \u201cnothing\u2019s changed about our principles\u201d, yet he co-authored the post removing the weapons prohibition from Google\u2019s AI Principles. Both cannot be true.<\/p>\n

    Many hope that when things get bad enough, someone powerful will say \u201cno.\u201d I tested that for months. Anthropic defended its red lines, but most did not. Pledges of conscience often vaporize on contact with power.<\/p>\n<\/blockquote>\n

    More specifically:<\/p>\n

    \n

    The way I saw it was: I don\u2019t need to get Jeff to agree to quit over this. In a sense, he\u2019d <\/span>already<\/em> agreed to quit over it. He had signed a pledge in 2018:<\/span><\/p>\n

    <\/span>Lethal Autonomous Weapons Pledge<\/a><\/strong><\/p>\n

    Signed by 5,218 people<\/p>\n

    The decision to take a human life should never be delegated to a machine. There is a moral component to this position, that we should not allow machines to make life-taking decisions for which others\u2060\u2014or nobody\u2060\u2014will be culpable. There is also a powerful pragmatic argument: lethal autonomous weapons, selecting and engaging targets without human intervention, would be dangerously destabilizing for every country and individual. [\u2026]<\/p>\n

    By removing the risk, attributability, and difficulty of taking human lives, lethal autonomous weapons could become powerful instruments of violence and oppression, especially when linked to surveillance and data systems. [\u2026]<\/p>\n

    We, the undersigned, call upon governments and government leaders to create a future with strong international norms, regulations and laws against lethal autonomous weapons. These currently being absent, we opt to hold ourselves to a high standard: we will neither participate in nor support the development, manufacture, trade, or use of lethal autonomous weapons.\u2019<\/p>\n

    Signers include: Google DeepMind (the organization), Demis Hassabis, Shane Legg (cofounder, <\/span>now Chief AGI Scientist<\/a>), Raia Hadsell (<\/span>VP of Research<\/a>), Jay Yagnik (<\/span>VP and Engineering Fellow at Google, leading large parts of Google AI<\/a>) and Jeff Dean.<\/span><\/p>\n<\/blockquote>\n

    Alas, no, that did not translate into willingness to quit. <\/p>\n

    Many at Google DeepMind made various efforts to avoid this outcome. They failed.<\/p>\n

    The mechanisms of self-government of DeepMind failed. Its employees spoke up, some prominently, but it was not enough, and those with the highest leverage did not use it. The promises made when they sold to Google have been broken.<\/p>\n

    In the long run, this will impact DeepMind\u2019s ability to recruit, motivate and retain the best talent. Google and DeepMind are losing to OpenAI and Anthropic, despite Google\u2019s numerous large advantages. Inability to recruit and retain and properly use the best talent is a lot of why. They have some great talent in leadership, but those leaders cannot do it alone. <\/p>\n

    Demis Hassabis is to be applauded for his statement, but calls on others to do things, without any consequences following, risks becoming only cheap talk. <\/p>\n

    My hope is that it can become more than cheap talk, and move the conversation forward. It won\u2019t be easy, but we have an opportunity. Let\u2019s make the most of it.<\/p>\n<\/div>\n

    Source link <\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    Google CEO Demis Hassabis offered us a first rate second rate essay, A Framework for Frontier AI and the Dawning of a New Age. I\u2019ll go over that essay and various responses to it in Part 1. Part 2 of this post then covers Alex Turner\u2019s resignation, and his story about how he tried and […]<\/p>\n","protected":false},"author":16,"featured_media":22496,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[143],"tags":[],"class_list":["post-22495","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai"],"_links":{"self":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/22495","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/comments?post=22495"}],"version-history":[{"count":0,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/posts\/22495\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media\/22496"}],"wp:attachment":[{"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/media?parent=22495"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/categories?post=22495"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/scannn.com\/lv\/wp-json\/wp\/v2\/tags?post=22495"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}