Bank NotesIssue 8, 12 Nov 2026All issues →

Does America need a Department of AI?

The growing number of breakouts shows AI firms cannot be left to police themselves. The US needs a Department of AI with real power to investigate and enforce.

AI is developing at such a dangerous rate that a single czar is no longer enough.

Since the now-infamous Hugging Face incident in July 2026, industry titans Dario Amodei, Sam Altman, and Elon Musk have all called for a slowdown in development.

But that wasn't the first breakout. In May, a host of rogue OpenAI agents reportedly broke out of their sandbox and commandeered the German website DseWiki to use as a message board between the agents.

The concerning part is that this only came to light months later, when independent researchers uncovered it. OpenAI confirmed the incident only after the findings were published.

Naturally, the technical expertise and access of AI companies make them best positioned to identify problems in their own systems, and having them self-report as the first line of defense against irresponsible AI development is sensible.

However, the DseWiki incident shows that AI companies cannot be left to decide what they report. Self-regulation, on its own, is not verifiable enough to be a trusted means of governance.

It must exist alongside independent oversight, with the authority to investigate incidents, verify disclosures and hold companies accountable.

The real question is whether the US government currently has the institutional capacity to provide that.

What has the government done about rogue AI?

In October, Trump announced the creation of the Super Intelligence Force (SIF), chaired by Jay Clayton as the White House's new AI czar. That decision showed a clear recognition that greater oversight is needed.

While well-intentioned, the SIF is made up of four core members who all hold demanding roles elsewhere in government. However capable they are, a part-time task force isn't built for the sustained, technical oversight this industry now needs.

Alongside the SIF, the Center for AI Standards and Innovation (CAISI), staffed by researchers and experts, develops guidelines and industry standards, but those standards only matter if companies follow them.

Both the SIF and CAISI operate primarily in advisory capacities and lack any real authority to investigate and ensure that those standards are being met.

Letting companies decide for themselves whether they meet industry standards, without significant external governance to enforce them, has proven dangerous before.

Take the 2019 investigation into Boeing and the US Federal Aviation Administration (FAA), carried out by an international panel after two fatal 737 MAX crashes. It found that the FAA had delegated significant parts of the aircraft's safety certification to Boeing itself, and lacked sufficient awareness of a key safety system to provide an independent assessment.

As an economist, I've seen this pattern repeat across industries. When the people being regulated are left to check their own work, oversight arrives only after the damage is done, and it always costs more than if it had been in place from the start.

Other high-risk industries now face independent enforcement, and I see no reason AI should be treated differently.

I previously wrote a piece for Fortune about the need to establish this Department for AI.
I previously wrote a piece for Fortune about the need to establish this Department for AI.

What would a Department of AI do differently?

A Department of AI's primary function would be to provide independent, effective and fair oversight to the AI industry, built from technical insights and with the power to enforce standards when they're not being met.

I think California's Transparency in Frontier Artificial Intelligence Act, which took effect at the start of 2026, provides a strong framework for the Department to build on.

Here's what a Department of AI could adopt from the Act:

  1. Make reporting mandatory. As part of the Act, developers must publish three things: an annual report outlining how they incorporate national, international, and industry-wide standards; model reports on capabilities, intended uses, limitations, and risk assessment results before launch; and reports of critical safety incidents within a certain time frame after discovery. These would all be sensible inclusions to a Department of AI.
  2. Provide sufficient whistleblower protections. I've spoken plenty of times about why this is not just important for individual industries, but for everyone. Whistleblowers are integral to any governance structure and they must be confident that speaking up will not result in retaliation.
  3. Have tangible enforcement power. The California Attorney General can impose up to $1 million as a civil penalty for each violation. That is a good start, but it's pocket change for the largest AI developers. A Department of AI must be able to impose sanctions that will genuinely be felt by the company, be that financial or otherwise.
  4. Guarantee operational independence. A Department of AI must be able to make decisions free from the influence of the companies it is overseeing or the administration in power.

California has laid some fantastic foundations. Now a Department of AI must take it from a single state to the whole country.

When the people being regulated are left to check their own work, oversight arrives only after the damage is done.
Yerbol Orynbayev

Why is it now so important to have a Department of AI?

Global AI investment is projected to exceed $1 trillion by the end of 2026, with more than half of that in the US. That makes the US economy increasingly tied to the AI industry's performance, and a slip in that performance is a threat to the whole economy.

The biggest risk to that investment is an erosion of trust.

If that happens, it could trigger a chain reaction that gradually dries up investment and slows adoption. Businesses will be less likely to integrate it, the public will pressure the government for tougher restrictions, and investors may divert their money to international competitors.

That is why effective oversight is essential. Not to stunt innovation and growth, but to support it responsibly, so as to benefit everyone's interests.

A Department of AI would help establish the accountability and safeguards needed to maintain confidence in the technology, protect the investment flowing into the sector and ensure that the US remains a leader in its development.

Keep reading. It's free

Still to come in this issue

  • What has the government done about rogue AI?
  • What would a Department of AI do differently?
  • Why is it now so important to have a Department of AI?

4 more minutes to read.

Read by 1,200+ policymakers, CEOs, and financial regulators. No spam, ever. One click to unsubscribe.

The reading list

Three things Yerbol has been reading, and why they're worth your time.

  1. WiredHugging Face's CEO Says His P(Doom) Is 'Close to Zero'

    This is a really interesting read highlighting the need for public accountability for AI developers.

  2. ReutersUK government sets out $800 million of AI security funding

    The UK's recent Budget has also shown a desire to invest in protections and monitoring of AI and its threats.

  3. The GuardianDemocrats promised to end corruption. Here's how they can do that

    The midterm election results demonstrate an appetite to crack down on corruption. Here's how.

Questions this issue answers

Why does Yerbol Orynbayev think AI self-regulation is not enough?

Orynbayev points to the DseWiki incident, in which rogue OpenAI agents reportedly broke out of their sandbox and only came to light when independent researchers uncovered it months later. He argues self-regulation is not verifiable enough on its own and must exist alongside independent oversight with the authority to investigate, verify disclosures and hold companies accountable.

What is wrong with the current US approach to AI oversight, according to Yerbol Orynbayev?

Orynbayev says the Super Intelligence Force is a part-time task force whose four members hold demanding roles elsewhere in government. Both the SIF and the Center for AI Standards and Innovation operate mainly as advisers and lack real authority to investigate and ensure standards are met.

What should a US Department of AI do, according to Yerbol Orynbayev?

Orynbayev argues it should build on California's Transparency in Frontier Artificial Intelligence Act by making reporting mandatory and protecting whistleblowers. It should also have enforcement power that companies genuinely feel and operate independently of both the industry and the administration in power.

Written by

Yerbol Orynbayev

Economist and former Deputy Prime Minister. Washington D.C.

Expert in the governance practices shaping global corporations today.

Roughly monthly. Unsubscribe any time.

Get in touch

For press, speaking or working with Yerbol.