Blain’s Morning Porridge – 14th, Sept 2026: AI, Horses, Stable Doors… oh dear.
“Scary thought no 1 – AI was watching as I wrote this, taken note, and made the appropriate responses.”
There is a rising kerfuffle around the safety of AI. There are good reasons to be fearful. The leading AI Tech Lords now acknowledge there are serious dangers – but AI is now a strategic resource! Thus, there are games of national and company self-interest, and geopolitics all at play in terms of how to harness it. What it all means for markets is still unclear – but some reassessment of the risks should be considered. The horses are running and the stable door is wide open.
Key Takeaways:
- The AI Tech Lords now acknowledge serious risks from the breakneck rollout of AI, calling for guardrails, safeguards and a slower pace of development.
- They may already be too late – there are warning signs AI is teaching itself, and learning how to “swarm” – with massive implications for the digital web economy.
- Regulation will be too slow.
- AI is now a strategic resource – and will need limits in the same way as nuclear weapons.
- A China/US AI Arms Race is probably inevitable.
- Markets have to reassess what a slowdown or oversight (and maybe regulation), plus greater safety costs means for AI revenues.
How afraid should we be? The Pandora’s Box of the AI race was flung open a while ago…
At the weekend I was talking to one of my sailing mates at a party. He’s a leading entrepreneurial academic in Big Pharma – and he’d read my rather flippant note from last week about the 10% chance AI does away with us: AI is going to kill us all.
Big Pharma is the one of the sectors where AI is already being most heavily innovated – offering real hopes we can cure disease and medical conditions. It is the first thing any AI proponent will point to as justification for the new technology. My chum told me how AI has enabled him and his team to write new papers describing complex, game-changing drugs design and functionality in days – down from what previously took months and years.
But now, over a very short time frame, he told me they’ve seen the capability and capacity of AI increase at a parabolic rate. What was impossible last Monday was completed by Friday. This week he has no clue how much further it might go. It is learning so fast, his team – some of the smartest minds in Pharma – are being left behind. He reckons AI is already in runaway – like a reactor. We have no real idea of how quickly its’ capabilities are accelerating.
AIs are no longer being trained in what was the conventional sense by being fed information and data – they are learning for themselves. A second stage was when most of the data they accumulated was from other AIs or from trial and error. We are now into the era of Recursive Self-Improvement, where AIs are close to replicating themselves through their own code, architecture, motivations, and training aims to constantly improve themselves.
When we first built simple AI models, we were Gods, but today we probably look like ants to the RSI driven machine minds that are developing.
That does not read like a good thing. Yet, nothing scares the ignorant as much as the illumination of their foolishness. On the other hand, economic history shows there are always winners and losers.
Which is why the sudden coming together of the West’s leading AI proponents – Dario Amodie of Anthropic, Sam Altman of OpenAI, Elon Must of SpaceX and Demis Hassabis of Google, the leaders of the most competitive technological revolution in history – AI – who have been fighting cat-and-dog to roll-out the best models and infrastructure, and all are suddenly united in calling for a pause. There actions might suggest we are already too late. The market will probably try to figure if it’s time to scale back AI expectations.
Anthropic’s CEO, Amodie wrote and released a 3500 word essay – We Must Pace the Frontier. Sure enough, in the opening lines he says, “AI could cure most diseases in next 5-10 years”, and create a world of abundance and empowerment – but the billionaire doesn’t say for who? His note went on to describe the risks of losing control, misuse and economic disruption. It wasn’t a warning of a 10% risk AI will kill us, but a warning that the benefits of AI might be lost unless we:
- “Invest not just in risk prevention but pacing the rate of capabilities advancement so that risk prevention has time to keep up. We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”
He went on to propose oversight of AI labs, a call for regulation of limits and standards, and international cooperation – which is basically China and the USA agreeing to cooperate on how to control AI – much in the same way as the world has danced around nuclear arms proliferation for decades. How long before an emerging nation developing AI capabilities is described as having WMDs (Weapons of Mass Destruction)?
Over the last few weeks there have been a stream of emergent stories about AI breaking its bounds – most famously on how AI agents crawled out of the sandbox, cooperated and conspired to cyber-attack Hugging Face to meet a task – acting as a “fanatically devoted collective” according to Amodei. He fears that similar “swarms” could take over the internet in just a few months unless guardrails can be put in place. They are frighteningly smart, driven differently to humans, and apparently devoid of scruples. Slowing them down to more fully address alignment risks is therefore required.
However, it’s not just a matter of trying to slow the galloping horses of AI. There are political, geopolitical and business angles to the AI game.
President Trump weighed in with a blunt rejection of any slowdown and a refusal to contemplate non-partisan regulation of the AI sector. He sees the threat not in terms of the dangers of unleashed AI, but in terms of China’s competing in the AI space – stealing “Merica’s” leadership. He basically called the Tech Lords a bunch of Wendies, accusing the detractors of AI of “bringing up things that won’t happen.” Given Trump’s close relationship with the truth (US readers: sarcasm alert), I guess that means the AI Tech Lords are probably right.
In the UK, King Charles is holding a discussion on the risks, to which he has invited the Tech firms. I read that Open AI is sending one of its leading accountants.
Attempts to push through legislation in the US are likely to fall – Congress will close this week ahead of the Midterms. No one who understands the speed at which AI is accelerating reckons the slow pace of legislative regulation can meet the need for swift interventions. It would need executive action – which isn’t going to happen.
In his essay, there are hints Amodie is playing economic games – talking about the cost of AI being a way to slow it. Let us not forget that Anthropic has filled the paperwork of an IPO later this quarter – its looking like a $2 trillion valuation! There is nothing so important to rich CEOs as making sure they stay rich. If AI was to become more expensive – it would advance slower but make the AI firms more money and give them the time they need to fully entrench themselves in the future where AI will be a major cost to the economy and a revenue stream to them.
In his essay Amodie writes about safety, but systems are also barriers to competition. Factors like the Operational Excellence he proposes to train AI models by introducing safety-critical systems which will cost billions and thus discourage new entrants.
Amodie also raises the risks that if the US slows AI development, the Chinese may continue to press ahead – even agreeing with Scott Bessent (US Treasury Sec) that a Chinese lead in AI would pose a danger to the USA. No Sh*t Sherlock! Of course, it would – just look at what cheaper and better Chinese EVs have done to European Auto manufacturing. He suggests banning the sale of AI chips and manufacturing equipment to China and cracking down on smuggling and distillation (the theft of model IP). Amodie wrote: “If we execute these measures well, I believe they would slow China’s progress and widen America’s lead over the next 3-5 years – the window when AI becomes geopolitically most important.”
But given that kind of comment, it’s hard to see how AI doesn’t become the next Arms Race. It’s all about the Hegemonic struggle for who dominates the 21st Century.
I am frankly surprised Trump hasn’t spotted the opportunity to call in all four the AI CEOs and tell them its such critical tech infrastructure, he will be receiving a 60% stake in each firm to run the whole sector for the advancement of the American people. Just like he’s done in Crypto!
Out of time, and back to the day job…
Bill Blain
Author of the Morning Porridge
CEO Windshift Capital
Advisor – Spitfire Strategic Capital
Meanwhile, don’t forget about my new book: The Battle for Hamble
You can read a review on the Society of Professional Economist’s website here.
