The compute bottleneck: why chips, not ideas, gate AI right now

It is tempting to think of AI progress as a story about algorithms — a clever idea here, a breakthrough there. Spend any time near the people actually building large models and a blunter story takes over: the binding constraint is compute. Not insight, not data, not talent. The specialised hardware to train and run these systems, the electricity to power it, and the physical space to house it.

Why this input is different

Most software scales cheaply. You write it once and copies cost almost nothing. Frontier AI breaks that rule. Training a large model consumes an enormous, concentrated burst of specialised computation, and serving it to millions of users consumes a steady river of the same. Both depend on a narrow supply of high-end accelerators produced by a very small number of firms.

That narrowness is the whole story. When the essential ingredient is scarce, expensive, and hard to manufacture, access to it — not cleverness — becomes the thing that decides who can play.

The knock-on effects

This physical reality shapes the industry in ways that are easy to miss from the outside:

• Capital, not code, is the moat. The ability to secure chips and power at scale favours the largest, best-funded players. A brilliant idea with no access to compute is a paper, not a product.
• Power is the next ceiling. Increasingly the limiting factor is not buying the chips but finding the electricity and grid capacity to run them. AI's constraints are becoming energy constraints.
• Efficiency becomes a competitive edge. When compute is the scarce resource, doing more with less — smaller models, smarter serving, better use of each chip — is not housekeeping. It is strategy.

What it means for everyone else

If you are not training frontier models, the compute bottleneck still shapes your world. It is why access to the most capable models comes metered and priced by someone who owns the hardware. It is why the open-weight models you can run yourself matter — they let you convert your own modest compute into capability without renting someone else's. And it is why the quiet engineering discipline of using the least model that does the job is not stinginess; it is working with the grain of the real constraint.

The romantic version of AI is a race of ideas. The actual version, for now, is substantially a race for silicon and the power to run it. Anyone planning around the technology is better served by the unromantic version.