The AI Stack
The industrial machine behind the chatbot: chips, data centres, gigawatts and cloud economics, and where the profit finally settles.
Every AI answer is produced by a physical machine, acres of silicon, steel and power lines that someone had to build and pay for. The cloud is not a cloud: it is the largest industrial build-out since the railroads, and value settles where the constraint is, which is moving from chips to power to distribution.
What this argues
- The stack has six layersSilicon, data centres, power, cloud, models and apps, each with different economics.
- Power is the new bottleneckChips are easing; electricity and grid connections are not.
- Compute is commoditisingGPU rental prices collapsed 60%+, a warning for the middle of the stack.
- Value pools at the edgesScarce inputs (chips, power) and sticky outputs (apps) win; the middle gets squeezed.
- Watch the constraintWhoever owns the binding bottleneck at each moment sets the price.
What has changed
Updated November 2026
Hyperscaler capex guidance holds near $720-740B for 2026
- Alphabet (Q3 results, 28 Oct 2026): 2026 capex of $195-205B, unchanged; CFO said capex will increase significantly in 2027.
- Amazon (Q3 results, 29 Oct 2026): about $220B of 2026 cash capex reaffirmed; CEO said it expects to spend meaningfully more in 2027.
- Meta (Q3 results, 21 Oct 2026): $135-145B including finance-lease principal; 2027 plans to be shared on the Q4 call.
- Definitions differ by company (calendar year, cash, finance leases), so the sum is indicative only. Source: company earnings releases as summarised by Luminix, Oct 2026.
Microsoft trims 2026 capex as third-party capacity slips
Microsoft (fiscal Q1 2027 results, 28 Oct 2026) guided calendar 2026 capex to about $170B, down from about $175B, citing delays in third-party data-centre deliveries. It expects capex above $50B next quarter, and its CFO said demand still exceeds supply. Source: Microsoft earnings materials as summarised by Luminix, Oct 2026.
Data-centre power demand grew 17% in 2025
The IEA's April 2026 update reported data-centre electricity use rose 17% in 2025, more than five times faster than the roughly 3% growth in global demand, and that AI-related demand is expected to triple by 2030 while total data-centre demand doubles. Source: IEA via W.Media, 20 Apr 2026.
GPU rental prices have stabilised well below the 2024 peak
H100 prices were near $8/hr in early 2024. One-year contracts were about $2.35/hr by March 2026 (SemiAnalysis), and the trade-based Ornn index hit $2.94 on 25 May 2026 before easing. Nebius raised its on-demand H100 rate from $3.85 to $4.50 per GPU-hour from 1 Oct 2026, so the decline has paused or partly reversed at some providers. Sources: KuCoin News on Ornn, May 2026; IntuitionLabs, Oct 2026.
Not refreshed: the inference-cost series (Stanford HAI 2025), the capex share breakdown, GPU depreciation lives, nuclear and SMR timeline status, sovereign AI deal counts, and the 2024 baseline of 415 TWh. The 2030 figure of 945 TWh was not re-verified at source in this update.
The Stack, Mapped
Six layers, one machine
Every AI answer is produced by a physical machine, acres of silicon, steel and power lines that someone had to build and pay for. The cloud is not a cloud. It is the largest industrial build-out since the railroads.
From sand to answer: the six layers of AI
The six layers, top to bottom
| What it is | |
|---|---|
| 06 Applications | ChatGPT, Copilot, vertical tools: what users touch and pay for. |
| 05 Models | the trained intelligence: OpenAI, Anthropic, Google, Meta. |
| 04 Cloud platforms | AWS, Azure, GCP: rent the machine by the hour. |
| 03 Data centres | the buildings: racks, cooling, networking at city scale. |
| 02 Power & grid | gigawatts of electricity: the emerging bottleneck. |
| 01 Silicon | GPUs and accelerators: NVIDIA's kingdom. |
What happens when you ask an AI a question
- Prompt: your words travel to a data centre, often in another state.
- Route: a scheduler assigns your request to a rack of GPUs.
- Compute: the model's weights run in a few hundred milliseconds.
- Return: the answer streams back, costing the operator a fraction of a cent.
Training and inference stress the stack differently
| Training | Inference | |
|---|---|---|
| Shape | one giant job, months long | billions of tiny jobs |
| Needs | 100k GPUs tightly networked | GPUs near the users |
| Site | one massive campus | distributed, everywhere |
| Buys | the biggest clusters on Earth | steady, growing capacity |
The IEA expects this to more than double by 2030, and its April 2026 update shows data-centre use grew 17% in 2025 alone. The internet's physical footprint is becoming visible in national energy statistics.
Data Centres
The factories of intelligence
An AI data centre is a factory: its raw material is electricity, its machines are GPUs, and its product is computation. Thinking of it as a factory, not an office, explains every design decision.
One frontier AI campus, by the numbers
GPUs are 60% of the bill, and they depreciate fastest
View the data
| Share of capex | |
|---|---|
| GPUs & compute | 60 |
| Power & cooling | 15 |
| Building & land | 12 |
| Networking | 8 |
| Storage | 5 |
Source: Industry teardowns; hyperscaler disclosures
What makes AI data centres different
- Density: AI racks draw 5-10x the power of normal racks, so cooling becomes the design problem.
- Liquid cooling: air can no longer remove the heat; water and direct-to-chip loops take over.
- Networking: 100k GPUs must talk constantly; the network is as critical as the chips.
- Location: sites are chosen for power and water first, latency second.
- Speed: the race rewards whoever energises capacity first.
- Depreciation: the $60B of GPUs inside will be obsolete within five years.
Cooling has to escalate with chip density
- Classic racks: fans and cold aisles (air-cooled).
- Rear-door: chilled doors on racks.
- Direct-to-chip: liquid loops on the silicon.
- Immersion: servers submerged in fluid (liquid-cooled).
The brutal math of a fast-aging asset
The Power Problem
The bottleneck moves to the grid
Data-centre electricity demand more than doubles by 2030
View the data
| Demand | |
|---|---|
| 2020 | 290 |
| 2024 | 415 |
| 2030e | 945 |
Source: IEA, Energy and AI (2025)
AI moves at software speed; the grid moves at permit speed
| AI build-out | Grid build-out | |
|---|---|---|
| Timeline | months to order GPUs | 5-15 years for lines & plants |
| Capital | effectively unlimited | regulated, rationed |
| Demand / Supply | doubling every 1-2 years | grows a few % a year |
| Decision | one board meeting | regulators, communities, courts |
How the industry is chasing electrons
- Nuclear revival: hyperscalers signing deals to restart and build reactors.
- Behind-the-meter: data centres built at the power plant, skipping the grid.
- Renewables + storage: solar and batteries where land and sun allow.
- Gas turbines: fast to deploy, climate targets quietly strained.
- Grid investment: transformers and lines becoming strategic goods.
- Efficiency: each GPU generation does more per watt, but demand grows faster.
The IEA projects US data centres will use more electricity than aluminium, steel, cement and chemicals production combined. Computation is becoming the defining industrial load of the American grid.
Big tech rediscovers the atom
- First PPAs: hyperscalers sign nuclear power deals.
- Three Mile Island: Microsoft funds a reactor restart.
- SMR orders: Google and Amazon back small reactors.
- First SMRs: new small reactors targeted online.
- Gigawatt fleets: dedicated nuclear for AI campuses.
The Compute Market
Renting the machine
You no longer buy AI compute, you rent it by the hour, and the rental price just collapsed. What happened to GPU prices in two years is the most under-reported story in AI.
H100 rental prices fell by roughly three-quarters
View the data
| H100 rental | |
|---|---|
| 2023 | 8 |
| H1'24 | 5.5 |
| H2'24 | 4.5 |
| 2025 | 2.8 |
| 2026 | 2 |
Source: GPU cloud price trackers (2023-26) · values read from the chart, approximate
And the cost of serving AI fell even faster
View the data
| Cost per query | |
|---|---|
| Nov'22 | 20 |
| Mid'23 | 10 |
| Late'23 | 4 |
| Mid'24 | 1 |
| Late'24 | 0.3 |
| 2025 | 0.07 |
Source: Stanford HAI AI Index (2025); log scale · values read from the chart, approximate
Who sells compute, and how
- Hyperscalers: AWS, Azure, GCP. Full platforms, premium prices, deep pockets.
- Neoclouds: CoreWeave, Lambda, Nebius. GPU specialists, aggressive pricing.
- NVIDIA itself: sells the chips and increasingly rents them too.
- Model labs: OpenAI and Anthropic buying capacity years ahead.
- Spot markets: idle GPUs auctioned by the hour, the price floor.
- Sovereign clouds: nations buying compute as strategic reserve.
The quiet winners selling into the build-out
- HBM memory: SK Hynix and Micron, the scarce memory bolted to every AI chip.
- Networking: NVIDIA InfiniBand, Broadcom, Arista, the fabric between 100k GPUs.
- Transformers & switchgear: multi-year waiting lists; the grid's quiet chokepoint.
- Liquid cooling: Vertiv and peers, where plumbing becomes strategic.
- Construction & EPC: data-centre builders booked out for years.
- Land & fibre: sites with power, water and connectivity command premiums.
Good for users, dangerous for the middle
| Winners of deflation | Squeezed by deflation | |
|---|---|---|
| 1 | Users & apps: every query gets cheaper | Bare GPU renters: price is their only product |
| 2 | Model labs: training budgets go further | Old-chip fleets: H100s undercut by B200s |
| 3 | Adopters: new use cases become viable | Late data centres: built for yesterday's prices |
| 4 | NVIDIA: sells the next generation anyway | Leveraged players: debt against falling rates |
Nations are buying compute like a strategic reserve
Source: Government announcements; industry reporting (2024-26).
The stack's self-feeding loop
- Cheaper compute
- More AI usage
- More revenue
- More capex
- Better chips (which leads back to cheaper compute: the build-out flywheel)
Where Value Settles
The shape of the profit pool
Scarcity versus stickiness across the stack
A two-by-two map with scarcity of the input on the horizontal axis and stickiness of the customer on the vertical axis. Quadrants: top-left loyal, abundant; top-right the fortress; bottom-left commodity; bottom-right scarce, fickle. Apps sit in the loyal, abundant quadrant (high stickiness, low scarcity). Cloud sits near the centre line on stickiness and just left of centre on scarcity. Models sit near the centre. Silicon sits in the fortress (high scarcity, high stickiness). Power & grid sits at the right edge on scarcity, near the middle on stickiness. Bare GPU rental sits in the commodity corner (low scarcity, low stickiness).
What history says about infrastructure booms
- The railroad lesson. The track builders went bust; the economy that ran on the track thrived.
- The fibre lesson. Dot-com fibre was overbuilt, bankrupted its owners, then powered two decades of the internet.
- The likely rhyme. Some of today's AI capex will never earn a return, but the capacity will not be wasted.
- The investor's task. Distinguish owning the boom's spending from owning what the boom leaves behind.
The stack's iron rule: value settles where the constraint is, and the constraint is moving from chips to power to distribution.
2023's bottleneck was GPUs. Today's is grid connections. Tomorrow's is likely customer relationships.
Risks & Debates
The honest chapter
Build-out or bubble?
| The build-out case | The bubble case | |
|---|---|---|
| 1 | Real demand: usage and tokens keep compounding | Capex >> revenue: spend runs years ahead of income |
| 2 | Cash-funded: hyperscalers spend from profits | Depreciation: GPUs age like fish, not wine |
| 3 | Deflation feeds demand: cheaper AI = more AI | Commoditisation: the middle is already collapsing |
| 4 | Infrastructure endures: the assets outlast the cycle | Power walls: grids may simply say no |
Six risks a serious investor tracks
- Return on capex: roughly $720-740B of 2026 hyperscaler capex guidance (Q3 2026 earnings; definitions differ) must eventually meet matching revenue.
- GPU depreciation: 3-5-year asset lives against 30-year financing instincts.
- Power denial: grid queues and local opposition stalling projects.
- Price collapse: compute deflating faster than contracts assume.
- Concentration: a handful of firms are the whole demand side.
- Policy shock: export controls, energy rules, data-centre moratoria.
The AI build-out is on track to absorb a trillion dollars a year, more than global oil-and-gas upstream investment. Whether that is visionary or reckless will define this market decade.
What to Watch
Signposts, not predictions
Four dials worth watching
Five signposts for the next 24 months
- Does compute keep deflating? The pace of GPU price decline sets everyone's margins.
- Do the gigawatts arrive? Energised capacity, not announced capacity.
- Does a neocloud stumble? The first casualty will mark the cycle's turn.
- Do nuclear deals materialise? Power partnerships moving from press release to plant.
- Does capex guidance crack? One hyperscaler blinking changes the whole narrative.
Reference
Glossary & sources
Plain-language glossary
| Meaning | |
|---|---|
| GPU | the chip that does AI's parallel maths, the stack's workhorse. |
| Data centre | a building full of racks of computers, AI's factory. |
| Hyperscaler | a giant cloud provider: AWS, Azure, Google Cloud. |
| Neocloud | a specialist GPU-rental cloud, e.g. CoreWeave. |
| Inference | running a trained model to answer queries. |
| TWh | terawatt-hour, the unit of national-scale electricity. |
| Interconnection queue | the waiting list to plug into the power grid. |
| Token | a chunk of text; AI compute is priced per million of them. |
The cloud turned out to be made of concrete, copper and gigawatts. Whoever controls the scarce layer of that machine, this year chips, next year power, quietly sets the price of intelligence.
The five conclusions to take away
- AI is an industrial system: six layers of physical infrastructure, not an app.
- Power is the new chip: electrons, not GPUs, now gate the build-out.
- Compute is commoditising: the middle of the stack is being repriced.
- Value sits at the edges: scarce inputs and sticky customers win.
- Watch the constraint: it moves, and profit moves with it.
Sources & further reading
- IEA: 'Energy and AI' report; data-centre demand projections (2025).
- Stanford HAI: AI Index Report 2025 (inference cost trends).
- GPU cloud price trackers: H100/B200 rental rates (2023-26).
- Company disclosures: hyperscaler capex and data-centre plans.
- Epoch AI: compute and training-cost trend data.
- McKinsey: data-centre and power-demand analyses (2024-25).
- S&P Global / DCD: grid interconnection and energisation reporting.
- Figures marked 'estimate' or 'illustrative' are compilations.
Sources cited
IEA · Stanford HAI · GPU cloud price trackers · Company disclosures · Industry reporting · Industry teardowns · Hyperscaler disclosures · Industry estimates · Company announcements · Government announcements · Epoch AI · McKinsey · S&P Global / DCD · IEA, Apr 2026 (via W.Media) · Company earnings releases, Oct 2026 (via Luminix) · Ornn Compute Price Index, May 2026 · SemiAnalysis, Mar 2026 (via KuCoin News) · IntuitionLabs, Oct 2026
Keep reading
Research, not investment advice. Figures as of , from the sources cited.
Legal information and disclosures
These are my own views as of the update shown on this page. They can change without notice, and I have no duty to update them. Past performance does not indicate future results, and any investment can lose money.
This is for education only. It is not investment, legal, tax, medical or accounting advice, and it is not an offer to sell or a solicitation to buy any security in any jurisdiction. It does not consider your goals or circumstances.
Figures from company filings, public agencies and other third parties are believed to be reliable, but I have not independently verified them and cannot guarantee them. Forecasts and anything marked as an estimate rest on assumptions that may prove wrong.
Please do not copy or republish this material without my written consent.