← All research Research · Technology and AI · No. 13

The AI Stack

The industrial machine behind the chatbot: chips, data centres, gigawatts and cloud economics, and where the profit finally settles.

Every AI answer is produced by a physical machine, acres of silicon, steel and power lines that someone had to build and pay for. The cloud is not a cloud: it is the largest industrial build-out since the railroads, and value settles where the constraint is, which is moving from chips to power to distribution.

Chapters
9
Reading time
24 min
Updated
945 TWhData centres by 2030more power than Japan uses
~$2/hrH100 rentaldown from $8, a price collapse
~60%Of capex is GPUsthe chip dominates the bill
280xToken cost fallinference deflation in 2 years

What this argues

  1. The stack has six layersSilicon, data centres, power, cloud, models and apps, each with different economics.
  2. Power is the new bottleneckChips are easing; electricity and grid connections are not.
  3. Compute is commoditisingGPU rental prices collapsed 60%+, a warning for the middle of the stack.
  4. Value pools at the edgesScarce inputs (chips, power) and sticky outputs (apps) win; the middle gets squeezed.
  5. Watch the constraintWhoever owns the binding bottleneck at each moment sets the price.
00

What has changed

Updated November 2026

~$220BAmazon 2026 cash capexreaffirmed, 29 Oct 2026
~$170BMicrosoft calendar 2026 capextrimmed from ~$175B
$135-145BMeta 2026 capexfloor raised by $5B
+17%Data-centre power use in 2025vs ~3% for all global demand (IEA)
October 2026

Hyperscaler capex guidance holds near $720-740B for 2026

  • Alphabet (Q3 results, 28 Oct 2026): 2026 capex of $195-205B, unchanged; CFO said capex will increase significantly in 2027.
  • Amazon (Q3 results, 29 Oct 2026): about $220B of 2026 cash capex reaffirmed; CEO said it expects to spend meaningfully more in 2027.
  • Meta (Q3 results, 21 Oct 2026): $135-145B including finance-lease principal; 2027 plans to be shared on the Q4 call.
  • Definitions differ by company (calendar year, cash, finance leases), so the sum is indicative only. Source: company earnings releases as summarised by Luminix, Oct 2026.
October 2026

Microsoft trims 2026 capex as third-party capacity slips

Microsoft (fiscal Q1 2027 results, 28 Oct 2026) guided calendar 2026 capex to about $170B, down from about $175B, citing delays in third-party data-centre deliveries. It expects capex above $50B next quarter, and its CFO said demand still exceeds supply. Source: Microsoft earnings materials as summarised by Luminix, Oct 2026.

April 2026

Data-centre power demand grew 17% in 2025

The IEA's April 2026 update reported data-centre electricity use rose 17% in 2025, more than five times faster than the roughly 3% growth in global demand, and that AI-related demand is expected to triple by 2030 while total data-centre demand doubles. Source: IEA via W.Media, 20 Apr 2026.

May 2026

GPU rental prices have stabilised well below the 2024 peak

H100 prices were near $8/hr in early 2024. One-year contracts were about $2.35/hr by March 2026 (SemiAnalysis), and the trade-based Ornn index hit $2.94 on 25 May 2026 before easing. Nebius raised its on-demand H100 rate from $3.85 to $4.50 per GPU-hour from 1 Oct 2026, so the decline has paused or partly reversed at some providers. Sources: KuCoin News on Ornn, May 2026; IntuitionLabs, Oct 2026.

Not refreshed: the inference-cost series (Stanford HAI 2025), the capex share breakdown, GPU depreciation lives, nuclear and SMR timeline status, sovereign AI deal counts, and the 2024 baseline of 415 TWh. The 2030 figure of 945 TWh was not re-verified at source in this update.

Test yourself

Microsoft lowered its 2026 capex guidance from about $175B to about $170B. What reason did the chapter give?

The IEA said data-centre electricity use rose 17% in 2025 while global demand grew about 3%. How do the two compare?

01

The Stack, Mapped

Six layers, one machine

Every AI answer is produced by a physical machine, acres of silicon, steel and power lines that someone had to build and pay for. The cloud is not a cloud. It is the largest industrial build-out since the railroads.

The anatomy

From sand to answer: the six layers of AI

The six layers, top to bottom

What it is
06 ApplicationsChatGPT, Copilot, vertical tools: what users touch and pay for.
05 Modelsthe trained intelligence: OpenAI, Anthropic, Google, Meta.
04 Cloud platformsAWS, Azure, GCP: rent the machine by the hour.
03 Data centresthe buildings: racks, cooling, networking at city scale.
02 Power & gridgigawatts of electricity: the emerging bottleneck.
01 SiliconGPUs and accelerators: NVIDIA's kingdom.
The flow

What happens when you ask an AI a question

  1. Prompt: your words travel to a data centre, often in another state.
  2. Route: a scheduler assigns your request to a rack of GPUs.
  3. Compute: the model's weights run in a few hundred milliseconds.
  4. Return: the answer streams back, costing the operator a fraction of a cent.
Two workloads

Training and inference stress the stack differently

TrainingInference
Shapeone giant job, months longbillions of tiny jobs
Needs100k GPUs tightly networkedGPUs near the users
Siteone massive campusdistributed, everywhere
Buysthe biggest clusters on Earthsteady, growing capacity
~1.5%Of the world's electricity already flows into data centresAbout 415 TWh in 2024, before the AI build-out fully lands

The IEA expects this to more than double by 2030, and its April 2026 update shows data-centre use grew 17% in 2025 alone. The internet's physical footprint is becoming visible in national energy statistics.

Test yourself

Why does the chapter say scale is 'the business' in AI?

Which statement best describes how training and inference differ in where they are built?

02

Data Centres

The factories of intelligence

An AI data centre is a factory: its raw material is electricity, its machines are GPUs, and its product is computation. Thinking of it as a factory, not an office, explains every design decision.

The specimen

One frontier AI campus, by the numbers

~$10-40BTo buildrivals a nuclear plant
100k+GPUsin a single training cluster
1 GW+Power drawa mid-size city's worth
~2 yrsTo energiseif the grid cooperates
The bill

GPUs are 60% of the bill, and they depreciate fastest

Share of data-centre capex %
View the data
Share of capex
GPUs & compute60
Power & cooling15
Building & land12
Networking8
Storage5

Source: Industry teardowns; hyperscaler disclosures

The hard parts

What makes AI data centres different

  • Density: AI racks draw 5-10x the power of normal racks, so cooling becomes the design problem.
  • Liquid cooling: air can no longer remove the heat; water and direct-to-chip loops take over.
  • Networking: 100k GPUs must talk constantly; the network is as critical as the chips.
  • Location: sites are chosen for power and water first, latency second.
  • Speed: the race rewards whoever energises capacity first.
  • Depreciation: the $60B of GPUs inside will be obsolete within five years.
The heat problem

Cooling has to escalate with chip density

  1. Classic racks: fans and cold aisles (air-cooled).
  2. Rear-door: chilled doors on racks.
  3. Direct-to-chip: liquid loops on the silicon.
  4. Immersion: servers submerged in fluid (liquid-cooled).
Depreciation

The brutal math of a fast-aging asset

3-5 yrsGPU useful lifebefore obsolescence
~60%Of capexsits in that aging silicon
25%+Annual value losson the compute fleet
30 yrsBuilding lifethe shell outlives 6 fleets

Test yourself

A campus costs $40B and splits capex as the chapter's chart shows. About how much is spent on GPUs and compute?

Why can financing a data centre like an ordinary building be dangerous?

03

The Power Problem

The bottleneck moves to the grid

The curve

Data-centre electricity demand more than doubles by 2030

Data-centre electricity demand TWh
View the data
Demand
2020290
2024415
2030e945

Source: IEA, Energy and AI (2025)

The mismatch

AI moves at software speed; the grid moves at permit speed

AI build-outGrid build-out
Timelinemonths to order GPUs5-15 years for lines & plants
Capitaleffectively unlimitedregulated, rationed
Demand / Supplydoubling every 1-2 yearsgrows a few % a year
Decisionone board meetingregulators, communities, courts
The responses

How the industry is chasing electrons

  • Nuclear revival: hyperscalers signing deals to restart and build reactors.
  • Behind-the-meter: data centres built at the power plant, skipping the grid.
  • Renewables + storage: solar and batteries where land and sun allow.
  • Gas turbines: fast to deploy, climate targets quietly strained.
  • Grid investment: transformers and lines becoming strategic goods.
  • Efficiency: each GPU generation does more per watt, but demand grows faster.
More than steelBy 2030, US data centres out-consume all heavy manufacturingIEA projection

The IEA projects US data centres will use more electricity than aluminium, steel, cement and chemicals production combined. Computation is becoming the defining industrial load of the American grid.

The nuclear pivot

Big tech rediscovers the atom

  1. First PPAs: hyperscalers sign nuclear power deals.
  2. Three Mile Island: Microsoft funds a reactor restart.
  3. SMR orders: Google and Amazon back small reactors.
  4. First SMRs: new small reactors targeted online.
  5. Gigawatt fleets: dedicated nuclear for AI campuses.

Test yourself

According to the chapter, what increasingly decides where and when AI capacity gets built?

Data-centre demand goes from 415 TWh in 2024 to 945 TWh in 2030e. Which description fits?

04

The Compute Market

Renting the machine

You no longer buy AI compute, you rent it by the hour, and the rental price just collapsed. What happened to GPU prices in two years is the most under-reported story in AI.

The collapse

H100 rental prices fell by roughly three-quarters

H100 rental price $/hr
View the data
H100 rental
20238
H1'245.5
H2'244.5
20252.8
20262

Source: GPU cloud price trackers (2023-26) · values read from the chart, approximate

Downstream

And the cost of serving AI fell even faster

Cost of a GPT-3.5-class query (log scale) $
View the data
Cost per query
Nov'2220
Mid'2310
Late'234
Mid'241
Late'240.3
20250.07

Source: Stanford HAI AI Index (2025); log scale · values read from the chart, approximate

The players

Who sells compute, and how

  • Hyperscalers: AWS, Azure, GCP. Full platforms, premium prices, deep pockets.
  • Neoclouds: CoreWeave, Lambda, Nebius. GPU specialists, aggressive pricing.
  • NVIDIA itself: sells the chips and increasingly rents them too.
  • Model labs: OpenAI and Anthropic buying capacity years ahead.
  • Spot markets: idle GPUs auctioned by the hour, the price floor.
  • Sovereign clouds: nations buying compute as strategic reserve.
The supporting cast

The quiet winners selling into the build-out

  • HBM memory: SK Hynix and Micron, the scarce memory bolted to every AI chip.
  • Networking: NVIDIA InfiniBand, Broadcom, Arista, the fabric between 100k GPUs.
  • Transformers & switchgear: multi-year waiting lists; the grid's quiet chokepoint.
  • Liquid cooling: Vertiv and peers, where plumbing becomes strategic.
  • Construction & EPC: data-centre builders booked out for years.
  • Land & fibre: sites with power, water and connectivity command premiums.
The squeeze

Good for users, dangerous for the middle

Winners of deflationSqueezed by deflation
1Users & apps: every query gets cheaperBare GPU renters: price is their only product
2Model labs: training budgets go furtherOld-chip fleets: H100s undercut by B200s
3Adopters: new use cases become viableLate data centres: built for yesterday's prices
4NVIDIA: sells the next generation anywayLeveraged players: debt against falling rates
Sovereign AI

Nations are buying compute like a strategic reserve

30+Countrieswith national AI compute plans
$10B+Single dealsGulf states buying GPU campuses
Export rulesUS controlswho may buy the best chips
2 blocsTech spheresUS-aligned and China-aligned stacks

Source: Government announcements; industry reporting (2024-26).

The flywheel

The stack's self-feeding loop

  1. Cheaper compute
  2. More AI usage
  3. More revenue
  4. More capex
  5. Better chips (which leads back to cheaper compute: the build-out flywheel)

Test yourself

H100 rental fell from about $8/hr to about $2/hr. Roughly what percentage fall is that?

A company's only product is renting out older H100 GPUs, and newer B200 chips arrive. What does the chapter suggest?

05

Where Value Settles

The shape of the profit pool

The map

Scarcity versus stickiness across the stack

A two-by-two map with scarcity of the input on the horizontal axis and stickiness of the customer on the vertical axis. Quadrants: top-left loyal, abundant; top-right the fortress; bottom-left commodity; bottom-right scarce, fickle. Apps sit in the loyal, abundant quadrant (high stickiness, low scarcity). Cloud sits near the centre line on stickiness and just left of centre on scarcity. Models sit near the centre. Silicon sits in the fortress (high scarcity, high stickiness). Power & grid sits at the right edge on scarcity, near the middle on stickiness. Bare GPU rental sits in the commodity corner (low scarcity, low stickiness).

The pattern

What history says about infrastructure booms

  • The railroad lesson. The track builders went bust; the economy that ran on the track thrived.
  • The fibre lesson. Dot-com fibre was overbuilt, bankrupted its owners, then powered two decades of the internet.
  • The likely rhyme. Some of today's AI capex will never earn a return, but the capacity will not be wasted.
  • The investor's task. Distinguish owning the boom's spending from owning what the boom leaves behind.

The stack's iron rule: value settles where the constraint is, and the constraint is moving from chips to power to distribution.

2023's bottleneck was GPUs. Today's is grid connections. Tomorrow's is likely customer relationships.

Test yourself

Why did bare GPU rental prices collapse first?

What does the railroad and fibre history suggest about today's AI capex?

06

Risks & Debates

The honest chapter

Both sides, fairly

Build-out or bubble?

The build-out caseThe bubble case
1Real demand: usage and tokens keep compoundingCapex >> revenue: spend runs years ahead of income
2Cash-funded: hyperscalers spend from profitsDepreciation: GPUs age like fish, not wine
3Deflation feeds demand: cheaper AI = more AICommoditisation: the middle is already collapsing
4Infrastructure endures: the assets outlast the cyclePower walls: grids may simply say no
The risk register

Six risks a serious investor tracks

  • Return on capex: roughly $720-740B of 2026 hyperscaler capex guidance (Q3 2026 earnings; definitions differ) must eventually meet matching revenue.
  • GPU depreciation: 3-5-year asset lives against 30-year financing instincts.
  • Power denial: grid queues and local opposition stalling projects.
  • Price collapse: compute deflating faster than contracts assume.
  • Concentration: a handful of firms are the whole demand side.
  • Policy shock: export controls, energy rules, data-centre moratoria.
$1 trillion+Projected annual big-tech capex by 2027

The AI build-out is on track to absorb a trillion dollars a year, more than global oil-and-gas upstream investment. Whether that is visionary or reckless will define this market decade.

Test yourself

Which of these is an argument from the build-out case rather than the bubble case?

What outcome does the chapter call 'likely'?

07

What to Watch

Signposts, not predictions

The dashboard

Four dials worth watching

GPU $/hrCompute pricedeflation's pace
GW energisedPower onlinethe real build rate
Capex / revenueDisciplinethe bubble gauge
Grid queuesInterconnectsthe binding constraint
Key takeaways

Five signposts for the next 24 months

  1. Does compute keep deflating? The pace of GPU price decline sets everyone's margins.
  2. Do the gigawatts arrive? Energised capacity, not announced capacity.
  3. Does a neocloud stumble? The first casualty will mark the cycle's turn.
  4. Do nuclear deals materialise? Power partnerships moving from press release to plant.
  5. Does capex guidance crack? One hyperscaler blinking changes the whole narrative.

Test yourself

A company announces plans for 5 GW of new data centres. What should you watch, per the chapter?

What does it suggest when capex growth and revenue growth converge?

08

Reference

Glossary & sources

Reference

Plain-language glossary

Meaning
GPUthe chip that does AI's parallel maths, the stack's workhorse.
Data centrea building full of racks of computers, AI's factory.
Hyperscalera giant cloud provider: AWS, Azure, Google Cloud.
Neoclouda specialist GPU-rental cloud, e.g. CoreWeave.
Inferencerunning a trained model to answer queries.
TWhterawatt-hour, the unit of national-scale electricity.
Interconnection queuethe waiting list to plug into the power grid.
Tokena chunk of text; AI compute is priced per million of them.

The cloud turned out to be made of concrete, copper and gigawatts. Whoever controls the scarce layer of that machine, this year chips, next year power, quietly sets the price of intelligence.

Key takeaways

The five conclusions to take away

  1. AI is an industrial system: six layers of physical infrastructure, not an app.
  2. Power is the new chip: electrons, not GPUs, now gate the build-out.
  3. Compute is commoditising: the middle of the stack is being repriced.
  4. Value sits at the edges: scarce inputs and sticky customers win.
  5. Watch the constraint: it moves, and profit moves with it.
Reference

Sources & further reading

  • IEA: 'Energy and AI' report; data-centre demand projections (2025).
  • Stanford HAI: AI Index Report 2025 (inference cost trends).
  • GPU cloud price trackers: H100/B200 rental rates (2023-26).
  • Company disclosures: hyperscaler capex and data-centre plans.
  • Epoch AI: compute and training-cost trend data.
  • McKinsey: data-centre and power-demand analyses (2024-25).
  • S&P Global / DCD: grid interconnection and energisation reporting.
  • Figures marked 'estimate' or 'illustrative' are compilations.

Test yourself

A new data centre project is described as 'stuck in the interconnection queue'. What does that mean?

What does the conclusion 'Power is the new chip' mean?

Sources cited

IEA · Stanford HAI · GPU cloud price trackers · Company disclosures · Industry reporting · Industry teardowns · Hyperscaler disclosures · Industry estimates · Company announcements · Government announcements · Epoch AI · McKinsey · S&P Global / DCD · IEA, Apr 2026 (via W.Media) · Company earnings releases, Oct 2026 (via Luminix) · Ornn Compute Price Index, May 2026 · SemiAnalysis, Mar 2026 (via KuCoin News) · IntuitionLabs, Oct 2026

Keep reading

Research, not investment advice. Figures as of , from the sources cited.

Legal information and disclosures

These are my own views as of the update shown on this page. They can change without notice, and I have no duty to update them. Past performance does not indicate future results, and any investment can lose money.

This is for education only. It is not investment, legal, tax, medical or accounting advice, and it is not an offer to sell or a solicitation to buy any security in any jurisdiction. It does not consider your goals or circumstances.

Figures from company filings, public agencies and other third parties are believed to be reliable, but I have not independently verified them and cannot guarantee them. Forecasts and anything marked as an estimate rest on assumptions that may prove wrong.

Please do not copy or republish this material without my written consent.