Putting a Computer Inside 500,000 GPUs — Pingu Touzi Zhenwu V900, China's System-Level Breakthrough, and the Economics of the Compute Gap
1. The “Power Grid” at Yunqi: Alibaba Betters on Power Stations First
On September 22, 2026, the Yunqi Conference kicked off in Hangzhou under the theme “Applied Intelligence” (智以致用). Many expected a “killer app.” Instead, Alibaba Group CEO Eddie Wu (吴泳铭) delivered an entire narrative about “power stations and the grid.” He decomposed machine-intelligence infrastructure into three pillars: AI models, AI chips, and AI cloud. “Machines are becoming the main force of thinking; intelligence is becoming a scalable commodity. The total amount of machine thinking will reach more than 1000 times that of humans, while today that ratio is still under 3%."TMTPost
Translated into engineering language: tokens will become the “electricity” of the AI era, chips are the “generators,” and the cloud is the “power grid.” Before a definitive product emerges, the most certain move is to build power stations and lay the grid first — this is the fundamental logic behind Alibaba’s heavy long-term investment in chips.36Kr
The hard-core protagonist of this narrative is Zhenwu V900 (真武V900), the next-generation self-developed AI chip from Alibaba’s semiconductor subsidiary, Pingu Touzi (平头哥). Its compute is directly 3× that of the previous Zhenwu M890, and a single cluster built on it can scale to 500,000 AI accelerators, targeting mass production in Q1 2027.EET-China
This article will not dwell on the single-point “3× benchmark” story. What truly matters about the V900 is the system-level breakthrough it represents, and the brutal economics of the compute gap revealed by IDC data.
Before expanding further, it is worth establishing a coordinate system: why is this a “system-level” event rather than a “single-chip” one? For the past two years, the domestic AI chip narrative has largely stayed at the level of “matching a single card against some NVIDIA generation.” The Zhenwu V900 raises the competitive dimension by an order of magnitude — it is no longer “a card” but the central processor of an entire “computer.” Fifty thousand V900s are wired through self-developed ICN networking into one unified compute domain, flanked by Yitian CPUs for orchestration, Panmai NICs for inbound/outbound traffic, and the T-Head SAIL software stack as the programming model. This is a full-frontal system war against NVIDIA’s “GB200 rack–DGX SuperPOD–CUDA full-stack” architecture. Only by grasping this coordinate system can one truly read the significance of the V900.
2. Dissecting Zhenwu V900: From “Per-Chip FLOPs” to “Per-Token Cost”
First, the paper muscles. The Zhenwu V900 is a training-and-inference (训推一体) AI chip built on Pingu Touzi’s self-developed parallel computing architecture, carrying 216GB of high-capacity memory, 1200GB/s of inter-chip interconnect bandwidth, and native support for FP8/FP4 low-precision training and inference formats. Per-chip performance is 3× the M890 (i.e., +200%), making it the most powerful self-developed Chinese AI chip on the market.EET-China
But the real differentiator is not peak FLOPs — it is the system-level capacity designed for very large models. A single compute cluster built on the V900 with Pingu Touzi’s self-developed ICN Switch chips can scale to 500,000 accelerators, directly supporting the training and inference of frontier ultra-large models in the 5-trillion-to-10-trillion parameter range.EET-China
This neatly meets the model-side plan revealed by Eddie Wu: the Qwen team is planning to train brand-new models with 5T-to-10T parameters. A “10T-parameter brain” needs not just a more powerful card, but an entire “compute city” that can hold it in memory and shuttle tokens between operators at high speed.
Figure 1: Zhenwu V900 core compute architecture
+-------------------------------------------------------------+
| Zhenwu V900 (train/infer) |
| |
| +----------------+ +----------------+ +--------------+ |
| | Compute Core | | Compute Core | | Compute Core | |
| | [FP8/FP4/FP16] | | [FP8/FP4/FP16] | | [FP8/FP4/..] | |
| +-------+--------+ +-------+--------+ +-------+------+ |
| | | | |
| +---------------------+--------------------+ |
| | |
| +---------------+----------------+ |
| | OnChip NoC | |
| +---------------+----------------+ |
| | |
| +-------------------------+-------------------------+ |
| | 216GB HBM stacks (high-capacity) | |
| +-------------------------+-------------------------+ |
| | |
| +-------------------------+-------------------------+ |
| | ICN inter-chip Ctrl (1200 GB/s per card) | |
| +-------------------------+-------------------------+ |
+-------------------------------------------------------------+
Figure 2: 500K-card supernode topology on V900
Cloud control/scheduling layer (alloc, fault-tol, token ops)
| | |
+-----v------+ +-----v------+ +-----v------+
| SupernodeA | | SupernodeB | | SupernodeC |
| (128-card) | | (128-card) | | (128-card) |
| ICN Switch | | ICN Switch | | ICN Switch |
+-----+------+ +-----+------+ +-----+------+
| \____ high-bw optical/copper ______/ |
| | |
======+==== PB-scale Fabric (Scale-out) ====+
| | |
+-----v---------------------v---------------------v---+
| 5T~10T param model (tensor/pp/ep parallel) |
+-----------------------------------------------------+
Underlying this system is the fact that Pingu Touzi has completed full-line coverage of all data-center core chips with self-developed designs: the Zhenwu GPU series, the Yitian CPU series, the Panmai smart NICs (智能网卡), and the ICN interconnect chips. Every core chip needed to assemble an ultra-large AI cluster is self-developed — this is the first layer of the “system-level breakthrough”: no single link in the chain can be choked by others.EET-China
One soft-power factor is often left out of coverage: the software stack. On July 18 this year, Pingu Touzi officially open-sourced its self-developed AI software stack, T-Head SAIL®, opening the core capabilities of efficient, open AI infrastructure to global developers and achieving broad compatibility with mainstream large-model ecosystems.EET-China In the GPU world, hardware is a “bare shell”; the software stack is what turns compute into productive capability. Domestic chip vendors have long been criticized for lacking “not chips, but software ecosystems.” Open-sourcing T-Head SAIL is a flanking deconstruction of the CUDA ecosystem moat, and one of the most crucial yet underrated tiles in Pingu Touzi’s system-level breakthrough.
Another set of numbers shows how “systemic” this really is. The 128-card supernode built on the Zhenwu M890, with the ICN Switch 1.0 interconnect chip, delivers communication latency down to the hundreds-of-nanoseconds scale, letting 128 chips “work as one computer” for massive concurrent Agent inference and large-model training.EET-China Scaling from “M890’s 128-card supernode” to “V900’s single 500K-card cluster” crosses an order of magnitude — testing not a single-chip designer, but a systems engineering army of network topology, routing algorithms, fault tolerance, and schedulers.
3. The Yitian CPU Roadmap: CPU and AI Chips “Meets” on ICN
The other dark horse of the breakthrough is the CPU. At this Yunqi Conference, Pingu Touzi unveiled the Yitian server CPU roadmap: the 720 and 730 generations in 2027, and then the Yitian 750 supporting Pingu Touzi’s self-developed ICN inter-chip interconnect bus protocol, enabling direct interconnection with Zhenwu AI chips, further improving CPU–AI-chip collaboration efficiency to meet the compute demand of the Agentic era.Beijing Business Today
Why does this matter? Because the workload structure of the Agentic era has changed. A single task from an agent like Meta Muse or OpenAI Astra may involve dozens of steps — planning, retrieval, document reading, tool calls, code execution, result checking, and error correction — each re-invoking the model. In this workload, the GPU handles “model thinking,” while the CPU shoulders tool execution and task orchestration, and HBM, DRAM, storage, and high-performance networking jointly carry “state movement.” When CPU and GPU talk directly over the same self-developed ICN bus protocol, skipping traditional PCIe relaying and protocol translation, co-scheduling latency and token operations cost can be systematically lowered.
Figure 3: Yitian 750 CPU ↔ Zhenwu AI chip direct ICN link
+---------------------+ +---------------------+
| Yitian750 ServerCPU | | Zhenwu V900 AI chip |
| (general/ormestrat.) | | (matrix/vector calc)|
+---+---------------+ +---+---------------+
| ICN bus | ICN |
| Host-Device Unified Bus | Host-Device Uni |
+--------------+-------------+--------------+
| |
+------v-------------v------+
| ICN Switch (low-latency) |
| latency ~100ns scale |
+--------------------------+
This “CPU-directly-connects-GPU” thought experiment, when translated into engineering, precisely responds to the pain point IDC’s Zhou Zhengang highlighted when interpreting the report — the “mismatch” between training and inference: clusters historically built for training and peak FLOPs do not adapt well to the “context-saving, high-throughput, low-latency” inference and agentic operations of the Agent era, resulting in structural waste (“invested a lot but unused”).Sina Finance What Pingu Touzi intends to do is use unified ICN interconnect plus full-stack co-tuning to smooth away this mismatch.
Seen one level deeper, the Yitian CPU’s role in the Agentic era is being redefined. Historically, the CPU in an AI server was a “supporting player” that fed data to the GPU. But agent tasks lean heavily on tool calls, task orchestration, context management, and multi-step state-machine progression — “control-flow” work that is precisely the CPU’s strength. When CPU and AI chip talk directly over one bus protocol, control flow and data flow no longer bounce between heterogeneous buses, and each tool call in an agent’s loop returns with lower latency. “CPU–AI-chip collaboration efficiency” is exactly this: converting agents from a slow “try-and-wait” loop into a fast “execute-and-feedback” loop. This is also why Alibaba made “Agentic compute” the core anchor of chip design at this Yunqi.
4. The Economics of the Compute Gap: A $380.9B Supply–Demand Scissors
If we only read Alibaba’s “energy supply” narrative, we risk ignoring the cruelty of the buy side. The day before Yunqi opened (September 21), IDC and Inspur jointly released the 2026 China Artificial Intelligence Computing Power Development Assessment Report, full of tension-ridden numbers.
Demand side: China’s intelligent-compute scale is expected to reach 2576.5 EFLOPS in 2026, up 87.9% YoY, and to grow to 10,524.3 EFLOPS by 2030. China’s AI compute market is expected to be $51.5B in 2026 (+37% YoY) and $150.1B by 2030, with a 2025–2030 CAGR of 31.8%.People’s Daily
Supply side: The global AI compute demand-satisfaction rate will fall from 79% in 2024 to a trough of 71% in 2027, recovering to only 77% by 2030; the absolute compute gap widens from $38.9B in 2024 to $380.9B by 2030, and the supply–demand scissors keeps expanding.China Securities Journal
Even more striking is the token multiplier. IDC forecasts that the number of active agents globally will grow from 79.4M in 2026 to 2.216B in 2030 (CAGR 129.8%), while token consumption will grow at a CAGR of 4822.6% — 37× the growth rate of agent numbers.ChinaTech News
Why does supply keep tearing apart from demand? IDC Vice President Zhou Zhengang cited two core reasons: first, HBM high-bandwidth memory undersupply — AI accelerators widely face HBM shortages, and HBM is almost monopolized by SK Hynix, Samsung, and Micron, expanding slowly behind high barriers; “memory and storage shortages will last at least into next year”; second, structural mismatch — traditional training-capacity does not fit Agent inference demand.Sina Finance
Here an counterintuitive economic law applies — the Jevons paradox: when technological progress lowers the unit cost of compute, total consumption does not fall; instead, because demand is massively stimulated, aggregate consumption keeps growing. The scaling of NVIDIA GPUs and Google TPUs is pushing compute demand far beyond GPUs into storage, advanced packaging, networking, and power systems.JRJ (金融界)
Figure 4: Global AI compute supply–demand gap model (2024–2030)
Demand(R) ████████████████████████████
Supply(S) ██████████████████
2024 2025 2026 2027 2028 2029 2030
Sat.rate: 79% ... 74%→ 71%trough ... ... 77%
Gap($B): 38.9 ... 380.9
|<-- the scissors keeps widening -->|
note: sat.rate = supply/demand; absolute gap by IDC
This “gap economics” is a double-edged sword for domestic chips. The bad side: everyone globally is short of cards, especially HBM, and domestic chip scale-up is also constrained by memory/advanced-packaging supply chains. The good side: it turns “the ability to supply itself” into a scarce asset — whoever can build a non-choked full-stack compute infrastructure faster holds the pricing power of the next cycle.
Placing the Jevons paradox into the Chinese context yields more concrete inferences. As AI efficiency improves, unit token cost falls, incentivizing more enterprises to deploy agents into production processes and thereby amplifying total token consumption — this is not panic; it is the deterministic direction of IDC’s model: China’s active-agent count is projected to grow at a 151.2% CAGR from 2026 to 2030, while token consumption over the same period grows at a stunning 3499.3% CAGR.Sina Finance In other words, the compute “famine” is not a temporary squeeze but a structural, long-term imbalance. That explains why Alibaba, Microsoft, Google, and AWS all prefer to carry heavy depreciation burdens to build out compute pools first — whoever owns generation capacity first controls the “distributed electricity price” of the machine-intelligence era.
There is also a frequently misread point: the essence of the compute gap is not just “shortage of cards,” but “shortage of the right system.” IDC explicitly judges that one of the industry’s biggest bottlenecks is HBM undersupply, and that “training-oriented clusters do not fit today’s inference and agentic operations.” Stacking accelerator cards alone does not solve the problem; one must fix both “can it be stored” (memory/storage) and “can it be used” (inference/agent adaptation). The V900’s 216GB memory + unified ICN interconnect + full-stack tuning targets precisely this “right system.”
5. Alibaba’s Full-Stack AI Shelf: A Meet-up of Chip, Cloud, and Model
The Zhenwu V900 is not an isolated chip launch; it is one tile of Alibaba’s “full-stack AI shelf.” As Beijing Business Today observed: Pingu Touzi makes chips, Alibaba Cloud builds the compute base, and the Qwen large model supplies intelligence, while the Qwen AI platform and Qwen Office are responsible for delivery to enterprises — nearly every link on the shelf rolled out new plans on that day.Beijing Business Today
Figure 5: Alibaba full-stack AI shelf (chip–cloud–model)
+------ application/delivery ------+
| Qwen AI platform | Qwen Office |
| Agent/industry solutions |
+-----------+---------------------+
|
+-----------+---------------------+
| Cloud (supply) |
| Alibaba Cloud · AI supernode |
| 2032 target >20GW |
+-----------+---------------------+
|
+-----------+---------------------+
| Model (intelligence) |
| Qwen 5-10T param plan |
| Qwen3.8-Max RSI iterations |
+-----------+---------------------+
|
+-----------+---------------------+
| Chip (generators) |
| Zhenwu GPU|Yitian CPU |
| Panmai NIC|ICN interconnect |
+---------------------------------+
Backing this shelf are hard capital and operating numbers. Citing financial reports, Beijing Business Today noted: in Q2 2026, Alibaba’s AI-cloud and compute-services revenue was about RMB 48.4B, up 45% YoY; capex was about RMB 67.7B, up 75% YoY.Beijing Business Today 21st Century Business Herald confirmed Alibaba’s quarterly capex of RMB 67.678B (+75% YoY), and that by mid-2026 roughly half of the three-year RMB 380B AI-hardware investment plan had been spent.21st Century Business Herald
On the scale dimension, Eddie Wu gave a long-term target never disclosed before: by 2032, the global data-center capacity operated by Alibaba Cloud will exceed 20GW — equivalent to more than two hundred large data-center parks, roughly 10× the pre-generative-AI level of 2022, placing Alibaba directly into the first tier of hyperscalers alongside Microsoft (38GW by 2032) and AWS (adding ~3.8GW per year).21st Century Business Herald
Notably, the V900 is no isolated “futures.” According to 21st Century Business Herald, the Lingjun supernode instance built on the Zhenwu M890 already stably runs Qwen3.8-Max, Kimi K3, and other models above 2T parameters; by June 2026 the Zhenwu series had served over 650 enterprise customers; and Pingu Touzi open-sourced its self-developed AI software stack, T-Head SAIL, in July this year.EET-China
6. The Landscape of Domestic AI Chips: From “Per-Card Parameter Wars” to “Full-Stack Fortress”
With the V900’s release, the competitive landscape of domestic AI chips is quietly shifting. Huawei Ascend, Cambricon, Biren, and others each have strengths, but Pingu Touzi’s style is the most “platform-like” — because behind it sits Alibaba Cloud as its own captive ground.
Figure 6: Domestic AI chip landscape (system view)
Vendor Compute path Differentiation Anchored
HuaweiAsc. Ascend 950DT/910B full-stack+CANN Huawei Cloud/enterprise
PinguTouzi Zhenwu V900 full-line+ICN Alibaba Cloud/Qwen
Cambricon Siyuan series inference/edge independent
Biren BR series general-purpose independent
BHT general+auto multimodal/auto independent/OEM
------------------------------------------------------------------
Focus: per-card FLOPs -> cluster efficiency -> per-token cost
The progression from “per-card parameter wars” to “cluster efficiency wars” and then to “per-token cost wars” is the advanced path of domestic chip breakout. Alibaba’s unique advantage is that it simultaneously holds three loops — chips (Zhenwu), cloud (Alibaba Cloud), and model (Qwen) — enabling multiplicative amplification of single-loop gains: model demand feeds back into chip design, hardware/software optimization enters cloud services, and the model itself participates in the next round of improvement. This collaborative feedback mechanism is a rare full-stack capability among global hyperscalers.
Notably, this coordination has moved from “concept” into an “engineering closed loop.” Per China Securities Journal’s field report from Yunqi, Qwen3.8-Max iterated for over a month with essentially zero human involvement, completing 33 valid iterations and raising its score on an evaluation benchmark from 40 to 45; RSI (recursive self-improvement) has already entered the model-training, inference, and chip-model co-tuning links.China Securities Journal “Chip-model co-tuning” means letting Qwen’s iteration needs steer Zhenwu chip trade-offs — what precision the model wants, how large a context the memory must hold, how low the latency must be, the chip design pushes in that direction. This “model-defines-chip” feedback loop is Alibaba’s second, rarely replicable layer of systemic advantage over independent chip vendors.
At the other end of the “full-stack shelf” lies the edge. At this Yunqi, Alibaba simultaneously laid out an edge-AI ecosystem: it open-sourced the Qwen 2.7B model so lightweight models can be deployed locally on desktops, and launched the Qwen mobile AI solution (Qwen Intelligence) to push large-model capabilities further toward end devices.PingWest/Phoenix The Qwen AI-glasses N1 series and the AI-native agentic PC “Qwen Book” also made their debut.China Securities Journal From a 500K-card supercompute pool in the cloud to a pair of smart glasses worn on the nose, this “cloud–edge–end” integrated pathway is the complete form of the “power grid” Alibaba intends to lay. The edge reaches users, the cloud supplies intelligence, the chip compresses cost — three tiers interlocked form the true scalable commodity.
Figure 7: Token production chain in AI cluster train/infer
request in -> orchestration(Yitian CPU) -> model sched(Qwen)
-> tensor/pipe parallel(Zhenwu GPU) -> KV Cache(216GB)
-> result aggregate -> return
\----- ICN high-speed link (1200GB/s) ------/
goal: raise token throughput per watt and per chip
7. A Code View: From “Chip Specs” to “System Economics”
A technical article that discusses only specs stays at the PPT level. Let us land the “compute-gap economics” and “500K-card cluster scheduling” in quantifiable simulations via code. Below, Go implements a simplified cluster scheduler/efficiency model, and Python implements the compute supply–demand gap extrapolation.
7.1 Go: resource scheduler skeleton for a 500K-card cluster
package main
import (
"fmt"
"math"
"sync"
)
// node describes resource state of one card in a supernode.
type node struct {
id int
hbmGB int // per-card memory (GB); V900 is 216
utilized float64 // token-side utilization 0~1
costUSD float64 // per-card hourly cost ($/h)
}
type cluster struct {
nodes []node
mu sync.Mutex
}
// schedule is a naive greedy: put slices into least-utilized cards.
func (c *cluster) schedule(needGB int, steps int) float64 {
done := 0
c.mu.Lock()
defer c.mu.Unlock()
for i := 0; i < steps; i++ {
var best *node
for j := range c.nodes {
if c.nodes[j].hbmGB >= needGB {
if best == nil || c.nodes[j].utilized < best.utilized {
best = &c.nodes[j]
}
}
}
if best == nil {
break
}
best.utilized = math.Min(1.0, best.utilized+0.15)
done++
}
return float64(done) / float64(steps)
}
// tokenCostPerGW estimates effective token throughput per GW (relative).
func tokenCostPerGW(clusterHourUSD float64, tokPerH float64) float64 {
return tokPerH / clusterHourUSD
}
func main() {
c := &cluster{nodes: make([]node, 500000)}
for i := range c.nodes {
c.nodes[i] = node{id: i, hbmGB: 216, costUSD: 3.5}
}
rate := c.schedule(180, 1000000)
fmt.Printf("schedule throughput: %.4f\n", rate)
fmt.Printf("token per dollar: %.2f\n", tokenCostPerGW(1750000, 1.2e12))
}
The values behind this excerpt encode real engineering judgment: single-card 216GB HBM determines whether a larger KV-cache slice can be held, and “tokens per dollar” is exactly the quantified expression of what Alibaba keeps hammering — “token throughput and token operations cost.” The ultimate yardstick of chip competitiveness is not FLOPs, but how much effective thinking one dollar can buy.
7.2 Python: compute-gap economics extrapolation
import numpy as np
YEARS = [2024, 2025, 2026, 2027, 2028, 2029, 2030]
SATISFACTION = {2024: 0.79, 2025: 0.76, 2026: 0.74,
2027: 0.71, 2028: 0.73, 2029: 0.75, 2030: 0.77}
GAP = {2024: 38.9, 2027: np.nan, 2030: 380.9} # gap ($B)
def demand_growth(y0=2576.5, cagr=0.503):
"""CN intelligent-compute (EFLOPS): 2026 base + CAGR 50.3%"""
return {2026+i: y0 * ((1+cagr)**i) for i in range(5)}
def model_gap_supply(satisfaction):
"""derive supply = demand * satisfaction (illustrative)"""
d = demand_growth()
return {y: d.get(y, 0) * satisfaction.get(y, sat(2030)) for y in satisfaction}
def sat(y):
return SATISFACTION.get(y, 0.77)
def main():
d = demand_growth()
print("CN intelligent compute (EFLOPS):")
for y, v in d.items():
print(f" {y}: {v:,.1f}")
s = model_gap_supply(SATISFACTION)
print("supply at satisfaction rate (EFLOPS, illustrative):")
for y, v in s.items():
print(f" {y}: {v:,.1f}")
print(f"Gap: 2024 {GAP[2024]}B$ -> 2030 {GAP[2030]}B$")
if __name__ == "__main__":
main()
This extrapolation makes IDC’s conclusion concrete: even as China’s intelligent compute races ahead at a 50.3% CAGR, the satisfaction rate still bottoms out in 2027, and the absolute gap balloons to $380.9B by 2030. The slope of demand is always steeper than the ramp of supply — this is the underlying equation of “compute-gap economics.”
7.3 Go: pipeline parallelism and collective communication in a 500K-card cluster
At a scale of 500,000 cards, “per-card” optimization stops mattering; gains increasingly come from pipeline parallelism and collective communication to amortize communication overhead. Below, Go sketches a simplified pipeline-shard and a ring AllReduce, showing how ICN interconnect is scheduled at the software layer:
package main
import (
"fmt"
"sync"
)
// stage models one pipeline-parallel shard holding a layer group.
type stage struct {
id int
load float64
}
// ringAllReduce is a simplified 1-D ring: each rank sums a local vector,
// forwards it along the ring, and converges on the global sum.
func ringAllReduce(local []float64, rank, size int) []float64 {
out := make([]float64, len(local))
copy(out, local)
for step := 1; step < size; step++ {
sender := (rank - step + size) % size // real impl uses ICN msgs
_ = sender
for i := range out {
out[i] += local[i] // placeholder for recv(sender)
}
}
return out
}
func main() {
var mu sync.Mutex
pipeline := []stage{{0, 0.3}, {1, 0.5}, {2, 0.2}}
for _, s := range pipeline {
mu.Lock()
s.load += 0.1 // a micro-batch drains through the pipeline
mu.Unlock()
}
grad := ringAllReduce([]float64{1, 2, 3}, 1, 8)
fmt.Println("pipeline:", len(pipeline), "grad len:", len(grad))
total := 0.0
for _, s := range pipeline {
total += s.load
}
fmt.Println("pipeline load balance:", total/float64(len(pipeline)))
}
Ring AllReduce matters because, in a 500K-card cluster, backpropagation must compute the global sum of every layer’s gradient across all cards. A naive all-to-one reduction toward a master card creates a single-point bottleneck; the ring topology lets every card exchange only with neighbors, decoupling communication volume from cluster size. This is precisely the benefit ICN high-speed interconnect is meant to deliver at the scheduler layer: pushing communication from “connectable” to “near-linear scalability.”
7.4 Python: KV-cache and per-token cost model
Inference in the Agent era has a voracious appetite for KV Cache — the longer the context and the higher the concurrency, the faster memory is consumed. This directly determines how long the “216GB memory” can sustain an agent session. Here is an order-of-magnitude estimate in Python:
def kv_cache_bytes(num_layers, num_heads, head_dim, batch, seq_len, elem=2):
# per-token KV footprint for a transformer; full-batch total bytes.
per_head = num_layers * 2 * num_heads * head_dim * elem
return per_head * batch * seq_len
model = dict(num_layers=96, num_heads=128, head_dim=128)
for seq in (8192, 32768, 131072):
gb = kv_cache_bytes(batch=1, seq_len=seq, **model) / 1e9
print(f"seq {seq:>6}: KV ~ {gb:.2f} GB")
def per_token_price(card_hour_usd=3.5, tokens_per_hour=1.2e12):
return card_hour_usd / tokens_per_hour
print(f"per-1K-token cost: {per_token_price()*1000:.6f} USD")
print(f"compute-gap: 2024 38.9B -> 2030 380.9B USD")
This estimate reveals a plain but critical engineering fact: pushing context from 8192 to 131072 (16×) consumes an order of magnitude more KV-cache memory. Why does IDC judge that “memory and storage shortages will last at least into next year”? Because agents must “remember” increasingly long contexts, while HBM/DRAM expansion cannot keep up. The value of the V900’s 216GB memory lies precisely in pushing up the curve of “how long a session can be / how many agents can run concurrently” — which, far more than raw FLOPs, determines real agent production efficiency and the per-token cost.
7.5 Python: the “training-to-agent” mismatch simulator
IDC’s Zhou Zhengang flagged a second bottleneck beyond memory: mismatch — clusters built for training, which demand very high peak FLOPs, do not fit agent inference that needs large context saving, high throughput, and low latency. The following simulator contrasts the utilization of a training-tuned pool versus an agent-tuned pool under a mixed workload:
class Pool:
def __init__(self, peak, ctx_bw, kind):
self.peak = peak # peak FLOPs capacity
self.ctx = ctx_bw # context/token bandwidth
self.kind = kind
def util(pool, jobs):
train = sum(j['type'] == 'train' and j['f'] for j in jobs)
agent = sum(j['type'] == 'agent' and j['t'] for j in jobs)
if pool.kind == 'train_tuned':
return min(1.0, train / pool.peak) # wastes agent w/ high ctx
return min(1.0, agent / pool.ctx) # wastes train w/ high peak
train_pool = Pool(peak=100, ctx_bw=10, kind='train_tuned')
agent_pool = Pool(peak=100, ctx_bw=100, kind='agent_tuned')
mixed = [{'type': 'train', 'f': 60, 't': 0},
{'type': 'agent', 'f': 10, 't': 80}]
for p in (train_pool, agent_pool):
print(p.kind, "->", util(p, mixed))
# sweep the agent share to see where each pool starts to starve
for share in (0.2, 0.5, 0.8):
jobs = [{'type': 'agent', 'f': 10, 't': 100 * share}]
print(f"agent share {share:.0%}: train_pool={util(train_pool, jobs):.2f}"
f" agent_pool={util(agent_pool, jobs):.2f}")
The output starkly shows why “invested a lot but unused” happens: a training-tuned pool reports high idle under agent workloads, while an agent-tuned pool can keep serving. This is the quantitative side of the “compute gap” that is not about missing cards at all — it is about deploying the right system. A full line-up (Zhenwu GPU, Yitian CPU, Panmai NIC, ICN) plus full-stack co-tuning is exactly an attempt to make any single rack adaptive to both training bursts and long-running agent contexts.
7.6 Go + Python: power-aware placement and the Jevons ramp
Beyond memory and system fit, the third binding constraint noted by IDC is energy. A data-center shell may be capped in months, but the regional substation and grid expansion behind it can take years. Power-aware co-placement of model shards onto racks is therefore part of “compute-gap economics.” A greedy rack-placement sketch in Go:
package main
import "fmt"
type rack struct{ gw, usedGW int }
// place tries to co-locate a GPU on a rack if power budget allows.
func place(r *rack, gb, gw int) bool {
if r.usedGW+gw > r.gw {
return false
}
r.usedGW += gw
return true
}
func main() {
r := rack{gw: 200}
ok := place(&r, 216, 20)
fmt.Println("rack placed:", ok, "usedGW:", r.usedGW)
}
And the Jevons counter-ramp — cheaper tokens stimulate more total demand, so the infrastructure must be over-provisioned in advance:
def jevons_ramp(prices, demands):
# prices: unit cost falling over years; demands: rising volume in EFLOPS.
return sum(p * d for p, d in zip(prices, demands))
prices = [10, 5, 2, 1]
demands = [2, 4, 9, 20] # placeholder EFLOPS-equivalents
print("revenue under Jevons ramp:", jevons_ramp(prices, demands))
print("satisfaction trough 2027 = 71%; gap 2030 = $380.9B")
The lesson encoded here is the strategic bet behind Alibaba’s 20GW plan: because cheaper, more efficient compute stimulates (rather than sates) demand, the economically rational move is to over-build power and compute ahead of confirmed demand — exactly what “placing racks before you know the load” means at the scale of a power utility. This is why the 2032 goal of exceeding 20GW is framed as a grid, not merely a rack count.
7.7 Go + Python: budgeted rack packing and fleet forecast
When supply is scarce and demand is steep, operators must pack work onto racks while respecting both power and memory budgets, amortizing overallocation over time. A budgeted packing pass in Go:
package main
import (
"fmt"
"sort"
)
type pool struct{ gw, memGB int }
type job struct{ gw, memGB int; name string }
func pack(racks []*pool, jobs []job) []string {
sort.Slice(jobs, func(i, j int) bool { return jobs[i].gw > jobs[j].gw })
var placed []string
for _, j := range jobs {
for _, r := range racks {
if r.gw >= j.gw && r.memGB >= j.memGB {
placed = append(placed, j.name)
r.gw -= j.gw
r.memGB -= j.memGB
break
}
}
}
return placed
}
func main() {
racks := []*pool{{250, 4096}, {120, 2048}}
jobs := []job{{96, 1536, "modelA"}, {64, 1024, "modelB"}, {32, 512, "modelC"}}
fmt.Println("placed:", pack(racks, jobs))
}
A corresponding fleet-build forecast in Python, turning the 20GW/2032 target and the widening gap into a years-ahead capacity plan:
def fleet(gw_start, target, years):
cur = gw_start
plan = []
for y in range(years):
add = max(0, (target - cur) / (years - y))
cur += add
plan.append((y + 2026, round(cur, 1)))
return plan
plan = fleet(2.0, 20.0, 6)
for y, gw in plan:
print(f"year {y}: cumulative GW = {gw}")
print("fleet (GW, year):", plan)
print("satisfaction 2024->2027->2030: 79% -> 71% -> 77%")
print("absolute gap 2024->2030: $38.9B -> $380.9B")
These two together show the operator’s dual imperative under compute-gap economics: pack the present tightly (budget power+memory) while front-loading the future buildout — precisely the logic encoded in the >20GW 2032 grid goal and the “$53B/3yr” capex ramp.
7.8 Go: a blended token-price index
Because “chip competitiveness = tokens per dollar,” it is useful to track a blended token-price index across model tiers, showing how moving work to cheaper tiers (or to more efficient self-developed silicon) presses the unit price. A weighted index in Go:
package main
import "fmt"
var tiers = []struct {
name string
price float64 // USD per 1M tokens
weight float64 // share of traffic
}{
{"qwen-small", 0.15, 0.5},
{"qwen-mid", 0.60, 0.3},
{"qwen-max", 2.40, 0.2},
}
func main() {
var idx float64
for _, t := range tiers {
idx += t.price * t.weight
}
fmt.Printf("blended token price: $%.3f/M, gap 2030: $380.9B\n", idx)
// forecast: price elasticity lowers unit cost but demand outruns it.
price, demand := 1.00, 100.0 // normalized base
for y := 0; y < 5; y++ {
price *= 0.82 // cheaper every year (efficiency up)
demand *= 2.10 // demand roughly doubles (agents/tokens)
spend := price * demand
fmt.Printf("year %d: price %.3f, demand %.0f, spend %.1f\n", y, price, demand, spend)
}
}
Setting a floor-price on tokens matters because the arithmetic of the gap is unrelenting: as long as demand grows faster than supply, the marginal token keeps its premium, and whoever owns the cheapest generator (the most tokens-per-dollar chip) both captures that premium and — per the Jevons logic — stimulates even more total demand. This is the closed loop that turns a single chip launch into a structural bet on the entire AI economy.
8. Opportunity and Risk: The B-Side of Full-Stack Breakout
Every grand narrative has a B-side. Wen Yuan Zhi Ku founder Wang Chao expressed a sober concern to Beijing Business Today: Alibaba’s “big and comprehensive” orientation — the full-chain investment spanning models, chips, and compute infrastructure simultaneously — places extremely high demands on resource coordination, technical accumulation, and commercialization capability, and faces considerable uncertainty in the current environment.Beijing Business Today
The risk list is clear:
- Exogenous HBM and advanced-packaging constraints: even if self-developed GPUs hit performance targets, memory, CoWoS packaging, and power are global bottlenecks; IDC explicitly judges “memory and storage shortages will last at least into next year.”
- 21st Century Business Herald estimated quarterly AI-related product revenue at RMB 8.971B, still a “sliver” relative to RMB 67.7B capex — the commercial payoff curve of AI revenue has yet to be proven.36Kr
- The “one-generation-per-year” commitment: V900 targets mass production in Q1 2027 and the J900 is slated for Q3 2027. The pace is fast, but production ramp-up and customer onboarding are hard battles.
- Demand-assumption risk: the 20GW 2032 target rests on the premise that large-model and agent demand maintain a steep upslope; if the demand curve slows, heavy-asset depreciation will bite cash flow.
But the other side of the coin: in a market where the compute gap keeps widening, “the ability to supply” is itself the scarcest capability. The meaning of the Zhenwu V900 goes far beyond “one domestic GPU” — it is a system-level declaration of intent in which Alibaba holds the “generator, the grid, and the appliances” in one hand. As Eddie Wu put it, today’s AI Coding may be just the electric light of 1882; but a light can only illuminate an era because of the people who built the power stations first, not the tungsten filament inside the bulb.21st Century Business Herald
Viewed across the whole chain, the V900 launch delivers a near watershed answer: as the global AI compute satisfaction rate slides toward the 71% trough in 2027 and the absolute gap races toward $380.9 billion, the value anchor of domestic chips is quietly shifting from “benchmarking a single card” to “supplying an entire compute ecosystem.” The decisive factor in the system-level breakthrough is not whether some chip’s peak FLOPs set a record, but whether one can, like Alibaba, knit chips, CPUs, networking, software stacks, cloud, models, and even end devices tightly into the same fabric. The ledger of the compute gap has never been merely an arithmetic problem — it is a contest about “who holds the very ability to produce intelligence.” The long war over “generators, the grid, and the price of electricity” has only just begun.
Sources: EET-China, TMTPost, 36Kr, People’s Daily, Sina Finance (National Business Daily), China Securities Journal, Beijing Business Today, 21st Century Business Herald, JRJ (IDC/Inspur “2026 China AI Computing Power Development Assessment Report”)