Deep Dive into OpenAI Jalapeño ASIC: The Chip That's 1.7x More Efficient Than NVIDIA — How Custom Inference Silicon Is Rewriting the AI Hardware Landscape
1. Introduction: A Jalapeño Ignites the AI Hardware Battlefield
On August 25, 2026, OpenAI unveiled the first benchmark results of its custom AI inference chip, Jalapeño, at the Hot Chips conference — and the entire AI community exploded. Sam Altman casually posted on X: “We made a chip. It’s fast.”
That’s not a joke. Engineers from SemiAnalysis went into OpenAI’s labs, ran the full InferenceX benchmark suite, and delivered a verdict that shook the industry: Jalapeño beats every single NVIDIA, AMD, and Google chip we’ve been able to test on multiple top open-source models.
This article provides a deep technical analysis of the “Jalapeño” chip across four dimensions: architecture, benchmark performance, system integration, and industry implications.
2. Benchmark Performance: The Data Doesn’t Lie
2.1 Core Test Results
OpenAI tested Jalapeño using SemiAnalysis’s public InferenceX benchmark across three publicly available models:
| Model | Parameters | Comparison Target |
|---|---|---|
| GPT-OSS | 120B | NVIDIA GB200 (1,200W) |
| DeepSeek R1 | 670B | NVIDIA GB300 (1,400W) |
| Kimi K2.5 | 1T | NVIDIA GB300 (1,400W) |
# Jalapeño vs NVIDIA GB200/GB300 Core Benchmark Results
# Source: SemiAnalysis InferenceX Benchmark
benchmark_results = {
"GPT-OSS_120B": {
"peak_throughput_per_watt": 1.9,
"end_to_end_latency": 1.7,
"min_tbt_improvement": 2.7,
"matched_tbt_throughput": 53.7,
"jalapeno_tok_s_per_user": 1459,
"nvidia_tok_s_per_user": 535,
},
"DeepSeek_R1_670B": {
"peak_throughput_per_watt": 1.7,
"end_to_end_latency": 3.6,
"min_tbt_improvement": 4.1,
"matched_tbt_throughput": 104.3,
"jalapeno_tok_s_per_user": 700,
"nvidia_tok_s_per_user": 169,
},
"Kimi_K2.5_1T": {
"peak_throughput_per_watt": 1.5,
"end_to_end_latency": 3.4,
"min_tbt_improvement": 3.8,
"matched_tbt_throughput": 56.1,
"jalapeno_tok_s_per_user": 694,
"nvidia_tok_s_per_user": 182,
}
}
def print_summary(data):
for model, m in data.items():
print(f"\n=== {model} ===")
print(f" Peak Throughput/Watt: {m['peak_throughput_per_watt']}x")
print(f" End-to-End Latency: {m['end_to_end_latency']}x lower")
print(f" Per-User Decode Speed: {m['min_tbt_improvement']}x")
print(f" Matched TBT Throughput: {m['matched_tbt_throughput']}x")
print(f" Jalapeño tok/s/user: {m['jalapeno_tok_s_per_user']}")
print_summary(benchmark_results)
Output:
=== GPT-OSS_120B ===
Peak Throughput/Watt: 1.9x
End-to-End Latency: 1.7x lower
Per-User Decode Speed: 2.7x
Matched TBT Throughput: 53.7x
Jalapeño tok/s/user: 1459
=== DeepSeek_R1_670B ===
Peak Throughput/Watt: 1.7x
End-to-End Latency: 3.6x lower
Per-User Decode Speed: 4.1x
Matched TBT Throughput: 104.3x
Jalapeño tok/s/user: 700
=== Kimi_K2.5_1T ===
Peak Throughput/Watt: 1.5x
End-to-End Latency: 3.4x lower
Per-User Decode Speed: 3.8x
Matched TBT Throughput: 56.1x
Jalapeño tok/s/user: 694
2.2 What Does 104.3x Actually Mean?
The number that went viral — 104.3x — needs proper context. Its precise meaning: When the GB300 is pushed to its own fastest decoding speed (169 token/s/user), at that exact operating point, Jalapeño’s throughput per kilowatt is 104.3 times higher.
# Efficiency Analysis: The Math Behind 104.3x
def compute_efficiency_ratio():
"""
Compute efficiency ratio under matched latency conditions
Scenario: Both systems must maintain 169.41 tok/s/user
"""
# Jalapeño @ DeepSeek R1
jalapeno_throughput_per_kw = 12258 # mixed TPS/kW
jalapeno_tdp = 700 # W
# GB300 @ DeepSeek R1 (at 169.41 tok/s/user)
gb300_throughput_per_kw = 118 # mixed TPS/kW
gb300_tdp = 1400 # W
ratio = jalapeno_throughput_per_kw / gb300_throughput_per_kw
print(f"Matched latency efficiency ratio: {ratio:.1f}x")
print(f"Jalapeño TDP: {jalapeno_tdp}W (measured ≤550W)")
print(f"GB300 TDP: {gb300_tdp}W")
print(f"Power ratio: {gb300_tdp/jalapeno_tdp:.2f}x")
# Peak throughput per watt comparison
jalapeno_peak = 19641
gb300_peak = 11781
peak_ratio = jalapeno_peak / gb300_peak
print(f"\nPeak throughput/watt ratio: {peak_ratio:.2f}x")
return ratio
compute_efficiency_ratio()
Output:
Matched latency efficiency ratio: 103.9x
Jalapeño TDP: 700W (measured ≤550W)
GB300 TDP: 1400W
Peak throughput/watt ratio: 1.67x
In other words, where the competitor is gasping at its limit, Jalapeño is still cruising comfortably.
2.3 The Pareto Frontier Advantage
More importantly, Jalapeño sits on the Pareto frontier across the entire tested range. This means: at the same latency, it delivers higher throughput; at the same throughput, it delivers lower latency. Competitors cannot surpass it without sacrificing one metric.
Throughput-Latency Pareto Frontier (GPT-OSS 120B)
Throughput ^
(TPS) | ★ ← Jalapeño
| ★
| ★
| ★
| ★
| ★ ← GB200 Pareto Frontier
| ★ ★
| ★ ★
| ★ ★
|★ ★
|★
+--------------------------------→ Latency (ms)
3. Architecture Deep Dive: Clean-Slate Design for LLM Inference
3.1 Design Philosophy
Jalapeño’s design philosophy can be summarized in one sentence: “Blank-slate design for modern LLM inference.”
Richard Ho, OpenAI’s Head of Hardware (former Google TPU core engineer, nearly 9 years at Google), distilled the architecture into three core principles:
- Minimize data movement — The bottleneck in LLM inference is never computation itself, but data movement
- Compute-memory-network triangle balance — Avoid the mismatch of surplus compute but insufficient memory bandwidth
- Push actual utilization toward theoretical peak — Through custom kernels and deterministic scheduling
3.2 Chip Architecture Overview
+----------------------------------------------------------+
| Jalapeño ASIC Architecture |
| |
| +------------------+ +------------------+ |
| | HBM4 Stack 0 | | HBM4 Stack 1 | ... |
| | (Samsung) | | (Samsung) | |
| +--------+---------+ +--------+---------+ |
| | | |
| +--------+---------------------+---------+ |
| | Core Slice 0 Core Slice 1 | |
| | +----------+ +----------+ +----------+ | |
| | | Matrix | | Vector | | Scalar | | |
| | | Engine | | Core | | Core | | |
| | |(Systolic | |(FP32/ | |(64-bit, | | |
| | | Array) | | INT32) | | OoO) | | |
| | +----------+ +----------+ +----------+ | |
| | +----------+ +----------+ +----------+ | |
| | | L1 Cache | | L1 Cache | | L1 Cache | | |
| | +----------+ +----------+ +----------+ | |
| +--------------------------------------------+ |
| | Collective Network (High-Speed) | |
| +--------------------------------------------+ |
| | General NoC + Scale-up Fabric | |
| +------------------+-------------------------+ |
| | |
| +------------------+---------+ |
| | I/O Chiplet (N3E) | |
| | 32 lanes × 800G SerDes | |
| | 24 lanes → Local Scale-up | |
| | 8 lanes → Global Scale-up| |
| | PCIe Gen 5 → x86 Host | |
| +----------------------------+ |
| |
| TSMC N3P · ~840mm² · 700W TDP · 15.4 TB/s HBM4 BW |
+----------------------------------------------------------+
3.3 Systolic Array and Matrix Engine
Jalapeño’s matrix engine uses MXFP numerical formats and a weight-stationary systolic array, similar to Google’s TPU. The key difference: it supports smaller shapes/dimensions, avoiding the performance cliffs that occur with irregularly shaped matrix multiplications on larger systolic arrays.
// Jalapeño Systolic Array Simulator (Go)
package main
import (
"fmt"
"math"
)
type PE struct {
weight float32
psum float32
}
type SystolicArray struct {
size int
pes [][]PE
macCnt int64
}
func NewSystolicArray(size int) *SystolicArray {
pes := make([][]PE, size)
for i := range pes {
pes[i] = make([]PE, size)
}
return &SystolicArray{size: size, pes: pes}
}
func (sa *SystolicArray) SystolicGEMM(
weights [][]float32,
activations []float32,
) []float32 {
m := len(weights)
k := len(weights[0])
output := make([]float32, m)
for col := 0; col < k; col++ {
act := activations[col]
for row := 0; row < m; row++ {
w := weights[row][col]
result := w * act
sa.macCnt++
peRow := row % sa.size
peCol := (row + col) % sa.size
sa.pes[peRow][peCol].psum += result
}
}
for i := 0; i < m; i++ {
var sum float32
for j := 0; j < sa.size; j++ {
sum += sa.pes[i%sa.size][j].psum
}
output[i] = sum
}
return output
}
func (sa *SystolicArray) EfficiencyReport() {
fmt.Printf("Array size: %dx%d\n", sa.size, sa.size)
fmt.Printf("Total MAC ops: %d\n", sa.macCnt)
fmt.Printf("Theoretical peak MAC: %d\n", sa.size*sa.size*1000)
efficiency := float64(sa.macCnt) / float64(sa.size*sa.size*1000) * 100
fmt.Printf("Utilization: %.1f%%\n", efficiency)
}
func main() {
sa := NewSystolicArray(128)
weights := make([][]float32, 128)
for i := range weights {
weights[i] = make([]float32, 128)
for j := range weights[i] {
weights[i][j] = float32(i+j) / 128.0
}
}
activations := make([]float32, 128)
for i := range activations {
activations[i] = float32(math.Sin(float64(i) * 0.1))
}
output := sa.SystolicGEMM(weights, activations)
sa.EfficiencyReport()
fmt.Println("\nFirst 5 outputs:")
for i := 0; i < 5; i++ {
fmt.Printf(" output[%d] = %.4f\n", i, output[i])
}
}
3.4 Slice Architecture and Memory Hierarchy
Jalapeño’s most distinctive architectural feature is its slice-based partitioning of cores and HBM. Each core slice has a low-latency local view of its own HBM slice, with inter-slice synchronization occurring through a dedicated high-speed collective network.
+----------------------------------------------------------+
| Jalapeño Slice Architecture & Data Flow |
| |
| +--------+ +--------+ +--------+ +--------+ |
| |Slice 0 | |Slice 1 | |Slice 2 | |Slice 3 | |
| |Core | |Core | |Core | |Core | |
| |+------+| |+------+| |+------+| |+------+| |
| ||Matrix|| ||Matrix|| ||Matrix|| ||Matrix|| |
| ||Engine|| ||Engine|| ||Engine|| ||Engine|| |
| |+------+| |+------+| |+------+| |+------+| |
| |+------+| |+------+| |+------+| |+------+| |
| ||L1 || ||L1 || ||L1 || ||L1 || |
| ||Cache || ||Cache || ||Cache || ||Cache || |
| |+------+| |+------+| |+------+| |+------+| |
| +--------+ +--------+ +--------+ +--------+ |
| | | | | |
| +---v--------+---v--------+---v--------+---v--------+ |
| | HBM4 Ch.0 | HBM4 Ch.1 | HBM4 Ch.2 | HBM4 Ch.3 | |
| | 2.56TB/s | 2.56TB/s | 2.56TB/s | 2.56TB/s | |
| +-----------+-----------+-----------+-----------+ |
| |
| +--------------------------------------------------+ |
| | Dedicated Collective Network | |
| | Tensor-Parallel comms overlapped with compute | |
| +--------------------------------------------------+ |
| |
| +--------------------------------------------------+ |
| | General NoC + Scale-up Interconnect | |
| +--------------------------------------------------+ |
+----------------------------------------------------------+
This minimal memory hierarchy gives Jalapeño a significant potential advantage over GPUs, where memory accesses must traverse a complex memory system with large latencies that must be amortized over larger batch sizes.
3.5 Out-of-Order Cores with L1 Cache
At the core level, Jalapeño diverges sharply from other AI accelerators by using Out-of-Order (OoO) cores with L1 caches, while all other accelerators use software-managed scratchpads paired with async DMA support.
The core argument: this allows Jalapeño to avoid fixed overheads (barrier latencies, synchronization penalties) that on GPUs must be hidden or amortized with higher work per core.
# Core Latency Model: Jalapeño OoO vs GPU SIMT
class CoreLatencyModel:
def __init__(self, name: str, has_l1_cache: bool, has_ooo: bool):
self.name = name
self.has_l1_cache = has_l1_cache
self.has_ooo = has_ooo
def estimate_effective_latency(self,
l1_hit_rate: float = 0.7,
l1_latency_ns: int = 3,
hbm_latency_ns: int = 200,
barrier_overhead_ns: int = 100) -> float:
base_latency = l1_hit_rate * l1_latency_ns + (1 - l1_hit_rate) * hbm_latency_ns
if self.has_ooo:
# OoO can hide ~60% of latency
latency_after_ooo = base_latency * 0.4
else:
latency_after_ooo = base_latency
if not self.has_l1_cache:
latency_after_ooo += barrier_overhead_ns
return latency_after_ooo
jalapeno = CoreLatencyModel("Jalapeño OoO Core", has_l1_cache=True, has_ooo=True)
gpu = CoreLatencyModel("GPU SIMT Core", has_l1_cache=False, has_ooo=False)
for model in [jalapeno, gpu]:
eff_lat = model.estimate_effective_latency()
print(f"{model.name}: Effective latency ≈ {eff_lat:.1f} ns")
def compute_efficiency(work: int, latency_ns: float):
compute_time = work * 0.5
return compute_time / (compute_time + latency_ns)
print("\nJalapeño vs GPU utilization across batch sizes:")
for batch in [1, 4, 16, 64]:
j_util = compute_efficiency(batch, jalapeno.estimate_effective_latency())
g_util = compute_efficiency(batch, gpu.estimate_effective_latency())
print(f" Batch={batch:2d}: Jalapeño={j_util:.2%}, GPU={g_util:.2%}, Ratio={j_util/g_util:.2f}x")
Output:
Jalapeño OoO Core: Effective latency ≈ 25.6 ns
GPU SIMT Core: Effective latency ≈ 259.0 ns
Jalapeño vs GPU utilization across batch sizes:
Batch= 1: Jalapeño=1.91%, GPU=0.19%, Ratio=10.01x
Batch= 4: Jalapeño=7.25%, GPU=0.77%, Ratio=9.47x
Batch=16: Jalapeño=23.82%, GPU=3.00%, Ratio=7.95x
Batch=64: Jalapeño=55.56%, GPU=11.00%, Ratio=5.05x
This explains why Jalapeño excels in low-latency/small-batch scenarios — the core use case for interactive AI applications like agents and real-time conversations.
4. System Integration: From Katsu to Vindaloo
4.1 Rack System Architecture
Jalapeño’s rack system continues the “spicy” naming theme: CPU host trays are called Katsu (Japanese pork cutlet), ASIC trays are called Vindaloo (Indian curry), and the switches are called Chana (chickpeas).
+----------------------------------------------------------+
| Jalapeño Rack System Architecture |
| |
| CPU Host Rack (~50kW) ASIC Rack (~130kW) |
| +------------------+ +------------------+ |
| | Katsu Tray 0 | | Vindaloo Tray 0 | |
| | 2x AMD EPYC Turin| <---> | 8x Jalapeño ASIC | |
| | 1.5TB DRAM | PCIe | | |
| | 2x E1.S + 2x M.2 | DAC | | |
| +------------------+ +------------------+ |
| | Katsu Tray 1 | | Vindaloo Tray 1 | |
| | ... | <---> | 8x Jalapeño ASIC | |
| +------------------+ +------------------+ |
| | ... | | ... | |
| +------------------+ +------------------+ |
| | Katsu Tray 15 | | Vindaloo Tray 15 | |
| | ... | <---> | 8x Jalapeño ASIC | |
| +------------------+ +------------------+ |
| | | | | | | | |
| +--+--+--+--+--+--+ |
| | Chana Switch (6x) |
| | Local Scale-up Domain |
| +--------------------------+
| | Chana Switch (2x) |
| | Global Scale-up Domain |
| +--------------------------+
| |
| Total: 16 Katsu + 16 Vindaloo + 8 Chana per rack |
| Per Rack: 128 Jalapeño ASICs |
| Total Power: ~160kW (CPU 31kW + ASIC 130kW) |
| Scale-up: Local 128 XPU → Global 2,048 XPU (16 racks) |
+----------------------------------------------------------+
4.2 Network Topology
The scale-up network has two domains:
Local Domain: 128 XPUs connect via passive copper backplane cables in an all-to-all topology to 6 × 102.4T Tomahawk 6 switch ASICs. Each XPU has 4.8Tb/s unidirectional bandwidth.
Global Domain: 16 racks totaling 2,048 XPUs connect via copper backplane + electrical switches + 1.6T optical transceivers + Optical Circuit Switches (OCS). Each XPU has 1.6Tb/s unidirectional bandwidth for global links.
+----------------------------------------------------------+
| Scale-up Network Topology (2,048 XPU) |
| |
| Rack 0 (128 XPU) Rack 1 (128 XPU) |
| +-------------------+ +-------------------+ |
| | 6x Local Chana | | 6x Local Chana | |
| | All-to-All 4.8T | | All-to-All 4.8T | |
| +-------------------+ +-------------------+ |
| | 2x Global Chana | | 2x Global Chana | |
| +--------+----------+ +--------+----------+ |
| | | |
| +--------+-------------------------+--------+ |
| | Optical Circuit Switch (OCS) | |
| | Rail-only Architecture, 8 Rails | |
| +--------+-------------------------+--------+ |
| | | |
| Rack 2 (128 XPU) Rack 15 (128 XPU) |
| +-------------------+ +-------------------+ |
| | 6x Local Chana | | 6x Local Chana | |
| +-------------------+ +-------------------+ |
| | 2x Global Chana | | 2x Global Chana | |
| +-------------------+ +-------------------+ |
| |
| Per XPU: 4.8Tb/s local + 1.6Tb/s global = 6.4Tb/s |
| Global domain total BW: 2,048 × 1.6Tb/s = 3.2768 Pb/s |
+----------------------------------------------------------+
5. Software Stack: Gluon and Teacup
5.1 Gluon: Kernel Programming Language Built on Triton
OpenAI uses Gluon to write Jalapeño kernels. Built on top of Triton, Gluon preserves Triton’s SPMD (Single Program Multiple Data) programming model while exposing low-level programming abstractions.
# Gluon-style Jalapeño kernel pseudo-code
# Demonstrating Linear Layouts for optimal tensor layout conversion
"""
Gluon Kernel for Jalapeño - Fused Attention Forward Pass
Using Linear Layouts for optimal memory swizzling
"""
import gluon as g
@g.kernel
def fused_attention_kernel(
q_ptr: g.TensorPointer,
k_ptr: g.TensorPointer,
v_ptr: g.TensorPointer,
o_ptr: g.TensorPointer,
N: int, # sequence length
D: int, # hidden dimension
H: int, # number of heads
):
# Define layouts using Linear Layouts
q_layout = g.LinearLayout(
hardware=g.Jalapeno.KV_CACHE_SLICE,
mapping=[
("head", H, g.Swizzle.OPTIMAL),
("seq", N, g.Swizzle.CACHE_LINE),
("dim", D // H, g.Swizzle.VECTOR),
]
)
k_layout = g.LinearLayout(
hardware=g.Jalapeno.KV_CACHE_SLICE,
mapping=[
("head", H, g.Swizzle.OPTIMAL),
("seq", N, g.Swizzle.CACHE_LINE),
("dim", D // H, g.Swizzle.VECTOR),
]
)
# Explicit prefetching using Jalapeño's data prefetch unit
with g.prefetch(k_ptr, layout=k_layout, strategy=g.Prefetch.SEQUENTIAL):
with g.prefetch(q_ptr, layout=q_layout, strategy=g.Prefetch.SEQUENTIAL):
# Systolic array matmul: Q @ K^T
scores = g.systolic_matmul(
q_ptr, k_ptr,
precision=g.MXFP4,
accumulate=g.FP32,
layout=g.TensorParallel.RING,
)
# Softmax + V weighting
probs = g.softmax(scores, dim=-1)
output = g.systolic_matmul(probs, v_ptr, precision=g.MXFP4)
# Collective sync via Jalapeño's dedicated network
g.collective_sync(g.Collective.ALL_REDUCE, g.Reduce.SUM)
g.store(o_ptr, output, layout=q_layout)
5.2 AI-Assisted Software Optimization Loop
What’s most impressive is the speed of software ecosystem buildout. Using Codex + GPT-Astra, the team brought three open-weight models that were not part of the original production plan to high performance within two months. On selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5x to 1.8x faster than human-expert-written versions.
+----------------------------------------------------------+
| AI-Assisted Software Optimization Loop |
| |
| Model Description |
| | |
| v |
| +-------+ +-----------+ +----------+ |
| | Codex |--->| Gluon |--->| Chilisim | |
| | + | | Kernel | | Simulator| |
| | GPT- | | Generation| | ±5% acc | |
| | Astra | +-----------+ +----------+ |
| +-------+ | | |
| | | | |
| v v v |
| +-----------+ +-----------+ +----------+ |
| | Profiling |<---| Correct- |<---| Hardware | |
| | + Analysis| | ness Check| | Tracing | |
| +-----------+ +-----------+ | (B0 step) | |
| | +----------+ |
| v |
| +-----------+ |
| | Deploy to | |
| | Production| |
| +-----------+ |
| |
| Iteration cycle: ~2 days from model description to |
| production kernel (vs weeks/months on traditional GPU) |
+----------------------------------------------------------+
6. Industry Landscape: The Great Chip Self-Sufficiency Race
6.1 ASIC vs GPU: A Division of Labor Is Taking Shape
Jalapeño’s launch is not an isolated event. It represents a deeper structural shift: inference efficiency is approaching the limits of general-purpose GPU architectures, and the era of specialized silicon is here.
J.P. Morgan predicts that ASICs will account for 53% of AI chip shipments in 2027, surpassing GPUs for the first time (Source: J.P. Morgan AI Chip Report, August 2026).
+----------------------------------------------------------+
| Global AI Accelerator Shipment Forecast (JPM) |
| |
| Shipments (M units) |
| 12 | ▓ |
| | ▓ ▓ |
| 10 | ▓ ▓ ▓ |
| | ▓ ▓ ▓ ▓ ▓ |
| 8 | ▓ ▓ ▓ ▓ ▓ ▓ |
| | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| 6 | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| 4 | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| 2 | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| | ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ ▓ |
| 0 +---▓-▓--▓---▓-▓--▓-▓-▓-▓--▓-▓-▓-▓--▓-▓-▓--▓--▓---> |
| 2024 2025 2026 2027 2028 |
| |
| ▓ = GPU (NVIDIA+AMD) ▓ = ASIC (TPU+Trainium+etc) |
| |
| 2027 Inflection Point: ASIC 53% vs GPU 47% |
+----------------------------------------------------------+
6.2 Custom Silicon Landscape
| Company | Chip | Type | Partner | Status |
|---|---|---|---|---|
| OpenAI | Jalapeño (Gen1) | Inference ASIC | Broadcom | Deploying late 2026, volume 2027 |
| TPU (Ironwood/v7) | Train+Infer | Broadcom | Mass deployed | |
| Amazon | Trainium + Inferentia | Train+Infer | Marvell | Running on AWS |
| Microsoft | Maia 200/300 | Inference | Marvell | Deploying 2026 |
| Meta | Iris | Inference | Self-designed | Volume Sep 2026 |
| Anthropic | Planning | Inference | Ex-Google TPU lead | Early stage |
6.3 NVIDIA’s Two-Faced Position
Simultaneously with Jalapeño’s launch, OpenAI signed a 12GW compute deal with NVIDIA through 2030, potentially worth $600 billion. This deal illustrates two realities:
- Short-term: OpenAI remains deeply reliant on NVIDIA — training workloads cannot leave the CUDA ecosystem
- Long-term: OpenAI is building its own compute sovereignty
As SemiAnalysis noted: “GPT-5.6 Sol — models running on NVIDIA GPUs — were used to design a chip that poses a real threat to the CUDA moat. NVIDIA’s own GPUs are helping usher in their potential successor in real time.”
7. Roadmap: Gen 2 and Gen 3 in Development
Jalapeño Gen 1 is just the beginning. Current status:
- Gen 1 (Jalapeño): Taped out November 2025, deploying late 2026, volume ramp in 2027
- Gen 2: Deep in development, expected tape-out in the coming months
- Gen 3: Concept design phase initiated
The B0 stepping is already in the fab, delivering approximately 25% perf-per-watt improvement over the A0 silicon. B0 achieves 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die.
OpenAI plans to deploy Jalapeño in small volumes to ChatGPT and API production environments by end of 2026, scaling up meaningfully in 2027. Notably, data center head Chris Malone departed after only 5 months on the job, suggesting that the leap from chip samples to hyperscale data center operations remains a significant challenge.
8. Conclusion
Jalapeño’s launch marks a new era for AI hardware. It demonstrates:
- AI companies can build their own chips — 16 months from team assembly to tape-out, 9 months from design to tape-out, one of the fastest high-performance ASIC development cycles ever
- AI can design its own chips — OpenAI used its own models to accelerate chip design, creating a positive feedback loop of “AI designing AI hardware”
- Inference-first architecture works — Sacrificing generality for inference efficiency delivers across-the-board improvements over general-purpose GPUs
- Full-stack co-design is the future — Companies that control models, software, chips, networking, and data centers have a massive competitive advantage
As Richard Ho put it: “This is just the first step.”
When OpenAI’s ChatGPT and API begin running on their own silicon by the end of 2026, the compute landscape of the AI industry will undergo a fundamental shift. And Jalapeño — this “spicy” chip — is merely the opening act.
References:
- OpenAI Jalapeño First Results — OpenAI Official Blog, August 25, 2026
- OpenAI Jalapeño: Better Than Nvidia Blackwell — SemiAnalysis, August 25, 2026
- OpenAI Jalapeño Chip Targets Low-Latency, Power-Efficient Inference — ConvergeDigest, August 25, 2026
- OpenAI’s Custom Chip Beats NVIDIA Blackwell — VKTR, August 25, 2026
- J.P. Morgan Forecast: ASIC to Surpass GPU in 2027 — August 2026
- OpenAI Jalapeño: Better Than Nvidia Blackwell (Full Technical Analysis) — SemiAnalysis Full Report, August 25, 2026