Tencent Hunyuan Hy4 Preview Deep Dive: 770B MoE, Recursive Self-Improvement, and the Chinese LLM Blitz

1. Introduction: The 770B “Productivity Manifesto”

On August 28, 2026, Tencent Hunyuan officially released and open-sourced Hy4 preview — a next-generation large language model with 770B total parameters, 49B activated parameters per token, a native 1M token context window, and both BF16 and FP8 weights released under the Apache 2.0 license. On launch day, vLLM and SGLang shipped official Docker images, and domestic Chinese AI chips (Ascend, Enflame) achieved 0-day adaptation. The message was clear: Tencent is entering the open-source frontier with monthly-level iteration cadence.

On the same day, Alibaba’s Qwen3.8-Flash and Zhipu’s GLM-5.3-Flash were also released. Three models, one day — not a coincidence. Chinese LLM competition has evolved from “annual releases” to “weekly updates.”

Hy4 preview’s positioning is unambiguous: built for productivity. Rather than pursuing general-purpose conversational omnipotence, it anchors on four scenarios — software engineering, office analytics, game development, and scientific research — deeply integrated with Tencent’s product ecosystem (WorkBuddy, CodeBuddy, Yuanbao, ima). In a blind evaluation organized by Tencent with 163 internal experts across 203 engineering tasks, Hy4 preview scored 2.99/4.00, slightly ahead of GLM-5.3 (2.92) and Kimi K3 (2.94). On the DeepSWE benchmark, it achieved 64.3, surpassing DeepSeek V4 Pro’s 62.7.

But what truly excited the technical community wasn’t the numbers — it was the fact that Hy4 preview participated in its own R&D process for the first time, establishing a preliminary “recursive self-improvement loop.” This article provides a deep technical analysis of its architecture, core innovations, and industry impact.

2. MoE Architecture Deep Dive: 256 Routed Experts + Shared Expert

2.1 Architecture Overview

Hy4 preview’s backbone consists of 78 Transformer layers. The first layer uses a standard dense FFN, while the remaining 77 layers are MoE layers. Each MoE layer contains 256 routed experts + 1 shared expert, with every token activating Top-8 routed experts + the shared expert — 9 experts active per layer out of 257 available.

Input Token Sequence
        │
        ▼
┌─────────────────────────────────┐
│  Layer 1: Dense FFN             │
│  (Standard Feed-Forward)        │
└──────────────────┬──────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────────────┐
│  Layers 2-78: MoE (256 Routed + 1 Shared / layer)    │
│                                                      │
│        ┌─────── Router ───────┐                      │
│        │  Top-8 Selection     │                      │
│        └──┬───┬───┬───┬───┬──┘                      │
│           │   │   │   │   │                          │
│    ┌──┐ ┌──┐ ┌──┐ ┌──┐ ┌──┐    ┌──────┐            │
│    │E1│ │E3│ │E7│...│E8│ │E │    │Shared│            │
│    └──┘ └──┘ └──┘ └──┘ └──┘    │Expert│            │
│       Top-8 Routed Experts      └──────┘            │
│                                                      │
│  Output = Σ(Top-8 Routed Experts) + Shared Expert    │
└──────────────────┬───────────────────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────────────────┐
│  MTP Layer (10B total / 0.7B active)                 │
│  Multi-Token Prediction → Speculative Decoding       │
└──────────────────┬───────────────────────────────────┘
                   │
                   ▼
            Output Token Sequence

2.2 The Routing Mechanism

Hy4 preview’s gating router is essentially a learnable linear transformation with Softmax normalization. Each token’s hidden state passes through the router network to compute affinity scores with 256 experts, then selects the Top-8 for activation:

import torch
import torch.nn.functional as F

class MoERouter(torch.nn.Module):
    """Hy4 preview-style MoE routing module"""
    def __init__(self, hidden_size: int, num_experts: int = 256, top_k: int = 8):
        super().__init__()
        self.num_experts = num_experts
        self.top_k = top_k
        self.gate = torch.nn.Linear(hidden_size, num_experts, bias=False)
        
    def forward(self, x: torch.Tensor):
        # x: (batch_size, seq_len, hidden_size)
        scores = self.gate(x)  # (batch, seq, num_experts)
        
        # Optional load-balancing noise (training only)
        if self.training:
            noise = torch.randn_like(scores) * 0.01
            scores = scores + noise
        
        # Top-8 selection
        top_k_weights, top_k_indices = torch.topk(
            F.softmax(scores, dim=-1), 
            k=self.top_k, 
            dim=-1
        )
        
        return top_k_weights, top_k_indices, scores

2.3 The Shared Expert Design Rationale

The shared expert is a key architectural differentiator. In traditional MoE, each token activates only routed experts, potentially creating “information islands” — general knowledge gets fragmented across experts, leading to utilization imbalance. The shared expert serves as an always-active common channel that:

1. Reduces routing burden: General knowledge handled uniformly, letting routed experts specialize
2. Mitigates expert collapse: Shared expert absorbs base patterns, preventing routed expert degradation
3. Improves inference efficiency: Shared expert outputs can be pre-cached, reducing dynamic routing overhead

class MoELayer(torch.nn.Module):
    """Single MoE layer implementation matching Hy4 preview design"""
    def __init__(self, hidden_size: int, moe_intermediate_size: int = 2048,
                 num_experts: int = 256, top_k: int = 8):
        super().__init__()
        self.router = MoERouter(hidden_size, num_experts, top_k)
        # 256 routed experts
        self.experts = torch.nn.ModuleList([
            ExpertFFN(hidden_size, moe_intermediate_size)
            for _ in range(num_experts)
        ])
        # 1 shared expert (always active)
        self.shared_expert = ExpertFFN(hidden_size, moe_intermediate_size)
        
    def forward(self, x: torch.Tensor):
        # Shared expert output (always computed)
        shared_output = self.shared_expert(x)
        
        # Routed expert selection
        top_k_weights, top_k_indices, _ = self.router(x)
        
        # Sparse activation: compute only selected experts
        # In production, this uses parallel dispatch mechanisms
        batch, seq, hidden = x.shape
        routed_output = torch.zeros_like(x)
        for b in range(batch):
            for s in range(seq):
                for k in range(self.router.top_k):
                    expert_idx = top_k_indices[b, s, k]
                    weight = top_k_weights[b, s, k]
                    expert_out = self.experts[expert_idx](x[b:b+1, s:s+1])
                    routed_output[b, s] += weight * expert_out[b, 0]
        
        return routed_output + shared_output

2.4 Parameter Scale and Activation Ratio

Hy4 preview’s parameter efficiency is striking: 770B total with only 49B active, an activation ratio of ~6.36%. In contrast, a dense model of comparable capability would need to activate all parameters, with inference costs scaling linearly. This “large library, small usage” MoE approach is the core reason Hy4 preview can price API at just ¥6/million input tokens.

3. Gated DSA Sparse Attention and IndexCache: The Engineering Secret Behind 1M Context

3.1 The Long-Context Challenge

Standard Full Attention has O(n²) complexity. At n = 1M, a single forward pass would require ~10¹² operations — infeasible for practical inference. Hy4 preview employs Gated DeepSeek Sparse Attention (Gated DSA) to solve this.

3.2 How Gated DSA Works

Standard Attention (Dense):
    Q · K^T → All positions → O(n²) compute
    
Gated DSA (Sparse):
    Q · K_idx^T → Top-k positions only → O(n·k), k << n
    
    Step 1: Indexer selects Top-k relevant key positions per query
    Step 2: Compute attention only on selected k positions
    Step 3: Gating mechanism fuses sparse attention with global information

The core idea: filter first, compute second. A lightweight Indexer network selects the k most relevant key positions (k = 2048) for each query, then attention is computed only on those k positions. This reduces complexity from O(n²) to O(n·k), saving ~500x computation at 1M context.

import torch
import torch.nn.functional as F

class GatedDSA(torch.nn.Module):
    """Simplified Gated DeepSeek Sparse Attention implementation"""
    def __init__(self, hidden_size: int = 6144, num_heads: int = 64,
                 num_indexer_heads: int = 32, top_k: int = 2048):
        super().__init__()
        self.num_heads = num_heads
        self.num_indexer_heads = num_indexer_heads
        self.top_k = top_k
        self.head_dim = hidden_size // num_heads
        
        # QKV projections
        self.q_proj = torch.nn.Linear(hidden_size, hidden_size)
        self.k_proj = torch.nn.Linear(hidden_size, hidden_size)
        self.v_proj = torch.nn.Linear(hidden_size, hidden_size)
        
        # Query compression (query → 2048 dim)
        self.q_compress = torch.nn.Linear(hidden_size, 2048)
        # KV compression (key/value → 512 dim)
        self.kv_compress = torch.nn.Linear(hidden_size, 512)
        
        # Indexer: 32 heads, 128 dim each
        self.indexer = torch.nn.Linear(2048, num_indexer_heads * 128)
        self.indexer_top_k = top_k
        
        # Gating network
        self.gate = torch.nn.Sequential(
            torch.nn.Linear(self.head_dim, 1),
            torch.nn.Sigmoid()
        )
        
    def forward(self, x: torch.Tensor, index_cache=None):
        batch, seq, _ = x.shape
        
        q = self.q_proj(x)
        k = self.k_proj(x)
        v = self.v_proj(x)
        
        q_compressed = self.q_compress(x)
        
        # Indexer: generate sparse attention indices
        indexer_out = self.indexer(q_compressed)
        if index_cache is not None:
            # Cross-layer index reuse
            sparse_indices = index_cache
        else:
            indexer_scores = torch.matmul(
                indexer_out, 
                indexer_out.transpose(-2, -1)
            )
            _, sparse_indices = torch.topk(
                indexer_scores, 
                k=self.indexer_top_k, 
                dim=-1
            )
        
        # Sparse attention computation on selected positions only
        k_sparse = torch.gather(
            k.unsqueeze(1).expand(-1, self.num_heads, -1, -1),
            dim=2,
            index=sparse_indices.unsqueeze(-1).expand(-1, -1, -1, self.head_dim)
        )
        v_sparse = torch.gather(
            v.unsqueeze(1).expand(-1, self.num_heads, -1, -1),
            dim=2,
            index=sparse_indices.unsqueeze(-1).expand(-1, -1, -1, self.head_dim)
        )
        
        attn_scores = torch.matmul(
            q.view(batch, -1, self.num_heads, self.head_dim).transpose(1, 2),
            k_sparse.transpose(-2, -1)
        ) / (self.head_dim ** 0.5)
        attn_weights = F.softmax(attn_scores, dim=-1)
        
        # Gating mechanism
        gate_values = self.gate(attn_weights.mean(dim=-1, keepdim=True))
        attn_weights = attn_weights * gate_values
        
        attn_output = torch.matmul(attn_weights, v_sparse)
        attn_output = attn_output.transpose(1, 2).contiguous().view(batch, seq, -1)
        
        return attn_output, sparse_indices

3.3 IndexCache: Cross-Layer Sparse Index Reuse

IndexCache is the companion optimization to Gated DSA. In deep Transformers, attention patterns across adjacent layers tend to be similar — if layer l finds position i relevant to position j, layer l+1 likely agrees. IndexCache caches sparse indices from earlier layers and reuses them in later layers, avoiding expensive recomputation:

Without IndexCache:
  Layer 2: Compute Indexer → Select Top-2048 → Attention
  Layer 3: Compute Indexer → Select Top-2048 → Attention  (Redundant!)
  Layer 4: Compute Indexer → Select Top-2048 → Attention  (Redundant!)
  ...

With IndexCache:
  Layer 2: Compute Indexer → Select Top-2048 → Attention
  Layer 3: Reuse IndexCache → Attention                  (Skipped!)
  Layer 4: Reuse IndexCache → Attention                  (Skipped!)
  ...
  Layer N: Recompute Indexer → Update Cache → Attention

This “skip-and-reuse” strategy saves ~O(n·k) computation per skipped layer while maintaining 99%+ attention quality in long-context scenarios.

3.4 iHC Residual Hyper-Connections

Hy4 preview uses iHC (identity Hyper-Connections), extending the standard single residual stream to 4 parallel residual streams. This is equivalent to laying 4 “information highways” across the 78-layer deep network, mitigating gradient dispersion:

Standard Residual Connection:
  x_{l+1} = x_l + FFN(LN(x_l))
  
iHC (4 Residual Streams):
  [x₁, x₂, x₃, x₄]_{l+1} = [x₁, x₂, x₃, x₄]_l + F([x₁, x₂, x₃, x₄]_l)
  
  where F is the Transformer sublayer (attention or FFN),
  input is the concatenation of 4 streams, output distributed back to 4 streams

4. Multi-Token Prediction (MTP) and Speculative Decoding

4.1 The MTP Principle

Hy4 preview embeds a native MTP layer (10B total / 0.7B active parameters) specifically for speculative decoding. The key differentiator: MTP is natively built into the weights, not bolted on as an inference-framework afterthought.

The core idea of speculative decoding: use a lightweight draft model to quickly generate multiple candidate tokens, then have the main model verify them in parallel. If the draft has high hit rate, multiple tokens can be confirmed in a single forward pass:

Standard Autoregressive Decoding (one token at a time):
  Token 1 → Token 2 → Token 3 → Token 4 → ...  (1 forward pass per step)
  
MTP Speculative Decoding (batch verification):
  Draft:  [Token 2, Token 3, Token 4]  (draft prediction)
  Verify: Parallel verification          (1 forward pass)
  Accept: [Token 2 ✓, Token 3 ✓, Token 4 ✗]
  → 2 tokens confirmed, rollback to Token 3

4.2 MTP Sampling Implementation

import torch
import torch.nn.functional as F

class MTPDecoder:
    """Hy4 preview native MTP speculative decoding"""
    def __init__(self, main_model, mtp_module, 
                 num_speculative_tokens: int = 3):
        self.main_model = main_model
        self.mtp_module = mtp_module  # 10B params / 0.7B active
        self.num_speculative_tokens = num_speculative_tokens
        
    @torch.no_grad()
    def generate(self, input_ids: torch.Tensor, max_new_tokens: int = 1024):
        generated = input_ids.clone()
        prompt_len = input_ids.shape[1]
        
        while generated.shape[1] - prompt_len < max_new_tokens:
            # Step 1: MTP module generates draft tokens
            draft_tokens = self._draft(generated)
            
            # Step 2: Main model verifies drafts in parallel
            accepted = self._verify(generated, draft_tokens)
            
            # Step 3: Append accepted tokens
            generated = torch.cat([generated, accepted], dim=-1)
            
        return generated
    
    def _draft(self, context: torch.Tensor) -> torch.Tensor:
        """MTP draft generation"""
        drafts = []
        current = context
        
        for _ in range(self.num_speculative_tokens):
            logits = self.mtp_module(current)
            next_token = self._sample(logits[:, -1, :])
            drafts.append(next_token)
            current = torch.cat([current, next_token], dim=-1)
            
        return torch.cat(drafts, dim=-1)
    
    def _verify(self, context: torch.Tensor, 
                drafts: torch.Tensor) -> torch.Tensor:
        """Parallel verification by main model"""
        full_input = torch.cat([context, drafts], dim=-1)
        main_logits = self.main_model(full_input)
        
        accepted = []
        for i in range(drafts.shape[1]):
            draft_token = drafts[:, i:i+1]
            target_logits = main_logits[:, context.shape[1] + i, :]
            target_probs = F.softmax(target_logits, dim=-1)
            
            draft_prob = target_probs.gather(-1, draft_token)
            accept_prob = torch.min(
                torch.ones_like(draft_prob),
                draft_prob / (draft_prob + 1e-8)
            )
            
            if torch.rand(1) < accept_prob:
                accepted.append(draft_token)
            else:
                # Resample from corrected distribution
                adjusted_probs = F.softmax(target_logits, dim=-1)
                corrected = F.relu(adjusted_probs - draft_prob * adjusted_probs)
                corrected = corrected / corrected.sum(dim=-1, keepdim=True)
                resampled = torch.multinomial(corrected, 1)
                accepted.append(resampled)
                break
                
        return torch.cat(accepted, dim=-1) if accepted else drafts[:, :1]
    
    @staticmethod
    def _sample(logits: torch.Tensor, temperature: float = 0.9):
        probs = F.softmax(logits / temperature, dim=-1)
        return torch.multinomial(probs, 1)

4.3 Deployment Configuration

# Official vLLM deployment (8-GPU tensor parallelism)
docker run --gpus all \
  -p 8000:8000 \
  --ipc=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  vllm/vllm-openai:hy4-preview \
  tencent/Hy4-preview-FP8 \
  --tensor-parallel-size 8 \
  --speculative-config '{"num_speculative_tokens": 3, "method": "mtp"}' \
  --attention-backend FLASHMLA_SPARSE \
  --tool-call-parser hy_v4 \
  --reasoning-parser hy_v4 \
  --enable-auto-tool-choice \
  --port 8000 \
  --served-model-name hy4-preview

# SGLang deployment
docker run --gpus all --ipc=host -p 8000:8000 \
  lmsysorg/sglang:hy4-preview \
  python3 -m sglang.launch_server \
  --model tencent/Hy4-preview-FP8 \
  --tp-size 8 \
  --reasoning-parser auto \
  --tool-call-parser auto \
  --speculative-algorithm NEXTN \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --port 8000 \
  --served-model-name hy4-preview

5. The Recursive Self-Improvement Loop: Models Building Themselves

5.1 From “AI Writing Code” to “AI Optimizing AI”

Hy4 preview’s most forward-looking breakthrough isn’t parameter scale or benchmark scores — it’s the fact that for the first time, the model participated in its own full R&D pipeline. According to Tencent, Hy4 preview contributed to automated optimization of:

1. Training methods: proposing candidate training strategies, running comparative experiments
2. Data strategies: analyzing data distribution, proposing sampling adjustments
3. Evaluation frameworks: designing evaluation protocols, identifying blind spots
4. Low-level operators: analyzing inference bottlenecks, proposing operator fusion plans

Recursive Self-Improvement Loop:

                    ┌───────────────┐
                    │  Propose      │
                    │  Solution     │
                    │  (Hy4 preview)│
                    └───────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │  Run          │
                    │  Experiment   │
                    │  (Codex Ses.) │
                    └───────┬───────┘
                            │
                            ▼
                    ┌───────────────┐
                    │  Read Results │
                    │  Analyze Logs │
                    │  Compare Eval │
                    └───────┬───────┘
                            │
                    ┌───────┴───────┐
                    │               │
                    ▼               ▼
            ┌─────────────┐  ┌─────────────┐
            │  Solution OK │  │  Solution   │
            │  → Next Gen  │  │  Failed     │
            └──────┬──────┘  │  → Analyze   │
                   │         └──────┬──────┘
                   └───────┬───────┘
                           │
                           ▼
                    ┌───────────────┐
                    │  Code/Logs/   │
                    │  Feedback     │
                    │  → Next Round │
                    └───────────────┘

5.2 Inference System Self-Optimization: 31.8% Throughput Gain

The most concrete result is in autonomous inference infrastructure optimization. Hy4 preview analyzed its own inference system bottlenecks, conducted multiple rounds of operator fusion and communication optimization, and achieved a 31.8% end-to-end throughput improvement over baseline — consistently across different context lengths (4K-256K) and concurrency levels (1-64):

class InferenceOptimizer:
    """Abstract representation of Hy4 preview's autonomous inference optimization"""
    
    def analyze_bottleneck(self, profile_data: dict) -> str:
        op_times = profile_data['op_times']
        comm_overhead = profile_data['communication_latency']
        memory_bandwidth = profile_data['memory_bandwidth_util']
        
        if op_times['attention'] > 0.4 * sum(op_times.values()):
            return 'attention_is_bottleneck'
        elif comm_overhead > 0.3 * profile_data['total_time']:
            return 'communication_is_bottleneck'
        elif memory_bandwidth < 0.5:
            return 'memory_bound'
        else:
            return 'compute_bound'
    
    def propose_optimization(self, bottleneck: str) -> dict:
        optimizations = {
            'attention_is_bottleneck': {
                'action': 'fuse_attention_ops',
                'target': 'Fuse QKV projection + attention + output projection',
                'expected_gain': '0.15-0.25'
            },
            'communication_is_bottleneck': {
                'action': 'optimize_allreduce',
                'target': 'Coalesce AllReduce + hide latency with async comm',
                'expected_gain': '0.10-0.20'
            },
            'memory_bound': {
                'action': 'kv_cache_compression',
                'target': 'Low-precision quantization + sparse KV cache',
                'expected_gain': '0.20-0.35'
            },
            'compute_bound': {
                'action': 'tensor_parallel_resharding',
                'target': 'Adjust tensor parallelism to reduce compute skew',
                'expected_gain': '0.05-0.15'
            }
        }
        return optimizations.get(bottleneck, {})
    
    def run_experiment(self, optimization: dict) -> dict:
        return {
            'throughput_gain': 0.318,
            'latency_reduction': 0.24,
            'memory_saved_gb': 12.5,
            'stability': 'consistent_across_context_lengths'
        }

5.3 Scientific Discovery with Self-Evolution

Hy4 preview’s scientific research capabilities are equally impressive:

• Molecular Dynamics: 32,512-atom lipid bilayer SO3LR simulation achieved 2.0× speedup over an already-optimized JAX implementation (54.9 ms/step), with a single high-end GPU accommodating 300,000 atoms
• Quantum Transport Device Design: autonomously built a quantum scattering solver, optimized a five-barrier structure, reducing average leakage rate from 48.2% to 4.8%
• 3D Blaschke–Lebesgue Problem (a century-old geometry problem): pushed the volume lower bound from 0.380799 to 0.41104, leaving only ~2% gap to the Meissner tetrahedron conjecture’s upper bound of 0.41986

6. Productivity Benchmarks: 4 Scenarios Verified

6.1 Software Engineering

DeepSWE jumping from 28.0 to 64.3 (more than doubling) represents a qualitative leap in long-codebase engineering capability.

6.2 Office Analytics

Hy4 preview can process 72 documents simultaneously to determine invoice compliance, extract active clauses from regulatory documents, and complete the full workflow from information processing to document, spreadsheet, and presentation delivery. OfficeQA Pro score: 66.2.

6.3 Game Development

Single-prompt game prototype generation: a natural language description produces a playable, interactive game prototype. The model integrates with Unreal Engine 5 via MCP protocol, enabling multi-turn iterative refinement.

6.4 Scientific Research

Beyond the results mentioned above, Hy4 preview scores 92.3 on GPQA Diamond and 55.4 (with tools) on Humanity’s Last Exam — both frontier-level among open-source models.

7. Open-Source Ecosystem and Deployment

7.1 Apache 2.0: Business Impact

Apache 2.0 is one of the most permissive open-source licenses. Key implications:

• Enterprises can build commercial products on Hy4 preview without source code disclosure
• No geographic or domain restrictions (unlike GLM-5.3’s custom license)
• Finance, government, and compliance-sensitive sectors can deploy with confidence

7.2 Domestic Chip 0-Day Adaptation

Hy4 preview achieved 0-day adaptation for Ascend and Enflame chips — a first for open-source frontier models:

Hy4 preview Domestic Chip Adaptation Matrix:

                    ┌─────────────────────────┐
                    │    Hy4 preview          │
                    │    (770B MoE)           │
                    └──────┬──────────┬───────┘
                           │          │
              ┌────────────┘          └────────────┐
              ▼                                     ▼
    ┌───────────────────┐               ┌───────────────────┐
    │   Ascend (华为)    │               │   Enflame (燧原)   │
    │   CANN adaptation  │               │   GCU adaptation  │
    │   vLLM-Ascend      │               │   Custom ops      │
    │   Docker:           │               │   Optimized stack │
    │   quay.io/ascend/   │               │                   │
    │   vllm-ascend:hy4  │               │                   │
    └───────────────────┘               └───────────────────┘

7.3 Hardware Requirements

8. The Chinese LLM Blitz: Competitive Landscape

8.1 Iteration Cadence: From “Annual” to “Monthly”

August 2026 saw unprecedented density in Chinese LLM releases:

August 2026 Chinese LLM Release Calendar:

Mid-August:  DeepSeek V4 Pro (official release)
Mid-August:  Qwen3.8-Flash
August 28:   Hy4 preview (open-source)
August 28:   GLM-5.3-Flash (open-source)
August 28:   Qwen3.8-Flash (same day)

Since rebuilding its pre-training and RL infrastructure in February 2026, Hunyuan has averaged one major version every two months. From Hy3 (295B/21B/256K) to Hy4 preview (770B/49B/1M): just under 2 months, 2.6× parameter scale, 4× context length.

8.2 Multi-Model Comparison

August 2026 Open-Source Flagship Models:

Model              Total Params  Active  Context  License    Input Price (/M)
─────────────────────────────────────────────────────────────────────────
Hy4 preview        770B          49B     1M       Apache 2.0  ¥6 ($0.83)
GLM-5.3            753B          40B     1M       Custom      ¥10 ($1.40)
GLM-5.3-Flash      320B          18B     1M       MIT         ¥1.10 ($0.15)
Qwen3.8-Flash      125B+51B      6B      262K→1M  Qwen Comm.  ¥1.20 ($0.16)
DeepSeek V4 Pro    1.2T+         ~60B    1M       Custom      ~¥3.20 ($0.44)

Price-Performance Trade-offs:
  ✦ Hy4 preview: Strongest capability tier, mid-range price, Apache 2.0
  ✦ GLM-5.3-Flash: Lightweight, MIT license, great value
  ✦ Qwen3.8-Flash: Extreme cost efficiency for batch inference
  ✦ DeepSeek V4 Pro: Largest parameters, more restrictive license

8.3 Shifting Competitive Focus

The competitive center of gravity is shifting from “parameter battles” to “productivity metrics.” Hy4 preview’s release strategy — preview-first, product-embedded, expert-blind-tested, fast-iterating — represents a new competitive paradigm. Tencent’s product matrix (WorkBuddy, CodeBuddy, Yuanbao, ima) turns every model release into a real-world feedback collection pipeline.

Led by Chief AI Scientist Yao Shunyu, this strategy has propelled Hunyuan from a follower to the frontier tier in under six months. As Tencent states in its announcement: “Through a preview-first approach, followed by official releases, Hunyuan continuously incorporates real-world feedback into its research and development process, enabling its models to improve by solving real-world problems.”

9. Known Limitations and Future Directions

Tencent explicitly acknowledges Hy4 preview’s known issues:

1. Over-thinking: The model tends to spend excessive time reasoning on complex tasks, increasing latency
2. Over-verifying: The model repeatedly checks its own outputs beyond useful necessity
3. No multimodality: Hy4 preview and Hy3 are text-only models, lacking multimodal capabilities

These limitations are precisely the optimization targets for the official Hy4 release. Tencent confirms the next version of Hy4 will roll out soon. Meanwhile, Hy4 preview’s “high” reasoning effort and “no_think” direct-response modes provide flexible choices for different scenarios — use no_think for speed on simple tasks, high for quality on complex ones.

10. Conclusion

Hy4 preview’s release marks Tencent Hunyuan’s official entry into the open-source LLM frontier. It delivers a generational leap in parameter scale, context length, and reasoning capability — but more importantly, it pioneers the “recursive self-improvement” paradigm where models participate in optimizing their own development. From Hy3 just 53 days ago to Hy4 preview today, the iteration cadence has compressed from “annual” to “monthly” — a testament to engineering excellence and organizational efficiency.

In the Chinese LLM blitz, Hy4 preview builds a differentiated competitive moat through Apache 2.0’s complete openness, 0-day domestic chip adaptation, and deep product ecosystem integration. For the developer community, the open release of 770B weights, 1M context, and native MTP speculative decoding means that beyond Tencent’s product ecosystem, the AI community can build their own productivity tools on top of Hy4 preview.

As Tencent notes, this is merely a “preview.” The true Hy4 series is still coming. But even as a preview, Hy4 preview has proven that the capability ceiling of open-source models is being redefined — and the pace of redefinition is accelerating.

References:

• Tencent Official Announcement: https://www.tencent.com/tencent-releases-and-open-sources-tencent-hy4-preview/
• Tencent Cloud Developer: https://cloud.tencent.cn/developer/article/2733836
• Science & Technology Daily: http://www.stdaily.com/web/gdxw/2026-08/28/content_571433.html
• Hugging Face Model Card: https://huggingface.co/tencent/Hy4-preview
• MindStudio Technical Analysis: https://www.mindstudio.ai/blog/tencent-hy4-preview-open-weight-model
• mer.vin Review: https://mer.vin/news/tencents-hy4-preview-edges-out-glm-5-3-and-kimi-k3/
• DataLearnerAI: https://www.datalearner.com/ai-models/pretrained-models/hy4-preview

11. Appendix A: Full Benchmark Results

11.1 Automated Benchmarks

11.2 Blind Human Evaluation

The blind evaluation involved 163 internal Tencent experts across 203 engineering tasks, providing a human-centric measure of real-world productivity — arguably more meaningful than automated benchmarks for assessing actual task completion quality.

12. Appendix B: Deployment and Fine-Tuning Ecosystem

12.1 Supported Inference Backends

Hy4 preview is supported by both major open-source inference engines:

vLLM:

• Official Docker image: vllm/vllm-openai:hy4-preview
• Custom attention backend: FLASHMLA_SPARSE
• MTP speculative decoding: built-in --speculative-config
• Tool-calling parser: hy_v4
• Reasoning parser: hy_v4

SGLang:

• Official Docker image: lmsysorg/sglang:hy4-preview
• Multi-architecture support (x86 + Arm)
• Speculative algorithm: NEXTN
• Tool-calling: auto-detect

12.2 Fine-Tuning Stack

Tencent provides a complete fine-tuning pipeline:

# DeepSpeed fine-tuning (ZeRO-3)
deepspeed --num_gpus=8 train.py \
  --model tencent/Hy4-preview \
  --deepspeed ds_config.json \
  --per_device_train_batch_size 1 \
  --gradient_accumulation_steps 8 \
  --learning_rate 1e-5 \
  --num_train_epochs 3

# LLaMA-Factory integration
llamafactory-cli train \
  --model tencent/Hy4-preview \
  --template hy_v4 \
  --dataset my_dataset \
  --finetuning_type lora \
  --lora_rank 64 \
  --output_dir ./hy4-lora

# ms-swift quantization & fine-tuning
swift sft \
  --model_id_or_path tencent/Hy4-preview-FP8 \
  --dataset my_custom_data.jsonl \
  --sft_type lora \
  --output_dir ./hy4-swift-ft

12.3 Quantization Options

Tencent’s AngelSlim toolkit provides model compression:

# FP8 quantization (recommended for production)
python -m angelslim.quantize \
  --model tencent/Hy4-preview \
  --output Hy4-preview-FP8 \
  --dtype fp8 \
  --calibration_dataset c4

# INT4 quantization (experimental)
python -m angelslim.quantize \
  --model tencent/Hy4-preview \
  --output Hy4-preview-INT4 \
  --dtype int4 \
  --calibration_dataset pile

13. Appendix C: Architectural Comparison with Competing Models

13.1 Architecture Parameter Comparison

13.2 Inference Cost Comparison

Cost per million tokens (USD):

Model              Input    Output   Cache Hit
───────────────────────────────────────────────
Hy4 preview        $0.834   $2.501   $0.042
GLM-5.3            $1.40    $4.40    —
GLM-5.3-Flash      $0.15    $0.50    —
Qwen3.8-Flash      $0.16    $0.47    —
DeepSeek V4 Pro    ~$0.44   ~$1.32   —

Hy4 preview's pricing is positioned between the ultra-low-cost Flash models
and the premium flagship tier, reflecting its 49B active parameter count.

14. Appendix D: Code Examples for Common Use Cases

14.1 OpenAI-Compatible API Usage

from openai import OpenAI

# Local deployment (vLLM/SGLang)
client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed"
)

# Cloud API (Tencent Cloud TokenHub)
# client = OpenAI(
#     base_url="https://api.tokenhub.tencent.com/v1",
#     api_key="your-tokenhub-key"
# )

# Standard chat completion
response = client.chat.completions.create(
    model="hy4-preview",
    messages=[
        {"role": "system", "content": "You are a senior software engineer."},
        {"role": "user", "content": "Refactor this Python function to be async:\n\n"
         "def fetch_data(urls):\n"
         "    results = []\n"
         "    for url in urls:\n"
         "        data = requests.get(url).json()\n"
         "        results.append(data)\n"
         "    return results"}
    ],
    temperature=0.9,
    top_p=1.0,
    max_tokens=4096,
)
print(response.choices[0].message.content)

14.2 Tool Calling with Hy4 preview

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed"
)

# Define tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "search_codebase",
            "description": "Search the codebase for relevant files",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {
                        "type": "string",
                        "description": "Search query"
                    },
                    "max_results": {
                        "type": "integer",
                        "description": "Max results to return",
                        "default": 10
                    }
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "run_tests",
            "description": "Run unit tests and return results",
            "parameters": {
                "type": "object",
                "properties": {
                    "test_path": {
                        "type": "string",
                        "description": "Path to test file or directory"
                    }
                },
                "required": ["test_path"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="hy4-preview",
    messages=[
        {"role": "system", "content": "You are a code review agent."},
        {"role": "user", "content": "Find all files that use the old API client "
         "and run their tests to verify compatibility."}
    ],
    tools=tools,
    tool_choice="auto",
)
print(response.choices[0].message)

14.3 High vs No-Think Mode

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

# High reasoning effort (default) — for complex problems
response_high = client.chat.completions.create(
    model="hy4-preview",
    messages=[
        {"role": "user", "content": "Prove that there are infinitely many prime numbers."}
    ],
    extra_body={"reasoning_effort": "high"}  # Default
)

# No-think mode — for direct, fast responses
response_fast = client.chat.completions.create(
    model="hy4-preview",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
    extra_body={"reasoning_effort": "no_think"}
)

15. Appendix E: The Generational Gap — Hy3 to Hy4

The most telling comparison is not against competitors, but against Hy4’s own predecessor:

Hy3 → Hy4 preview: Generational Comparison

Metric              Hy3 (Apr 2026)  Hy4 preview (Aug 2026)  Change
────────────────────────────────────────────────────────────────────
Total params        295B            770B                     2.6×
Active params       21B             49B                      2.3×
Context window      256K            1M                       4.0×
Routed experts      192             256                      +64
Shared expert       No              Yes                      New
MTP layer           No              Yes (10B/0.7B)           New
Residual streams    1               4 (iHC)                  New
Attention           Standard        Gated DSA + IndexCache   New
DeepSWE             28.0            64.3                     2.3×
Terminal Bench      70.8            85.4                     +14.6
Time between        53 days         53 days                  Same cadence
releases            (Hy3 preview)   (Hy4 preview)

The 53-day gap between Hy3 preview and Hy4 preview demonstrates
that Tencent has achieved a sustainable "monthly major release"
cadence — a pace previously only associated with the fastest
frontier labs.

16. Conclusion: The Road Ahead

Hy4 preview represents a pivotal moment in the open-source LLM landscape. It is not merely a larger model — it is a fundamentally different approach to how models are built, deployed, and improved. The combination of:

1. Massive but efficient MoE architecture (770B/49B, 256 experts + shared expert)
2. Long-context viability (1M tokens via Gated DSA + IndexCache)
3. Native speculative decoding (built-in MTP layer)
4. Recursive self-improvement loop (model optimizing its own development)
5. Aggressive open-source strategy (Apache 2.0, 0-day chip adaptation)
6. Product-ecosystem integration (WorkBuddy, CodeBuddy, Yuanbao, ima)

…creates a flywheel that is unique in the current landscape. Each release feeds real-world feedback into the next iteration; each model improvement is immediately deployed to millions of users through Tencent’s product matrix; and increasingly, the model itself is contributing to its own improvement cycle.

The “preview” label is a promise, not a disclaimer. If the Hy3 → Hy4 preview trajectory is any indication, the Hy4 official release will be substantially more capable — and it will arrive sooner than most expect.

For developers and enterprises, the message is clear: the open-source LLM frontier is no longer dominated by a single player. Hy4 preview has joined the ranks of DeepSeek, GLM, Qwen, and Kimi at the cutting edge — and the Apache 2.0 license means that for the first time, the most permissively licensed model is also one of the most capable.

The Chinese LLM blitz is not just about quantity of releases — it’s about a new paradigm of rapid, feedback-driven, product-integrated model development. Hy4 preview is the latest and most compelling evidence that this paradigm works.

This article was written on August 30, 2026. All benchmark data is based on Tencent’s official announcement and model card. Independent third-party verification is pending as of publication date.

17. Appendix F: The Recursive Self-Improvement Loop in Detail

17.1 The Optimization Pipeline

The recursive self-improvement loop implemented by Hy4 preview can be broken down into five distinct stages:

Stage 1: Bottleneck Identification
  ┌─────────────────────────────────────────────┐
  │  Profile inference execution → identify     │
  │  hot spots (attention, communication,       │
  │  memory bandwidth, compute utilization)      │
  └─────────────────────────────────────────────┘
                        │
                        ▼
Stage 2: Solution Proposal
  ┌─────────────────────────────────────────────┐
  │  Generate candidate optimization strategies │
  │  based on the identified bottleneck type    │
  │  (operator fusion, kernel rewrite,          │
  │   communication schedule, precision tuning) │
  └─────────────────────────────────────────────┘
                        │
                        ▼
Stage 3: Experiment Execution
  ┌─────────────────────────────────────────────┐
  │  Implement the proposed change in code      │
  │  Run benchmarks across multiple configs     │
  │  (varying context lengths, batch sizes,     │
  │   concurrency levels)                       │
  └─────────────────────────────────────────────┘
                        │
                        ▼
Stage 4: Result Analysis
  ┌─────────────────────────────────────────────┐
  │  Compare throughput, latency, memory usage  │
  │  against baseline. Determine if the change  │
  │  is a net positive across all conditions.   │
  └─────────────────────────────────────────────┘
                        │
                        ▼
Stage 5: Feedback Integration
  ┌─────────────────────────────────────────────┐
  │  Successful: merge into production stack    │
  │  Failed: analyze why, refine hypothesis     │
  │  Both paths feed back into Stage 1          │
  └─────────────────────────────────────────────┘

17.2 Key Results from the Self-Optimization Loop

17.3 Implications for Future Model Development

The recursive self-improvement loop represents a paradigm shift in how frontier models are developed. Traditionally, model optimization has been a purely human-driven process: researchers identify problems, write code, run experiments, and iterate. Hy4 preview demonstrates that at sufficient capability levels, the model itself can participate in this cycle — not replacing human researchers, but dramatically accelerating the iteration speed.

The key insight is that Hy4 preview’s self-improvement capability is not a separate “meta-learning” module — it emerges naturally from the model’s general reasoning and code generation abilities. When given the right tools (profiling data, code execution environment, experiment orchestration framework), the model can apply the same reasoning capabilities it uses for software engineering tasks to optimize its own infrastructure.

This has profound implications for the scaling laws of AI development: as models become more capable, they can increasingly contribute to their own improvement, potentially creating a compounding effect where each generation of models accelerates the development of its successor.

17.4 Comparison with Other Self-Improvement Approaches

Hy4 preview’s approach is unique in that it targets the entire inference stack — from low-level GPU kernels to high-level communication orchestration — rather than focusing solely on model weights or output quality. This holistic optimization scope is what makes the 31.8% throughput improvement particularly impressive: it’s not a single optimization but a coordinated set of improvements across the entire serving stack.

• ←Previous
• Next→