DeepSeek V4 Explained: 1T-Param MoE, Engram Memory, 1M Context

What DeepSeek V4 Actually Is

DeepSeek V4 shipped in March 2026 as a 1-trillion-parameter mixture-of-experts (MoE) model with roughly 32-37 billion parameters active per forward pass, a 1-million-token context window powered by a learned memory layer the team calls Engram, native multimodal generation, and an MIT license. The active-parameter count is what determines inference cost; the total parameter count is what determines VRAM. That distinction is the entire story for anyone trying to deploy V4 outside a hyperscaler — you need 320 GB of weights resident in memory at Q4_K_M to serve a model that's only doing 32B-parameter compute per token.

The other practical claim — that V4 holds coherent reasoning across 1M tokens — is the most-tested marketing claim in recent LLM history. The short answer: it works, but Engram is the reason it works, and Engram is the most interesting thing about the model.

Last updated: April 2026 — verified parameter counts from the technical report, benchmark scores against the LM Council April leaderboard, and API pricing from DeepSeek's platform page.

Engram: The Learned Memory Layer

Engram is not a positional-encoding hack. RoPE-extension tricks (YaRN, NTK-aware scaling, sliding window attention) get you longer effective context but degrade quality on long-range dependencies — the model "knows" about earlier tokens but doesn't reliably integrate them. Engram is a learned compression module: as the context grows past a threshold (~64K tokens in V4's case), the model writes summarized representations of older context into a compressed memory bank, then attends over the bank when newer tokens need older information.

The honest engineering view: it's similar in spirit to Compressive Transformers (DeepMind, 2019) and to retrieval-augmented memory in RWKV-Eagle, but the compression is learned end-to-end with the rest of the model rather than bolted on. The result is that on the genuinely-hard long-context evals (Needle in a Haystack at 800K, RULER, LongBench-v2) V4 doesn't degrade in the way YaRN-extended Llama 3 did at the same context lengths. It does cost compute — Engram adds roughly 8% to inference time at long contexts — but it preserves retrieval accuracy.

Definition: Engram = a learned compression layer that summarizes older context into a memory bank as the conversation grows. The model attends over both the raw recent tokens and the compressed memory, similar to how human episodic memory works in the loose analogy DeepSeek's paper draws.

Active vs Total Parameters: The VRAM Math

V4's MoE has 256 experts; each forward pass activates 8 experts (plus a shared expert), giving ~32-37B active parameters depending on routing. Why this matters in practice:

• VRAM is determined by total parameters. To serve V4 at full precision (FP16) you need roughly 2 TB of weights resident, plus KV cache. At Q4_K_M (4-bit) that drops to ~320 GB. At Q3 (aggressive) ~240 GB but quality starts hurting.
• Inference compute is determined by active parameters. Per-token speed is roughly equivalent to a dense 32B model — fast for the size.
• Practical hardware: 4× H100 80 GB ≈ 320 GB VRAM, runs Q4 V4 with reasonable headroom. 8× RTX 5090 (32 GB each) is the cheapest "actually plausible" home setup but not really practical at 256 GB without quality loss. See Qwen 3.5 VRAM requirements for the analogous math on a smaller MoE.

The MoE math is also why DeepSeek's API price is competitive — each token only pays compute on 32B parameters, so per-token inference cost is 30-40x cheaper than a dense 1T model would be. That cost advantage shows up directly in the API pricing.

Benchmark Scores: What V4 Actually Wins

Benchmark	DeepSeek V4	Claude Opus 4.7	GPT-5.4	Gemini 3.1 Pro
SWE-Bench Pro	72.4%	78.1%	74.2%	71.8%
LiveCodeBench (Mar 2026)	81.2%	83.9%	80.1%	79.4%
GPQA Diamond	89.7%	92.3%	91.6%	94.3%
AIME 2025	94.1%	92.8%	93.4%	93.9%
MMLU-Pro	87.6%	88.4%	89.1%	88.7%
RULER (1M ctx)	91.3%	n/a (200K max)	n/a (256K max)	87.4%
Aider polyglot	76.8%	83.2%	78.9%	74.1%

Three patterns from the table. First, V4 is competitive but not first-place on any frontier benchmark — it trades hundredths of a point with the closed models on most metrics. Second, it wins decisively on AIME math and on RULER long-context, which is the Engram payoff. Third, it loses meaningfully on Aider polyglot — the practical "agentic coding" benchmark — which suggests Anthropic's harness tuning still matters more than raw model capability. For real-world deployment context see Qwen vs DeepSeek vs GLM.

API Pricing and Real Costs

DeepSeek's hosted API in April 2026 charges roughly $0.27 per million input tokens and $1.10 per million output tokens — about 10x cheaper than Claude Opus 4.7 on input and 7x cheaper on output. With aggressive prompt caching, input drops to ~$0.05/M for cached prefixes. This matters for high-volume workloads:

• Long-context RAG: V4's 1M context plus low input cost makes "stuff the whole codebase in" a real option for code-aware tasks.
• Agentic loops: Per-step cost is low enough that long multi-turn agent runs cost cents, not dollars.
• Batch / asynchronous workloads: CI fixers, mass refactor jobs, doc generation across a large codebase. V4 + cheap input wins here over Claude on cost.

For full provider economics see LLM API pricing; for which agent harness to pair with V4 if cost matters, see AI coding assistants compared.

When V4 Is the Right Call vs When It Isn't

Honest framing: V4 is a high-quality model, but it's not Opus 4.7. The decision is about which axis matters.

• Pick V4 when: you need 1M-token context for actual long-context work (large codebase analysis, multi-doc reasoning), cost-per-token matters at volume (batch jobs, high-throughput agents), you want open weights for compliance / customization / fine-tuning, or you specifically need MIT license.
• Pick Opus 4.7 when: subjective code quality matters most (senior-engineer review preference still favors Opus), you need the best agentic-tool-use behavior (Anthropic's harness tuning), or you don't care about per-token cost.
• Pick Gemini 3.1 Pro when: graduate-level reasoning is the bottleneck (GPQA Diamond and ARC-AGI advantage), or you need multimodal-heavy long context with images.
• Self-host V4 when: compliance forces it, you have the hardware budget (4× H100 minimum), or you want to fine-tune for domain adaptation. See self-hosted LLM TCO for the amortization math.

The Politics: Worth Mentioning Honestly

DeepSeek is a Chinese AI lab. That introduces real procurement considerations for some teams: US export-control rules, data-residency questions when using the Chinese-hosted API (DeepSeek's platform routes through PRC infrastructure), and ongoing debates about training-data provenance and content moderation behavior.

What's true: the open weights are genuinely open and MIT-licensed, so self-hosting eliminates the data-residency concern entirely. What's also true: the model has been observed to refuse or deflect on a small list of politically sensitive topics — a pattern documented in academic evaluation work. For most engineering use cases (code, infrastructure, reasoning) this never comes up; for use cases involving content generation or analysis on geopolitical topics, test before committing.

Several Western infrastructure providers (Together AI, Fireworks, OpenRouter) host V4 weights on US/EU infrastructure if PRC routing is a non-starter for compliance.

How V4 Compares to Other 2026 Releases

The April 2026 LLM landscape is crowded. Where V4 fits relative to recent peers:

• vs GLM-5.1: GLM-5.1 wins on coding (SWE-Bench), V4 wins on long context and AIME math. GLM-5.1 has partial open weights; V4 has full open weights.
• vs Kimi K2.6: Kimi K2.6 is cheaper (~$0.10/M input) and faster but lacks the long-context Engram architecture.
• vs MiniMax M2.7: M2.7 markets "self-evolving" agentic features; V4 is a more conventional but more capable base model.
• vs Qwen 3.5 Series: Qwen 3.5 32B / 72B run on more accessible hardware; V4 needs 4× H100 minimum at any reasonable speed.

Frequently Asked Questions

Bottom Line

DeepSeek V4 is the most interesting open-weight LLM release of early 2026 not because it tops every benchmark — it doesn't — but because it ships a real architectural innovation (Engram) that solves long-context degradation in a way previous extensions didn't, and because the MIT license plus competitive cost make it a serious deployment option for compliance-sensitive teams or anyone running batch workloads at scale. For frontier-tier subjective code quality the closed models still win; for everything else V4 is on or near the frontier at a fraction of the cost.