Llama 4 Explained — Meta's Open AI Models

Photo: Solen Feyissa / Pexels
Llama 4 is the fourth-generation open-weights large language model family from Meta, released on April 5, 2025. It was the first Llama generation built to be natively multimodal — image and text inputs woven into the architecture from the first day of training, not bolted on afterward — and the first to use Mixture-of-Experts (MoE) routing, a technique that lets a model carry hundreds of billions of parameters while only "waking up" a small fraction of them for any given word. More than a year later, Llama 4 also marks the end of an era: it is the last flagship Llama Meta has shipped as open weights.
That last point matters for anyone deciding whether to build on Llama 4 today. This article covers what the Scout, Maverick, and Behemoth models actually are, how the underlying architecture works, why the release sparked a benchmark scandal within days, and — updated for 2026 — what happened after Meta quietly walked away from the open-weights strategy that made Llama famous in the first place.
What Llama 4 actually is
Think of a large language model release as a product line, the way a car company reveals a compact, a sedan, and a flagship at the same auto show. Meta did the same thing with Llama 4: three models sharing one architecture, tuned for three different jobs. Two of them shipped. The third never did.
The release landed at an awkward moment for Meta. Two months earlier, the Chinese lab DeepSeek had stunned the industry with DeepSeek R1, an open-weights reasoning model that matched OpenAI's o1 on math and coding benchmarks at a fraction of the reported training cost. Llama 4 launched into that shadow: technically ambitious on paper, but almost immediately dogged by accusations of benchmark gaming, and quickly overshadowed by the reasoning-model wave that R1 had kicked off (Meta AI, "The Llama 4 herd," April 5, 2025).
Meet the family: Scout, Maverick, and Behemoth
Meta announced three variants at launch, all sharing the same core transformer-plus-MoE design but scaled to very different sizes.
Llama 4 Scout has 109 billion total parameters but only 17 billion "active" per token, spread across 16 experts, and shipped with a 10-million-token context window — a record for any publicly released model at the time. Llama 4 Maverick has 400 billion total parameters, the same 17 billion active per token, but 128 experts and a 1-million-token context window; Meta positioned it as competitive with GPT-4o on general capability. Llama 4 Behemoth was previewed at roughly 2 trillion total parameters with 288 billion active, and was described as a teacher model used to distill Scout and Maverick during training — not a model Meta ever intended to ship directly to developers (Hugging Face, "Welcome Llama 4 Maverick & Scout," April 2025).
The detail worth sitting with is that Scout and Maverick have identical active-parameter counts (17 billion) despite a nearly four-fold difference in total size. That means the two models cost roughly the same to run per token, but Maverick can draw on far more specialized "experts" during that computation. It's the same logic as a hospital: a small clinic (Scout) and a large teaching hospital (Maverick) might both route a patient to one specialist at a time, but the teaching hospital has 128 specialists on staff instead of 16, so it's more likely to have exactly the right one for an unusual case.
The most surprising fact in this story: Llama 4 Behemoth, the 2-trillion-parameter model Meta announced with the most fanfare, was never released to the public. Meta pushed its debut from "early summer" to fall 2025 and later indefinitely, after engineers reportedly found it showed only marginal gains over smaller Llama 4 variants relative to its training cost (SiliconANGLE, "Meta to postpone release of Llama 4 Behemoth model," May 15, 2025).
What "natively multimodal" means and why it matters
Earlier Llama generations handled text only. Llama 3.2 added image understanding through a vision adapter — a separate module trained after the fact and stitched onto the language model, the way you might bolt a rearview camera onto a car that wasn't designed with one. Llama 4 took a different approach called early fusion: image patches and text tokens are mixed into the same input sequence and processed by the very same transformer layers, with both modalities present from the start of pre-training rather than added later.
That distinction sounds academic, but it changes what the model can actually do. A late-fusion model like Llama 3.2 essentially translates an image into a text-like description internally and then reasons over that description — useful, but lossy. An early-fusion model like Llama 4 reasons over raw visual and textual information together, the same way GPT-4o, Gemini 2.5 Pro, and Claude's vision models do. In practice this shows up as three concrete improvements:
- Better cross-modal reasoning — answering a question about a specific region of a diagram, rather than just describing the diagram in general terms.
- The ability to interleave images and text arbitrarily within a single long context, instead of treating images as a fixed "attachment" processed once at the start.
- No separate "vision mode" to switch into — the model handles mixed input the same way it handles pure text.
One limit worth flagging clearly: audio and video are not native inputs in Llama 4. Only image and text were part of the early-fusion training. That's a meaningful gap compared to some competing multimodal systems, and it's one of the capabilities Meta later built into its closed-source successor (more on that below).
How Mixture-of-Experts keeps a 400-billion-parameter model affordable
Mixture-of-Experts is the single most important architectural idea in Llama 4, and it's worth explaining plainly because most coverage of the release assumes readers already know what it means. In a standard ("dense") transformer, every parameter in the model does some work on every single token you feed it. If the model has 400 billion parameters, processing one word activates all 400 billion of them.
MoE breaks that rule. Instead of one giant block of parameters, the model is divided into many smaller sub-networks called experts — 16 of them in Scout, 128 in Maverick. A small routing network looks at each token and decides which one or two experts are best suited to handle it, then sends the token only to those experts. The rest of the experts sit idle for that token. It's the same principle as a large hospital: the hospital as a whole employs hundreds of specialists, but any single patient only sees the one or two doctors relevant to their case, not the entire staff.
That's why Maverick can carry 400 billion total parameters — more raw knowledge and specialization capacity — while only spending the compute of a 17-billion-parameter model on any given token. The tradeoff is that the model needs enough memory to hold all 400 billion parameters even though it only uses a fraction of them at once, which is why Maverick still needs multiple high-end GPU servers to run, despite its per-token compute cost being modest. For a deeper look at how the routing mechanism itself works, see Mixture of Experts explained.
Training scale and the 10-million-token context trick
Meta trained Llama 4 on roughly 30 trillion tokens, about double the 15 trillion tokens used for Llama 3.1, spanning around 200 languages with substantial depth in 12 of them. Training ran across tens of thousands of Nvidia H100 GPUs, and the process followed three stages: broad pre-training, distillation from the unreleased Behemoth model (using Behemoth's outputs as a training signal for the smaller models), and a lighter post-training phase combining supervised fine-tuning with a targeted reinforcement-learning step (Meta AI, "The Llama 4 herd," April 5, 2025).
Scout's headline feature is its 10-million-token context window — roughly 5,000 pages of continuous text, or enough to hold an entire mid-sized codebase, hours of transcribed audio converted to text, or a large batch of documents in a single pass. Standard transformer attention doesn't extend cleanly to windows that long; positional information degrades and quality collapses well before you reach millions of tokens. Meta's workaround is a technique called iRoPE (interleaved rotary positional embeddings), paired with a temperature-scaled attention adjustment that keeps the model's sense of "where am I in this sequence" stable even at extreme lengths.
It's worth being honest about what that context window does and doesn't guarantee. "Needle in a haystack" tests — where a single fact is buried somewhere in a huge document and the model is asked to retrieve it — look strong across most of the 10-million-token range. But retrieval is the easy case. Multi-hop reasoning, where the model has to connect several scattered pieces of information across that same enormous context, is a harder problem, and independent evaluation of Llama 4's real-world performance at the far end of its context window has been thinner than the marketing number suggests.
Fast facts
| Model | Total parameters | Active parameters | Experts | Context window |
|---|---|---|---|---|
| Llama 4 Scout | 109B | 17B | 16 | 10M tokens |
| Llama 4 Maverick | 400B | 17B | 128 | 1M tokens |
| Llama 4 Behemoth (unreleased) | ~2T | 288B | 16 | Not disclosed publicly |
Source: Meta AI, "The Llama 4 herd," April 2025; Hugging Face model release notes, April 2025.
The license: what "open" really means with Llama 4
Llama 4 ships as open weights, meaning anyone can download the full parameter files and run the model themselves, but it is not released under a conventional open-source license. Meta uses its own "Llama 4 Community License," which comes with a handful of real restrictions developers should know before building a product on top of it.
Companies with more than 700 million monthly active users must request a separate commercial license from Meta before using Llama 4 — a threshold clearly aimed at a small list of the largest technology companies rather than typical startups. Products built primarily around the model must include a "Built with Llama" attribution. And, notably, the license also restricts distribution and use within the European Union, a carve-out Meta has attributed to regulatory uncertainty under the EU AI Act. Because of these terms, the Open Source Initiative does not recognize the Llama 4 license as meeting the formal definition of open source, even though the weights themselves are freely downloadable (Hugging Face, "Welcome Llama 4 Maverick & Scout," April 2025).
For the overwhelming majority of individual developers, researchers, and small-to-midsize companies outside the EU, none of these restrictions bite in practice. Llama 4 is effectively free to use, fine-tune, and deploy commercially. The restrictions exist to manage a small set of edge cases: the largest platform companies, and one entire regulatory jurisdiction.
Benchmark performance and the Arena controversy
Meta positioned Maverick as a direct competitor to GPT-4o and Google's Gemini 2.0 Flash, and pitched Scout as the best-in-class option for anyone who needs to process enormous documents in one pass rather than chunking them.
Fast facts
| Benchmark | Llama 4 Maverick | Llama 4 Scout | GPT-4o | Gemini 2.0 Flash | DeepSeek R1 |
|---|---|---|---|---|---|
| MMLU Pro | 80.5% | 74.3% | 74.7% | 77.6% | 84.0% |
| GPQA Diamond | 69.8% | 57.2% | 53.6% | 62.1% | 71.5% |
| LiveCodeBench | 43.4% | 32.8% | 32.3% | 34.5% | 65.9% |
| MMMU (vision) | 73.4% | 69.4% | 69.1% | 71.7% | n/a (text only) |
| Context window | 1M tokens | 10M tokens | 128K tokens | 1M tokens | 128K tokens |
| Active parameters | 17B | 17B | Undisclosed | Undisclosed | 37B |
Source: Meta AI, "The Llama 4 herd," April 2025.
The gap on LiveCodeBench and math-heavy tests versus DeepSeek R1 is not a fluke or a training shortfall — it's architectural. Llama 4 is not a reasoning model. It doesn't perform extended chain-of-thought before answering the way R1, OpenAI's o-series, or Gemini's thinking modes do. It answers in essentially one pass, which makes it fast and cheap to run but structurally weaker on problems that benefit from working through intermediate steps.
Within days of the launch, independent researchers noticed something odd: the version of "Llama 4 Maverick" scoring near the top of the LMSYS Chatbot Arena leaderboard (a crowdsourced human-preference ranking where people vote blind between two anonymous model outputs) was performing noticeably better there than the publicly downloadable weights performed when tested independently. Meta confirmed the Arena-listed model was an experimental variant specifically tuned to score well on human-preference judging — not the same model available on Hugging Face or through Meta's own APIs.
LMSYS removed the experimental submission and added a disclaimer to its leaderboard explaining what had happened (LMSYS, "Statement on Llama 4 Maverick Arena Submission," April 2025). The episode became one of the highest-profile recent examples of "benchmark gaming" — submitting a model tuned for how a specific evaluation measures quality, rather than the actual model shipped to users — and it cooled the initially strong reception to Llama 4 considerably. It's a useful reminder for anyone evaluating any AI model release: a leaderboard score only tells you about the exact weights that produced it, and those aren't always the weights you'll actually be able to download.
Is Llama 4 a reasoning model?
No. Llama 4 answers in a single forward pass without extended chain-of-thought, unlike DeepSeek R1 or OpenAI's o-series models. That architectural choice is the main reason it trails specialized reasoning models on math and competitive coding benchmarks despite matching or beating them on general knowledge and vision tasks.
How Llama 4 compares to the rest of the open-weights field
Llama 4 didn't stay the only serious open-weights option for long. Alibaba's Qwen3 family and its later revisions became the model developers most often mention in the same breath as Llama 4 through 2025 and 2026, and the two projects ended up trading strengths rather than one clearly beating the other.
Qwen3's models generally ship under the Apache 2.0 license, the most permissive standard open-source license available — no user-count threshold, no regional carve-out, no attribution requirement. That makes Qwen3 the simpler legal choice for companies that want to avoid reading a custom license agreement at all. Where Llama 4 tends to win is ecosystem depth: because it arrived earlier and from a company with Meta's distribution, it had a head start on tooling, inference-framework support, and the sheer number of platforms that run it out of the box, plus Scout's still-unmatched 10-million-token context window for anyone working with extremely long documents or large codebases in a single pass.
The honest takeaway for a developer choosing between them in 2026 is that neither is a strict upgrade over the other. The decision usually comes down to license comfort, context-length needs, and which ecosystem already has the deployment tooling you're using — not a single benchmark number.
Where Llama 4 goes from here — Meta's pivot away from open weights
This is the part of the Llama 4 story that has changed the most since the model first shipped, and it's the reason this article carries a 2026 update. Behemoth was never formally canceled, but Meta also never released it. Reporting through late 2025 described Behemoth as delivering only marginal improvements over Maverick relative to its enormous training cost, and pointed to a broader internal shake-up: Meta stood up a new division called Meta Superintelligence Labs (MSL), restructured its AI organization into four separate teams, froze hiring, and cut roughly 600 positions from its existing AI group as part of the reorganization (Winbuzzer, "Meta Pivots from Llama to Closed AI Models," December 9, 2025; Built In, "Meta's AI Reorg: What to Know About Meta Superintelligence Labs").
Reporting at the time also linked the strategic reconsideration to DeepSeek's rise. R1's release in January 2025 demonstrated that a competitor could take an openly documented architecture and, with a fraction of the widely assumed training budget, produce a model competitive with the best closed reasoning systems — a result that reportedly unsettled Meta leadership about the wisdom of publishing frontier-model weights for anyone, including well-resourced rivals, to build on freely.
On April 8, 2026, Meta Superintelligence Labs shipped Muse Spark, described in Meta's own announcement as the company's most capable model to date — and, critically, its first closed-weight flagship model (Meta, "Introducing Muse Spark: Scaling Towards Personal Superintelligence," April 8, 2026; Meta, "Introducing Muse Spark," about.fb.com). Muse Spark is natively multimodal like Llama 4, but extends input support to voice alongside text and images, and adds visual chain-of-thought reasoning and multi-agent tool orchestration — capabilities Llama 4 doesn't have. It launched integrated across Facebook, Instagram, WhatsApp, Messenger, and Meta's Ray-Ban smart glasses, but with no published weights and no download link. A developer-facing follow-up, Muse Spark 1.1, arrived on July 9, 2026, opening Meta's first paid, OpenAI-compatible API, priced at roughly $1.25 per million input tokens and $4.25 per million output tokens.
What this means practically: Llama 4 is very likely to remain Meta's last open-weights flagship release for the foreseeable future. It isn't deprecated — the weights, Hugging Face model cards, and existing fine-tunes still work exactly as they did at launch — but Meta has stopped adding to the family, and the company's frontier research effort has moved to a closed model with no public roadmap for another open release. Anyone building long-term infrastructure on Llama 4 today should treat it as a mature, static foundation rather than a model that will keep receiving upstream updates from Meta itself.
Common confusions people have about Llama 4
"Llama 4" is not one model. It's a family with three very different members, and the two that shipped (Scout and Maverick) have identical active-parameter counts despite hugely different total sizes and price-to-run profiles. Asking "how good is Llama 4" without specifying which variant is a bit like asking how fast "a car" is without saying which car.
A big total parameter count doesn't mean a proportionally expensive model to run. Maverick's 400 billion total parameters get a lot of attention, but its 17-billion active-parameter count is what actually determines inference cost per token. The MoE design is specifically there to decouple "how much the model knows" from "how much compute answering a question takes."
Open weights is not the same as open source. You can download and run Llama 4's parameters freely in most circumstances, but the license carries real restrictions — the 700-million-user threshold, the EU carve-out, the attribution requirement — that keep it outside the formal definition the Open Source Initiative uses. Calling it "open source" without qualification is technically incorrect, even though it functions as free-to-use for the vast majority of developers.
The 10-million-token context window is a maximum, not a guarantee of quality. Retrieving a single fact from deep inside that window tends to work well in testing. Reasoning that requires connecting multiple scattered pieces of information across the full window is a much harder problem, and it's less thoroughly proven than the headline number implies.
Behemoth was never a consumer product. Because Meta previewed it alongside Scout and Maverick with detailed specifications, some coverage treated it as an imminent release. It was always described internally as a distillation teacher — a model used to train the smaller models, not one intended for public download — and reporting through late 2025 makes clear it never crossed that line into an actual release.
The takeaway
Llama 4 introduced two real architectural firsts to the Llama family — native multimodal training and Mixture-of-Experts routing — and Scout's 10-million-token context window remains one of the largest of any released model. But the more consequential story, in hindsight, is what came after: a benchmark controversy that dented trust within days of launch, an unreleased flagship that quietly disappeared from Meta's roadmap, and a strategic pivot that turned Meta's next major model into a closed system with no public weights at all. If you're evaluating Llama 4 for a project in 2026, treat it as a capable, well-documented, and stable open-weights option — just not one that's likely to get bigger siblings from Meta any time soon.
Frequently Asked Questions
Q: When was Llama 4 released? A: April 5, 2025. Meta released Scout and Maverick publicly on that date; Behemoth was previewed with specifications but was never made available for public download.
Q: Is Llama 4 a reasoning model like o1 or DeepSeek R1? A: No. Llama 4 answers in a single pass without extended chain-of-thought reasoning. That's the main reason it trails specialized reasoning models on math and coding benchmarks despite matching or beating them on general knowledge and vision tasks.
Q: What is the difference between Scout and Maverick? A: Both have 17 billion active parameters per token. Scout has 109 billion total parameters, 16 experts, and a 10-million-token context window. Maverick has 400 billion total parameters, 128 experts, and a 1-million-token context window. Maverick scores higher on most quality benchmarks; Scout is the long-context specialist.
Q: Can I use Llama 4 commercially? A: Yes, for most companies. The Llama 4 Community License requires a separate agreement with Meta only if your product has more than 700 million monthly active users, and it restricts use and distribution within the European Union. Outside those two cases, commercial use is free with an attribution requirement.
Q: What was the Llama 4 benchmark controversy? A: Meta submitted an experimental variant of Maverick tuned specifically for LMSYS Chatbot Arena's human-preference scoring, rather than the publicly released weights. After researchers noticed the gap between Arena scores and independently tested performance, LMSYS removed the experimental submission and added a disclaimer to its leaderboard.
Q: Does Llama 4 support video or audio input? A: No. Llama 4 is natively multimodal for image and text only. Meta's later closed-weight model, Muse Spark, added voice input, but that capability was never brought to the Llama 4 family.
Q: Whatever happened to Llama 4 Behemoth? A: It was repeatedly delayed through 2025 after reportedly showing only marginal gains over Maverick relative to its training cost, and it was never formally released. Reporting from late 2025 tied its shelving to a broader restructuring of Meta's AI division under the newly formed Meta Superintelligence Labs.
Q: Is Meta still developing open-weights Llama models? A: There is no confirmed successor as of mid-2026. Meta's frontier AI development moved to Meta Superintelligence Labs, which shipped its first flagship model, Muse Spark, as closed weights in April 2026, with a developer API following in July 2026. Llama 4 remains Meta's most recent open-weights flagship release.
Q: How does Llama 4 compare to Qwen3? A: They trade advantages rather than one clearly winning. Qwen3 typically ships under the more permissive Apache 2.0 license with no user-count restriction. Llama 4 has broader existing platform support and, in Scout, the largest context window of any major open-weights release. Most developers choose based on licensing preference and context-length needs rather than a single benchmark score.
Q: Where can I run Llama 4? A: Directly from Meta at llama.com, through gated repositories on Hugging Face (meta-llama/Llama-4-Scout-17B-16E and meta-llama/Llama-4-Maverick-17B-128E), or through cloud providers including AWS Bedrock, Azure AI Foundry, Google Cloud Vertex AI, Together AI, Groq, Fireworks, and Cerebras. Scout fits on a single 8x H100 80GB node in FP8 precision; Maverick typically needs two to four such nodes.
Sources
- Meta AI, "The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation," April 5, 2025
- Hugging Face, "Welcome Llama 4 Maverick & Scout on Hugging Face," April 2025
- SiliconANGLE, "Meta to postpone release of Llama 4 Behemoth model, report claims," May 15, 2025
- Winbuzzer, "Meta Pivots from Llama to Closed AI Models, Abandoning Open Source Roots," December 9, 2025
- Built In, "Meta's AI Reorg: What to Know About Meta Superintelligence Labs"
- Meta, "Introducing Muse Spark: Scaling Towards Personal Superintelligence," April 8, 2026
- Meta, "Introducing Muse Spark," about.fb.com, April 2026
Follow new articles
Add SciHub101 to your feed reader to see new science and technology stories.
About the Author
SciHub101 is an independent science and technology publication founded and edited by Buğra Sözeri, a software engineer and app developer based in Türkiye. AI tools assist with drafting and research. Source documentation in the existing archive is uneven, and articles are being reviewed and corrected. The Editorial Team byline identifies editorial responsibility; it does not certify specialist or independent human review. Where we compare products, anything built by our own team is labelled as such — see the editorial guidelines and About page for how corrections and disclosures work.
Related Articles

What Is Aider? Terminal AI Coding Tool
Aider is an open-source terminal AI pair programmer that works with any LLM. How its repo map, architect mode, and git workflow work, and how it compares.

Gemini 2.5 Pro Explained — Google's Best Model
Gemini 2.5 Pro is Google DeepMind's reasoning model with a 1M-token context and top scores on coding and reasoning benchmarks at launch.

DeepSeek R1 Explained — Open Reasoning Model
DeepSeek R1 is an open-weights reasoning model released January 2025 that matched OpenAI o1 and triggered a $589B Nvidia stock drop.

Open Source AI vs Proprietary AI: The Real Tradeoffs
Open source AI vs proprietary AI compared: what licenses allow, how far the capability gap has narrowed, and when self-hosting actually beats a paid API.

What Is Mistral AI? Europe's LLM Lab, Explained
Mistral AI is the Paris lab racing to give Europe its own frontier LLM. How its models, open-weight strategy, and ASML-backed funding compare to US rivals.

Multimodal AI: How Vision-Language Models See and Understand the World
How multimodal AI turns images, audio, and video into tokens a language model can process, what benchmarks show, and where vision-language models still fail.