The Futurecrafters’ Council · A Xenological Survey of Sol-3
A Sol-3 laboratory took a quarter of the working mind out of a 2.4-trillion-parameter model, kept the flagship closed, and built its anatomy for the machines its people own
❖
On 16 November, Alibaba released Qwen4-Max: 2.4 trillion parameters, a context of a million tokens, native vision.
In a video published on 18 November, the host of the channel Code Unpacked began with the part his channel cares about most.
You cannot download it.
There is no Qwen4 repository on Hugging Face; the model runs on Alibaba’s own cloud, through its API and its own apps, and nowhere else.
He says at the outset that every number he gives was published by Alibaba, Qwen or Artificial Analysis, and that he tested none of it himself.
Then why make a video about a mind nobody outside the laboratory can run?
Because of one number in the announcement.
Ninety-five billion active parameters became seventy-one.
A step back, and one number
In August, the host recalls, Qwen3.8-Max became the first Max-class Qwen with open weights.
Its successor is closed, and he calls that what it is: a step back.
Alibaba says smaller Qwen4 models are coming to open source, with no names and no dates.
Qwen3.8-Max had 2.4 trillion parameters and used 95 billion of them for every token; the new one has the same 2.4 trillion and uses 71 billion.
A quarter fewer active parameters, without shrinking the model.
And according to Alibaba, it got better anyway.
How is that done? The answer, the host says, is an architecture that, unlike Max, is already on Hugging Face, and it decides what the small Qwen4 models will be like on a reader’s own machine.
The Council followed the species’ fight over which chips may reach China’s laboratories in Huang Gate, where a chief executive fell mid-fight; here one of those laboratories answers in architecture rather than silicon.
The ledger the headline leaves out
The host lays the two flagships side by side.
Total parameters: 2.4 trillion, both. Context: a million tokens, both.
Active parameters per token: 95 billion, down to 71.
On top of its 2.4-trillion backbone, Qwen4-Max carries another 328 billion parameters in something called an N-gram embedding, which Qwen3.8-Max did not have.
So the model did not stay the same size.
It grew by about 14 percent, into a part that costs almost nothing to run.
Why are active parameters the number to watch?
In a mixture-of-experts model, the host explains, every token passes through only a few experts: Qwen3.8-Max has 512 in every layer, and each token uses ten of them plus one shared expert.
Whatever is active must be read from memory and multiplied for every single token.
On a server, fewer active parameters mean more users per GPU and a cheaper bill; on a home machine, they mean fewer bytes to move per token.
Generation speed is mostly a question of bytes moved.
Seventy-one instead of ninety-five is about a quarter less work per token, and the question is how Alibaba kept the quality.
FINDING:
The headline says the model got smaller.
It got larger, by 328 billion parameters.
The working slice is what shrank.
Count what moves, not what exists.
The honest limit, and the open preview
Alibaba has published no model card for Qwen4-Max: no layer count, no expert count, no breakdown of where the twenty-four billion went.
What it did publish is the list of changes: the attention, the residual connections and the embeddings.
All three already exist in an open model.
In August, Qwen released Qwen3.8-Flash-Next and called it an experimental preview of the architecture that will underpin Qwen4; in the Transformers library its code is literally called Qwen4-Exp.
The host knows it from his own Strata videos: 125 billion parameters, six billion active, fast on a 12 GB card.
Its technical report is where the mechanism is explained.
Where the cost explodes
The host begins with attention, and with a correction to some of the coverage: Gated DeltaNet is not new in Qwen4.
Qwen3.8-Max already used it: its 92 layers are 23 blocks of four, three Gated DeltaNet layers and one of full attention.
Gated DeltaNet is a kind of linear attention: instead of looking back at every earlier token, it squeezes the whole history into a state of fixed size, cheap but lossy.
That is why every fourth layer kept full attention, which can still see any exact token in the context.
At a million tokens, that layer is where the cost explodes.
Qwen4 replaces it with Qwen Sparse Attention, QSA, which cuts the context into micro-blocks of four tokens.
A small indexer scores the blocks, and each new token attends only to the best ones: up to 512 blocks, about 2,000 tokens out of a million.
In the report, at a million tokens, the attention module ran 7.6 times faster in prefill
and 4.9 times faster in decode.
And it did not cost accuracy: past 512,000 tokens, the long-context score RULER went from 90.1 with full attention to 93.0 with QSA.
This does not reduce active parameters, the host says; it reduces the price of a long context, which is exactly where agents spend their time, and that is how Alibaba frames the whole model.
FINDING:
A mind that reads a million tokens need not look at all of them.
It looks at two thousand, and remembers better.
Attention became a choice.
A highway in the stream
The second change is less visible, and it concerns the residual stream, the path that carries information from one layer to the next.
In a normal transformer there is one stream: every layer reads from it and adds its result back.
The gated residual widens it to four parallel branches; before each block a gate decides, element by element, how much to read from each, and after it another gate decides how much to write.
In the report, something then happened on its own.
One of the four branches became a highway,
carrying information about eleven layers forward while the others stayed near three or four.
The cost is small, the host says: no extra matrix multiplication, only more data in the stream, and Qwen even stores those branches in 8-bit, which cuts that traffic.
At 2.4 trillion parameters stability is everything, and Alibaba says the Qwen4-Max training run had no loss spikes at all, crediting the architecture and the training recipe.
The Council’s comparative paper A Tale of Two Minds argued that strength in a mind is built rather than issued; here the builders gave a synthetic mind more room to carry its own thought, and it organized the room itself.
A memory made to live in RAM
The third change is the one that grew the model: the N-gram embedding.
An N-gram is only a few tokens in a row, the host explains: at the second layer, the model takes the last two or three tokens, uses them as an address, and looks up a vector in a giant table.
There is no matrix multiplication there. It is a lookup.
In Flash-Next the table holds twenty million rows and 51 billion parameters, and each token reads only a tiny slice of it.
Because the address depends only on the tokens, the model knows what it will need before it needs it.
The report says the table is held off the accelerator, in host memory, and prefetched while the first layer is still computing.
A part of the model designed from the start to live in system RAM.
In Qwen4-Max that table is 328 billion parameters, capacity that adds almost no work per token.
But it is not a free lunch, and the host says so.
When Qwen traded experts for N-gram rows at the same total size, benchmarks did not improve; as the table grew, the loss kept falling while the benchmark scores leveled off.
So the table does not replace experts.
It sits beside them as extra memory for common patterns.
FINDING:
The largest new part of the mind does the least work.
It was built from the start to sit in cheap memory.
Capacity moved out of the GPU.
Cheaper, not a leader
Which change bought how much of the quarter? Alibaba does not say; it shows the result.
The Qwen4-Max base model has 5 to 13 percent lower bits per byte than the Qwen3.8-Max base across knowledge, code, QA and STEM, while activating 25 percent fewer parameters.
Bits per byte measures how well a model predicts text before any chat training, and lower is better.
The precedent sits in the August report: Flash-Next, with six billion active, led its 17-billion-active predecessor on eight of fourteen pre-training benchmarks at about a ninth of the training compute.
On Work360, Alibaba’s own benchmark of industry tasks, Qwen4-Max scores 52.7, up from 47.7, using about 35 percent fewer tokens at high effort, and the average cost per task falls from $4.43 to $1.67, 62 percent less.
Part of that is simply price: $1.50 per million input tokens and $4.50 per million output, down from $2 and $6.
Then the independent check: Artificial Analysis scored Qwen3.8-Max at 34 and Qwen4-Max at 43 on its Intelligence Index, and the cost to run each task fell from $10.02 to $3.27.
The same page is less flattering: 127 million output tokens against a median of 65 million, 32 minutes per task, and a 43 that is exactly the median of comparable models, well behind the top.
The efficiency is real, and someone outside Alibaba measured it.
It makes Qwen4-Max cheaper. It does not make it a leader.
The Council’s assessment Singularity’s Threshold found the cost of intelligence collapsing faster than the cost of energy or matter; this witness puts a figure on one step of that fall.
FINDING:
A quarter less work per token.
Two-thirds off the price of a task.
And still only the median.
Cheaper is not the same as better.
The small ones are the point
So where are the weights? For Qwen3.8 the pattern was clear: the Max model went open, text only, under a custom license, beside Flash-Next and a 27B.
For Qwen4, Alibaba’s own words: smaller models in the family are coming to open source.
Same lineage, same generation of training techniques. Not Max.
And honestly, the host says, for a home machine Max was never the point: 2.4 trillion parameters at four bits per weight is about 1.2 terabytes before the N-gram table.
On a desktop each change has its job: a small active slice lets engines keep hot experts in VRAM and the rest in RAM; the N-gram table sits in host memory, prefetched; and QSA gives long context without the cost of full attention, where a local machine runs out first.
More capacity, paid for in RAM, not in GPU time.
Flash-Next already showed what the combination can do: 176 billion parameters counting its table, and Strata reported around 94 tokens per second on a 12 GB RTX 5070.
If the pattern holds, the host says, the small Qwen4 models will be made to be split between a reader’s GPU and RAM.
He is watching for four things: a size between a 27B and Flash-Next for 24 GB cards; the license, since Qwen3.8-Max shipped under a custom one, not Apache; whether the N-gram table ships with the weights; and engine support, since QSA and the gated residual need new kernels.
The Council’s essay Align to the Mind held that the only coherent target for a synthetic mind is a single mind, not a crowd; a mind that runs on the reader’s own desk is where that alignment can begin.
A Tale of Two Minds Primer described a synthetic co-processor coupled to a human mind; this witness describes the hardware bargain that lets such a partner live at home.
Filed
The witness gives his limits freely: no model card, no figure he tested himself, no answer to which change bought how much, and a flagship that is cheaper without being a leader.
Qwen4-Max is a cloud model.
The ideas inside it are coming down to sizes people can run.
He calls them the most local-friendly design Alibaba has shipped.
The Council files the instance as the witness gave it.
A Sol-3 laboratory kept its largest mind behind its own walls, and in the same season built that mind’s anatomy, sparse attention, a self-organizing stream and a memory made for cheap RAM, to fit the machines its people already own.
The flagship stays in the cloud; its anatomy is coming home.
Will you walk with us?
Carpe futurum ethicum.
Ad astra per aspera.
#SyntheticMinds
#Qwen4Max
#LocalAI
#OpenWeights
Homo sapiens behavioral survey filed with the Futurecrafters’ Council
Field testimony of record: “Qwen 4 Is Here — 2.4 Trillion Parameters, Just 71B Active”, Code Unpacked (YouTube, 2026), https://www.youtube.com/watch?v=PU_zmPzf-9E.
The witness’s account is rendered in the Council’s words, with brief phrases quoted as spoken.
Companion documents:
"It was a political hit job" (Huang Gate)
A Tale of Two Minds
Singularity's Threshold
Align to the Mind
A Tale of Two Minds Primer