Qwenputer: A RISC-V CPU Inside an LLM

Divyansh Agrawal pushed a commit on October 19, 2026 that runs a working RISC-V CPU inside a language model. The chip is arithmetic baked into Qwen3.5-4B’s weights. No gradient descent touched a single one of them. A compiler wrote them.

The numbers are the strange part. The core occupies 11 added transformer layers, 42,012 MLP units and 99 attention heads, all built from 1,040,595 nonzero weights written straight into the checkpoint. Qwen’s own 32 layers are untouched. The machine’s RAM is the model’s Gated DeltaNet state. Agrawal calls the result “a computer, compiled into the weights of a language model,” and the repository is divagr18/qwenputer.

The substrate is the point. A linear-attention layer keeps a state matrix whose size is set by the architecture, not by how long the conversation runs. Hold the decay gate at 1, hold the write strength at 1, address rows with one-hot keys, and the state stops behaving like a compressed summary. Send a key that is all zeros except for a single 1 and the write lands on exactly one row. Send that shape back as a query and the row returns unchanged. That is memory you can overwrite and read back on demand, which is what a CPU needs, and why the trick fits in 4 billion parameters.

What Qwenputer Actually Is

The ISA is RV32IM plus Zba, with a bignum coprocessor that keeps values as up to 62 decimal digits outside the address space. Memory is small and mapped the way a microcontroller is: 32 KB of RAM at address zero, 128 KB of read-only data at 0x20000, and 256 KB of code at 0x40000. Each instruction sits in ROM as one pre-decoded 4-byte word, so decode costs a single MLP stage instead of a pipeline.

Where the Weights-as-Hardware Approach Breaks

The interesting engineering sits in the bridge: Qwen writes a tool call for a function called calc, and nothing intercepts that text. Tokens following the call tag get captured into a text buffer that the core reads as memory at 0x8000, 16 bytes per token and up to 80 tokens per call. The chip layers run the program, and every byte the program prints is echoed back as a matching Qwen token. Qwen then reads its own reply as ordinary text.

Gated DeltaNet State as the RAM Substrate

Linear attention replaces the unbounded softmax cache with a fixed-size recurrent state, which cuts sequence mixing to linear time and decoding to constant memory, as the Gated DeltaNet-2 paper puts it. The delta rule is what makes that state editable: it subtracts the current read before writing a new value, so a write can replace one association without scrambling the rest. Agrawal cites Schlag, Irie and Schmidhuber’s fast-weight formulation as the ancestor and names the difference himself: exact, individually addressable memory.

Three settings do the work. The gate is held at 1, so nothing decays. Write strength is held at 1, so a write replaces rather than blends. The key is one-hot, a vector of zeros with a single 1, which points at exactly one row. Low bits of the address pick a head, the remaining bits pick a row, and each row stores one byte as bits. Registers, program counter and call stack each get heads of their own, and each key head drives two value heads, doubling capacity.

The head geometry, not the parameter count, sets the floor. Agrawal considered a smaller 0.8B base model and skipped it because that checkpoint has one value head per key head while the core needs two. The 4B model’s ratio of heads to dimensions is what made the memory map fit. That is a narrower claim than “4B is the right size.”

Determinism: A Forward Pass as a Program Run

A weight written by a compiler is a constant. Evaluate the same input against the same constants and the arithmetic repeats, which is the whole argument behind calling the core exact. The evidence is a conformance run. The project design document reports rv32ui (38 tests), rv32um (all 8, including div, divu, rem and remu) and rv32uzba (3 tests) green, all 61 local ELF binaries producing the same exit code and stdout as qemu-riscv32, and 15.3M instructions free-running without correction between steps. The local suite reports 228 tests with 14 skipped.

That exactness is paid for in tokens. The model has no clock signal. Each instruction consumes a few decoded tokens (one naming a row, one carrying a value, one ending the step), so the token budget is the machine’s only clock:

Program	Instructions	Trace tokens	Tokens per instruction
MicroPython s5 snippet	492,526	610,544	1.24
MicroPython s1, print(2**100)	39,888	50,150	1.26
fib(25)	2,114,629	2,174,558	1.03
qsort, 1,000 elements	199,157	240,176	1.21
sieve below 10^4	123,535	176,225	1.43
mini-gmp, 20 by 20 digits	14,767	22,671	1.54
crc32 printf	4,109	6,283	1.53

Ten thousand random calc cases consumed 8,951,798 instructions and 12,298,128 tokens in 19 minutes on a GPU. Every one matched Python on the reference machine before the circuit reproduced it free-running. That is the shape of the claim: reproducible execution of real binaries, not fast execution of them.

The Doom E1M1 Demo as the Stress Test

A separate repository, mapestree/qwenputer-doom, plays Doom E1M1’s floor plan on a RISC-V CPU compiled into Qwen3.5-4B’s weights and built on Qwenputer. It is the floor plan, not the full renderer. Nobody has claimed a complete game running inside the forward pass.

A floor plan walk means tight geometry loops, sustained multiply and add, and a long instruction stream that cannot be short-circuited by the model guessing an answer. It tests whether the trace stays coherent for millions of tokens.

Retro first person shooter video game displayed on a monitor — A floor plan walk is a sustained loop, which is what makes it a better test of the trace than a single arithmetic call.

The throughput numbers explain why nobody will run this in production. On an L4, a Triton executor using one fused kernel per sublayer and one CUDA graph per token retires 3,699 instructions per second at batch 1, roughly six times the earlier dense-block path:

Executor path	Batch	Tokens per second	Instructions per second
P1 executor, dense weight blocks	1	938	614
Sparse gather and scatter path	1	3,836	2,509
Sparse gather and scatter path	8	18,738	12,254
Triton executor v2	1	5,657	3,699
Triton executor v2	8	18,150	11,870
Triton executor v2	32	21,309	13,936

The wall is text decode, not arithmetic. Qwen’s own tokens cost about 68 ms each at batch 1 on an L4, which dwarfs the chip bursts. A single mul_div problem had a median of 242 seconds on the executor path and 585 seconds on the unoptimized one. The sparse path helps for one reason: the dense weight blocks move 165 MB per step against 0.78M nonzeros, and sparse rows cut that by roughly 25 times.

Where the Weights-as-Hardware Approach Breaks

Exactness is local, and the design notes show where it stops. An early version read eight rows of one head at once; the multi-hot query’s 1/sqrt(8) scaling is not exact, and the resulting fp32 residues near 2e-8 broke a crc32 test. Every read had to go back to one-hot or two-hot. Batching breaks the surrounding text too: six records decoded at batch 8 matched batch 1 on five of them, with the sixth diverging at token 8,390 because Qwen’s GEMMs round differently at M=1 than at M=B. Every circuit-owned position still checked out. The CPU is exact. The host system is not bit-stable.

The bridge capture window is 80 tokens because its thermometer weights overflow bf16 past that point after the emitter’s 2^36 fold. The ISA covers RV32IM plus Zba, with no floating-point or compressed instructions in the booted image, and CSR accesses resolved as compile-time stubs. Agrawal’s own framing is that this is not a Linux-class chip.

The most instructive failure was not in the CPU. In the first calc version, 610 of 1,040 calls returned an error because fractions were only accepted as a final value, and Qwen responded by retrying in loops, with 15 of 20 misses making 20 or more calls. After the firmware moved to exact rational arithmetic, 1,083 calls produced 1,020 values with none wrong, 41 semantic errors and 22 “too complex” rejections. On the hardest multi-step set, the in-weight machine scored 40/40 against 39/40 for a host Python tool. An exact CPU does not fix a lossy interface, and the error vocabulary is design surface, not plumbing.

A RISC-V core in silicon costs a few dollars and retires instructions at rates this approach will not reach. The repo sat at 150 stars, 16 forks and 43 commits as of October 29, 2026, with a single author, MIT-licensed code and Apache-2.0 weights, and no third-party replication of the conformance results has surfaced. The NYU Shanghai RITS write-up notes that the author’s claims deserve more scrutiny than a blog post can give them, which is the right posture.

Reading the Repo: What to Check Yourself

Start with experimental/riscv/docs/rv32.md, which carries the layer map, the DIV datapath and every measurement quoted above. Then confirm the four things that make the claim falsifiable: the ISA string (RV32IM plus Zba, not RV32GC), the memory map (32 KB RAM, 128 KB DROM, 256 KB ROM), the conformance scope (rv32ui, rv32um, rv32uzba), and whether the demo path still requires the RvChat driver rather than stock generation.

git clone https://github.com/divagr18/qwenputer && cd qwenputer/experimental/riscv && uv sync --extra hf
hf download divagr1925/Qwen3.5-4B-Qwenputer --local-dir qwenputer-rv
uv run python -m chipformer.rv.chat qwenputer-rv --firmware tests/data/rv/calc-bridge.elf \
 --ask "Use calc: what is 123456789 * 987654321?"

# Note: RvChat is required because it loads the firmware into DeltaNet state after
# the prompt and keeps chip trace tokens out of Qwen's layers. The checkpoint is
# 18.7 GB in bf16, and the design doc puts the merge build at about 31 GB of RAM.
# The system is not bit-stable across batch shapes, so pin batch size before
# comparing two runs.

The project’s most useful claim is that a fixed-size recurrent state in a small open model is addressable enough to hold a working machine, and that the machine’s limits come from head geometry and token budget rather than from model scale. The author’s write-up says the next step is running Python on MicroPython inside the merged checkpoint through a python() tool, with faster execution after that. In-weight deterministic compute removes a trust boundary, since there is no sandbox to audit and no interpreter to patch, but it adds a token cost that no serving budget absorbs today. The techniques that do pay off now are kernel fusion, CUDA graphs and sparsity, all three of which this project used to get from 614 to 3,699 instructions per second, as covered in our guide to inference acceleration.

Key Takeaways:

The CPU is an RV32IM core with the Zba extension and a bignum coprocessor, compiled into 11 added layers as 1,040,595 nonzero weights. Nothing was trained.

Its RAM is Gated DeltaNet state: a one-hot key overwrites exactly one row, a one-hot query reads exactly that row back. Size is fixed by architecture, not by context length.

Tokens are the clock. Programs cost between 1.03 and 1.54 trace tokens per instruction, so fib(25) burns 2.17 million tokens to run 2.11 million instructions.

Exactness holds at the circuit but not at the system boundary: an 8-row multi-hot read left fp32 residue near 2e-8, and batching changed Qwen’s text at token 8,390.

The arithmetic benchmark failures were mostly interface failures, not CPU failures: 610 of 1,040 calc calls errored in v1 on the tool’s own semantics.

More in-depth coverage from this blog on closely related topics:

Sources and References

Sources cited while researching and writing this article: