Cerebras Inference Speed on Wafer-Scale Chip

Cerebras put three of its dinner-plate-sized processors into a single rack for the first time, and the number its cloud partner is quoting for that hardware is 3,000 output tokens per second. The system, the CS-4, was unveiled at the company’s Supernova event in August 2026 and the company said it would ship that quarter. The part worth slowing down on: the chip inside each of those three engines is not new. Cerebras overclocked the existing Wafer Scale Engine 3, branded the variant WSE-3T, and got the headline gains from packaging, clock, and system-level plumbing rather than from a smaller process node.

Key Takeaways:

• The CS-4 packs three WSE-3T wafer-scale engines per rack, Cerebras’s first multi-wafer system, and the silicon is an overclocked WSE-3 rather than a new generation.
• Inference economics at the frontier now hinge on decode throughput, which is bound by memory bandwidth per generated token, not by raw FLOPs.
• OpenAI’s Ultrafast tier runs GPT-5.6 Sol at a vendor-claimed 750 output tokens per second, with no public price or general-availability date.
• Sparse MoE models like Kimi K3.1 Pro (2.8T total, 104B active, 1M context) attack the same bottleneck from the software side by reading a fraction of the weights per token.
• Cerebras reported $25.4 billion in remaining performance obligations, but only 22% of that is expected to convert to revenue through June 2028.

The CS-4 Rack: Three Wafers, an Overclocked Chip

Every previous Cerebras system was a single wafer in a box. The CS-4 breaks that pattern by putting three WSE-3T processors into one rack, which is the first time the company has shipped a multi-wafer configuration. The Next Platform’s launch coverage frames the system under the “Nexus” name and describes the core change as overclocking the existing wafer-scale engine rather than introducing new silicon. HPCwire’s write-up of the same launch draws a deliberate line between the CS-4 and conventional HPC machines, which matters because a rack full of wafer-scale silicon invites comparisons to supercomputers that do not apply.

Three wafers per rack changes the capacity math in a way that single-wafer systems could not. A model has to fit, in the sense that the weights it reads on every decode step have to be reachable at high bandwidth. Adding engines to a rack raises the total on-chip memory available to one serving job and gives the scheduler more room to place experts or layers. It also raises the power and cooling density per rack, which is why the interesting number in the Gimlet Labs deal is measured in megawatts rather than chips.

The company’s own framing of the bottleneck is the part that ages well. As Gearlive’s analysis of the design puts it, most AI chips spend a lot of time waiting for the memory system to haul in the next slab of model weights while the math units sit idle. A dinner-plate wafer with tens of gigabytes of SRAM on die is an attempt to make that trip short enough that the math units never wait.

Why Decode Is a Bandwidth Problem

Generating one token requires reading the weights that participate in that token’s forward pass. There is no reuse across a batch of one: the multiply-accumulate count per byte fetched stays low, so the arithmetic units finish long before the next set of weights arrives. This is the roofline argument, and it is why token-per-second figures track memory bandwidth much more closely than they track peak FLOPS.

A 2024 benchmark paper on the wafer-scale engine, published on arXiv, lays out the relevant hardware properties: 850,000 cores, 40 GB of on-chip memory distributed across those cores, and 20 PB/s of memory bandwidth, with runtime-configurable routing between cores. The authors use a roofline model specifically to test whether that memory hierarchy can scale past the memory wall that limits memory-bound workloads. Whether or not the paper’s numbers still describe the current part, the framing is the right one for reading any inference benchmark: ask what fraction of peak bandwidth the workload achieves, not what fraction of peak FLOPS.

The practical consequence is that “faster” depends on the chip, the model shape, the batch size, and the prompt length together. A rack built for single-stream, long-output decoding will look unremarkable on a prefill-heavy workload that rewards dense matrix math, which is exactly why the industry keeps splitting the two phases apart.

Ultrafast Tiers: OpenAI, Gimlet, and General Compute

The most consequential Cerebras announcement of the year is not a rack. On August 13, 2026, OpenAI and Cerebras previewed a service tier called Ultrafast that runs GPT-5.6 Sol at a vendor-claimed 750 output tokens per second. The claim came without a price or a general-availability date, so treat 750 as a target until independent measurements exist. It is still a signal: the largest model vendor in the market decided that a latency tier was worth building on wafer-scale hardware.

The supply side scaled up in September. Reuters reported that Cerebras plans to supply AI systems with capacity to consume roughly 100 megawatts of power to Gimlet Labs, a cloud startup that will combine wafer-scale engines with GPUs and targets up to 3,000 tokens per second. Gimlet is a launch partner for the CS-4 and its first data center was slated to come online this year. A separate deal with General Compute, reported by Crypto Briefing, builds a hybrid Nvidia-Cerebras inference stack with capacity launching in Q1 2027.

The hybrid pattern is the tell. eWeek reported that AMD and Cerebras are splitting inference between AMD’s Helios rack-scale system and wafer-scale chips, an arrangement the outlet describes as disaggregated inference. Disaggregation means the prefill phase and the decode phase run on different hardware matched to each one’s bottleneck. Nobody is betting the whole serving path on one kind of silicon.

Kimi K3.1 Pro and the Sparsity Counterargument

Moonshot AI’s Kimi K3.1 Pro, released in late October 2026, takes the opposite route to the same bottleneck. It is a mixture-of-experts model with 2.8 trillion total parameters but only 104 billion active per token, alongside a 1-million-token context window and a score of 65.7% on Terminal-Bench 4.0, according to MarkTechPost’s release write-up. Several outlets rounded the parameter count to 3 trillion; the 2.8T figure with 104B active is the one that matters for hardware planning.

That ratio is the whole argument. If roughly 3.7% of the weight set participates in each token, the bytes a serving system must move per token drop by an order of magnitude, and a rack of GPUs becomes viable where dense weights would have forced a wafer-scale system or a heavy quantized offload. VentureBeat reported that the model is flirting with open-weights leadership, and Moonshot has said full weights should land by November 2026.

The caveats are real. Reuters reported that Moonshot deliberately reduced the model’s cyber-offensive capabilities before release, so benchmark-to-benchmark comparisons against unrestricted models can mislead. VentureBeat also reported allegations of paid influencer promotion around the launch. Neither affects the architecture, but both affect how much weight to put on launch-week benchmark scores.

Sparse routing has a cost on the memory side too. Every expert in the pool has to be resident somewhere even when it is not computing, which is why the research literature on offloading exists at all. One arXiv paper on predictive expert caching, ExpertFlow, reports cutting GPU memory use by up to 93.72% and improving throughput by up to 10x over strong offloading baselines on a single GPU. Another, QMoE, compresses a 1.6-trillion-parameter SwitchTransformer to under 160 GB at 0.8 bits per parameter with less than 5% runtime overhead on four A6000 cards. Both are software answers to the problem wafer-scale silicon solves with area.

Measuring Decode Rate Yourself

Vendor headline rates are measured under chosen prompt and output lengths, so the number you get in production will differ. The only defensible way to compare tiers is to measure time to first token and steady-state decode rate on your own traffic, then price them per million output tokens at your actual concurrency.

Hold the prompt constant, keep the output length fixed, and run enough samples to see a distribution. A tier that wins on median decode rate can still lose on time to first token for short outputs, and most chat traffic is short outputs.

Contracts, Cash, and the Limits of the Rack

The commercial picture is more mixed than the benchmark picture. Cerebras holds $25.4 billion in remaining performance obligations, but only 22% of that is expected to convert into revenue through June 2028, and cloud and services revenue grew 287% year over year to $127.7 million. A backlog that long is a scheduling asset and an execution risk at the same time, because the hardware it is committed against will be two generations old by the time most of it is recognized.

The stock has reflected that tension. Cerebras priced its IPO at $185 per share and Forbes described it as the biggest listing of 2026. By October 1, 2026 the stock closed at $169.52, 8.4% below the offer price, after a reported shift by Nvidia in the ultrafast tier and an $80.8 million share sale by the chief operating officer.

The technical limits have not moved. A wafer-scale system is a bet that a single job can use an enormous amount of on-chip memory, which means workloads that shard cleanly across many small devices do not automatically benefit. Software portability is narrower than with CUDA, and a customer who routes prefill to GPUs and decode to wafers now operates two toolchains and two failure domains. The 2026 answer from the market is disaggregation: keep the GPU fleet for what it does well, buy wafer-scale capacity for the decode step, and measure both against your own token-rate numbers rather than a launch slide.

Related Reading

More in-depth coverage from this blog on closely related topics:

• Why Ambitious Software Projects Fail
• AI Productivity Gains Outside Tech
• StubHub Stock Slips on Record Ticket Sales
• Kimi K3.1 Pro Benchmarks and Pricing
• AI Data Center Emissions by Region

Sources and References

Sources cited while researching and writing this article:

• Cerebras Overclocks WSE-3 Waferscale Engine To Boost Inference Oomph In “Nexus” CS-4
• Benchmarking the Performance of Large Language Models on the Cerebras Wafer Scale Engine
• GPT-5.6 Sol Now Runs at Real-Time Speed: OpenAI’s Ultrafast Preview Offers No Price or Date
• Moonshot AI Releases Kimi K3.1 Pro: A 2.8T Total/104B Active Params (MoE) Model with 1M Context
• Moonshot’s Kimi K3.1 Pro flirts with open weights leadership amid paid influencer scandal allegations
• Reuters reported
• ExpertFlow: Efficient Mixture-of-Experts Inference via Predictive Expert Caching and Token Scheduling
• QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models