Nvidia Announces Native GPU Support for Rust

Key Takeaways:

• At RustConf 2026 in Montreal, NVIDIA announced CUDA Rust: native GPU kernel authoring in Rust on two tracks, plus NVIDIA joining the Rust Foundation as a Platinum member.
• cuda-oxide compiles SIMT kernels directly to PTX through a custom rustc codegen backend; cutile-rs compiles tile kernels through CUDA Tile IR on stable Rust.
• Both catch pointer aliasing between kernel arguments at compile time. Neither guarantees shared-memory race safety between threads: cuda-oxide requires unsafe there, and cutile-rs avoids the problem by design.
• On B200, NVIDIA reports 2.07 PFlop/s on a persistent f16 GEMM (96.4 percent of cuBLAS) and 7 TB/s element-wise bandwidth, about 91 percent of the 8 TB/s HBM3e peak.
• cuda-oxide is early alpha; cutile-rs is further along and already used in Hugging Face’s Grout and mistral.rs. The safety improvement applies to argument aliasing, not to the entire kernel.

NVIDIA announced CUDA Rust on September 8, 2026, in a post on its Technical Blog by Sri Koundinyan, Melih Elibol, and Jonathan Bentz. The same week, at RustConf 2026 in Montreal, the Rust Foundation named NVIDIA a Platinum member alongside the Solana Foundation. The announcement changes how much of the GPU stack can be written in one language, and it carries a precise, narrow safety claim that is easy to overstate.

Rust has been expanding into the systems layer for years. NVIDIA’s Nova Linux driver is written in Rust, NVIDIA Dynamo runs on a Rust core, and NVTX ships Rust bindings. The kernel itself was the holdout: you could launch it from Rust, but the body usually had to be written in CUDA C++ and compiled with nvcc. CUDA Rust makes the kernel a Rust compilation target rather than a wrapper around code produced elsewhere.

What NVIDIA Shipped at RustConf 2026

CUDA Rust consists of two open-source projects under the NVlabs organization, each matching a programming model CUDA already exposes in C++ and Python. The SIMT track corresponds to CUDA C++: you describe the work of a single thread, then launch thousands. The Tile track is the newer model where you describe operations on a tile of data and the compiler maps tiles onto hardware.

NVIDIA recommends starting with Tile. As the announcement states, “the compiler decides how tiles map onto each architecture, so your source does not encode architecture-specific choices,” and you switch to SIMT when you need to control memory and threads directly. The company also plans inter-language interoperability between CUDA Rust, CUDA C++, and CUDA Python so that choosing a frontend does not lock a team out of the others.

Both repositories are active. As of this writing, NVlabs/cuda-oxide has 3,505 stars, 273 forks, and 68 open issues, and NVlabs/cutile-rs has 980 stars, 82 forks, and 45 open issues. Both are Apache-2.0 and were last updated within the past week. Those figures come from the GitHub API, not from the announcement.

The SIMT Track: cuda-oxide, DisjointSlice, and Launch Contracts

cuda-oxide is a custom rustc codegen backend. It intercepts compilation and routes #[kernel] functions through Rust MIR, the community Pliron IR framework, and LLVM IR down to PTX, while handing everything else to the standard backend. Installing it requires a pinned nightly toolchain:

# cuda-oxide requires: Linux, a GPU with compute capability 8.0+,
# CUDA toolkit 12.x or newer, clang with libclang headers, pinned nightly.
cargo +nightly-2026-04-03 install --git https://github.com/NVlabs/cuda-oxide.git cargo-oxide

# Scaffold a template project, verify the environment, then run it.
cargo oxide new vecadd_demo
cd vecadd_demo
cargo oxide doctor
cargo oxide run
# Prints: PASSED: all 1024 elements correct

# Note: the first run builds the codegen backend and is slow.
# Later runs reuse the cache.

The kernel signature carries the safety argument. Inputs are shared slices readable by any thread. The output is a DisjointSlice, which gives each thread exclusive access to its own element. It exists because &mut [f32] cannot represent thousands of simultaneous mutable borrows, and Rust forbids that.

use cuda_device::{kernel, launch_bounds, launch_contract, thread, DisjointSlice};

#[cuda_module]
mod kernels {
 use super::*;

 #[kernel]
 #[launch_bounds(256)]
 #[launch_contract(domain = 1, block = (256, 1, 1))]
 pub fn vecadd(a: &[f32], b: &[f32], mut c: DisjointSlice) {
 let idx = thread::index_1d();
 if let Some(c_elem) = c.get_mut(idx) {
 *c_elem = a[idx.get()] + b[idx.get()];
 }
 }
}
// Note: production kernels should add explicit buffer-size contracts,
// error handling for load failures, and a compute-sanitizer pass in CI.

According to NVIDIA’s cuda-oxide safety model documentation, the safety depends on a pair of types. ThreadIndex is an opaque wrapper around a usize with no public constructor; it can only be created from hardware built-in registers, and it is !Send + !Sync + !Copy + !Clone with a lifetime tied to the kernel body. DisjointSlice::get_mut() accepts only a matching index space, returns an Option, and bounds-checks the access, so out-of-bounds becomes a branch you handle rather than a memory error you find later. The #[launch_contract] attribute declares that the kernel indexes in one dimension with 256-thread blocks, and the host-side launcher validates your launch configuration against that declaration before any driver call. Kernels without a contract expose only raw unsafe launch methods.

The Tile Track: cutile-rs and Tensor Ownership

cutile-rs works at a higher level and with lighter requirements: stable Rust 1.89 or newer, CUDA 13.3, and Linux. It needs no nightly toolchain and no libclang. It is published on crates.io, so you can add it without cloning:

cargo new vecadd_demo
cd vecadd_demo
cargo add cutile

Element-wise addition for tiles looks like this, performing the same operation as the SIMT kernel:

use cutile::prelude::*;

#[cutile::module]
mod kernel {
 use cutile::core::*;

 #[cutile::entry()]
 fn add(z: &mut Tensor, // exclusive output
 x: &Tensor, // shared input
 y: &Tensor) {
 let tx = x.load_like(z);
 let ty = y.load_like(z);
 z.store(tx + ty); // elementwise across the tile
 }
}

fn main() -> Result<()> {
 let x = api::ones::<f32>(&[1024]);
 let y = api::ones::<f32>(&[1024]);
 let z = api::zeros::<f32>(&[1024]).partition([128]);

 let (_z, _x, _y) = kernel::add(z, x, y).sync()?;
 Ok(())
}
// Note: this sketch omits device selection, error propagation for the
// JIT step, and shape validation for tensors that are not evenly divisible.

There is no DisjointSlice here. Host-side partitioning assigns each tile block a writable sub-tensor that no other block overlaps, and that exclusivity matches what &mut already guarantees. Everything before .sync() is a lazy description rather than an immediate launch: ones, zeros, the kernel call, and the copy back are all recorded, and the whole program forms one chain with a single synchronization point.

The Fearless Concurrency on GPU paper (Elibol, Roesch, Gelado, Buehler, Garland) formalizes this: mutable outputs are split into disjoint pieces, and kernel launches preserve the host-side ownership contract. The ownership claim follows tensors across the launch boundary, which is the stronger of the two tracks’ guarantees.

What the Compile-Time Safety Actually Covers

Both tracks enforce memory safety at compile time, but the phrase requires clarification because two different bugs fall under “aliasing.” The first is pointer aliasing between kernel arguments in global memory: two parameters that turn out to be the same buffer, so a write through one is visible through the other. This is what CUDA C++’s __restrict__ addresses, and it is a fact about which arguments you passed, checkable without knowing anything about the other threads in the block.

Passing a SIMT kernel’s output buffer as one of its own inputs does not compile, whether or not that kernel would actually race:

module.vecadd(&stream, &prepared, &c_dev, &b_dev, &mut c_dev)?;
// error[E0502]: cannot borrow `c_dev` as mutable because it is also borrowed as immutable

The second bug is a race between threads in a block, usually over shared memory: thread 5 and thread 12 hit the same on-chip location with no ordering between them. This depends on concurrent execution, and __restrict__ says nothing about it. cuda-oxide’s own documentation is clear here. It organizes kernels into three tiers: Tier 1, built from shared read slices and DisjointSlice outputs launched through a checked contract, is safe by construction and covers the first bug. Tier 2, which includes shared memory, warp shuffles, atomics, and barriers, requires unsafe with documented contracts. Tier 3, covering TMA, tensor cores, and cluster-level communication, is fully manual.

The documentation lists what is not enforced today: thread-divergent control flow around barriers (worked around by disabling an LLVM optimization pass, not by proving anything) and warp convergence for shfl_sync and ballot_sync, where a mistake yields a silent hang. cutile-rs avoids the block-race problem rather than solving it, because a tile block is a single logical thread with no thread indices for you to get wrong.

The Cost of Bounds Checks and How to Pay It Once

Every a[i] in a kernel includes a check: is i inside the slice? That is a compare and a branch on every access, in every thread, and in a hot loop it shows. According to the cuda-oxide bounds-checks chapter, a naive GEMM spends most of its inner loop on two such checks per multiply-add and reaches about 2,940 GFLOPS on an RTX 5090; the same kernel with checks removed runs at 7,160 GFLOPS.

The solution is to check a fact once and carry the proof in the type. A MatrixView32 wraps a slice with its row width and checks a whole row or column in one comparison, returning a view whose reads need no further checks. The documentation reports that the safe view kernel reaches 7,159 GFLOPS against 7,161 GFLOPS for the hand-written unsafe raw-pointer twin, a safety cost of about 0.1 percent. There is also a blunt escape hatch, #[kernel(unchecked_indexing)], which deletes indexing checks entirely and leaves undefined behavior if the index is wrong.

Measured Performance on B200

The performance figures in the paper come from NVIDIA, and they compare the framework against the vendor library it is competing with.

Two points require attention. The 2.07 PFlop/s figure is about 92 percent of the B200’s 2.25 PFLOPS dense FP16 peak, and the 7 TB/s figure is achieved element-wise throughput, not the hardware’s rated bandwidth; the B200’s HBM3e peak is 8 TB/s. The 96.4 percent headline is a persistent-GEMM result that NVIDIA measured. An independent cross-architecture evaluation of the CUDA Tile stack reported a wider 52 to 79 percent of cuBLAS for GEMM, and only 53 percent of FlashAttention-2 throughput on the RTX PRO 6000 Blackwell Server Edition, with the authors noting that effectiveness depends strongly on workload and architecture. The gap between 96.4 percent and 52 to 79 percent reflects the difference between a tuned persistent kernel and generic tile code, and it is important to know which one you are reading about.

The paper also reports end-to-end results through Grout, a cuTile-Rust-based inference engine. In batch-1 decode, Grout reaches 171 generated tokens per second for Qwen3-4B on an RTX 5090 and 82 generated tokens per second for Qwen3-32B on a B200, which the authors describe as competitive with vLLM and SGLang. Those are the authors’ numbers from their own engine, not an independent head-to-head.

Maturity, Shared Memory, and API Stability

Neither project is ready for production. cuda-oxide is early alpha: because it hooks directly into rustc internal data structures as a custom codegen backend, it requires a pinned nightly toolchain and a dedicated LLVM build, and breaking changes from upstream compiler internals remain common. NVIDIA warns that coverage is incomplete and APIs will move. cutile-rs is further along, published on crates.io and already used outside NVIDIA in Hugging Face’s Grout inference engine and in mistral.rs as a backend for GEMM and attention operations. “Early alpha” applies to both repositories even where adoption exists, and the interop that would let a team migrate one kernel layer at a time without discarding existing CUDA C++ assets is still a stated plan rather than a shipped feature.

The practical reading: if your kernel is a straightforward element-wise, reduction, or GEMM operation that fits the tile model or a checked SIMT contract, both tracks can eliminate a real class of production bug at build time, and cutile-rs lets you do it on stable Rust today. If your kernel depends on hand-tuned shared memory, warp shuffles, or TMA, you are still writing unsafe and handling the aliasing reasoning yourself, because no type system in this release proves block-level races safe. The announcement means the safe subset now covers common kernels, and the boundary of that subset is documented rather than hidden.

Related Reading

More in-depth coverage from this blog on closely related topics:

• Cut LLM Token Costs with UCCP Compression
• Enterprise Kubernetes DNS via Tailscale VPN
• Implementing Kruskal’s Algorithm in Go

Sources and References

Sources cited while researching and writing this article:

• Introducing CUDA Rust: Two Tracks for Writing GPU Kernels
• Rust Foundation Announces Solana Foundation and NVIDIA as Platinum Members
• NVIDIA Dynamo
• GitHub – NVlabs/cuda-oxide: cuda-oxide is a Rust-to-CUDA compiler that …
• NVlabs/cutile-rs
• Pliron
• The Safety Model — cuda-oxide
• [2606.15991] Fearless Concurrency on the GPU – arXiv.org
• Grout
• mistral.rs