ONNX Runtime 1.30 Adds Quantized WebGPU KV Caches
ONNX Runtime 1.30, released September 10, adds INT8 key-value cache quantization and GPT-OSS support to its WebGPU provider, plus linear-attention kernels for Arm64 CPUs. These changes can reduce memory pressure or expand model coverage, but Microsoft published no browser benchmark for the release. Apps still need testing on each model, browser, GPU, and fallback path.
On this page
ONNX Runtime 1.30 extends the native WebGPU path
Microsoft released ONNX Runtime 1.30 on September 10 with a concentrated set of changes for generative AI. The native WebGPU execution provider now supports GPT-OSS models, adds INT8 block quantisation for key-value caches, and expands its PagedAttention implementation.
Those additions address two practical browser constraints. Paged attention stores the growing attention cache in blocks, while an INT8 cache uses less memory than a higher-precision cache. In principle, that can leave more GPU memory for model weights and longer conversations. It does not prove that every model will fit or become faster: the graph must use the supported path, quantisation can introduce accuracy differences, and browser, driver, and GPU behaviour still vary.
The release also expands WebGPU convolution fusion, adds more subgroup-matrix paths, improves pipeline compilation and cache keys, and fixes out-of-bounds loads for partial matrix tiles. These are runtime building blocks rather than a single user-visible feature. Microsoft did not publish a cross-browser speed, memory, or quality benchmark for version 1.30.
Arm64 CPUs gain linear-attention and normalisation kernels
The CPU execution provider adds fused LinearAttention kernels for Arm64 NEON and SVE, alongside an AVX-512 implementation for desktop and server CPUs. It also adds AVX2 LayerNorm and RMSNorm acceleration, improves INT4 weight prepacking, and enables a fast matrix-multiplication path on Darwin Arm64.
That makes the release relevant beyond desktop GPUs. Arm64 covers modern phones and Apple silicon, but inclusion in ONNX Runtime does not mean an Android or iOS app automatically uses the new kernels. The application must ship version 1.30, export a compatible graph, select the CPU provider for those nodes, and run on hardware with the required instructions.
The right measurement remains model-specific. Record model-load time, prompt processing, time to first visible output, sustained generation, peak memory, cancellation, and behaviour after the device warms up. A faster attention kernel cannot compensate for an oversized model or a long prompt that dominates prefill. First-token latency and generation speed are separate problems.
Model loading and browser reliability receive hardening
Version 1.30 limits nested model-graph depth, canonicalises external-data locations, and strengthens shape, rank, index, and allocation checks across CPU, CUDA, and browser paths. WebGPU fixes include rejecting foreign GPU handles, zero-initialising writable device buffers, and reporting an error instead of terminating the process when tensor-copy APIs are misused.
This matters for local applications that allow users to open models obtained elsewhere. A local model file is still untrusted input, and running it offline does not remove parser, memory-safety, or resource-exhaustion risk. Runtime hardening narrows that attack surface, but applications still need artifact verification, bounded downloads, controlled file access, and a clear failure state.
The release notes say their highlights were prepared with AI assistance. The linked pull requests and shipped tag are the underlying evidence, and the claims here are limited to changes documented there. No independent performance validation was available at publication time.
Upgrade only with the production model in hand
For a browser application, the most useful 1.30 test compares the exact production model on the current runtime and the candidate runtime. Check that WebGPU actually executes the intended attention and cache path, then repeat with WebAssembly fallback. A successful load is not enough if output quality changes, memory grows across turns, or cancellation leaves the interface stuck.
Native applications should also inspect package and execution-provider changes. Version 1.30 introduces Go bindings, Linux AArch64 packaging for the WebGPU plugin provider, and new defaults for some CUDA quantisation kernels. These can matter to a deployment pipeline without changing behaviour on a phone or in Chrome.
ONNX Runtime 1.30 is a credible upgrade candidate for local and browser inference because it combines broader generative-model coverage with memory and safety work. Treat its benefits as capabilities to verify, not performance promises. A complete offline browser stack still depends on model storage, service-worker behaviour, and tested fallbacks.