ExecuTorch 1.5 Adds Cancellation and New On-Device LLM Paths
ExecuTorch 1.5 was released on September 16, 2026. It adds bounded cancellation and batched request scheduling for language-model serving, a C++ SDK inside its Python wheels, and new export and backend work. Those are tools for developers building local AI apps, not an automatic update to existing apps. Android performance and Stop-button behaviour still need testing with the exact model and device.
On this page
Generation can be cancelled within a defined bound
PyTorch released ExecuTorch 1.5 on September 16. Its language-model serving changes include bounded cancellation, batched request scheduling, multi-method export, and layouts that keep the key-value cache outside the model graph. These are runtime capabilities for developers embedding models in phones, computers, and edge devices. The release does not update any consumer app by itself.
Cancellation is especially relevant to an interactive chat interface. Pressing Stop should end an active generation and leave the app ready for another prompt. A runtime cancellation mechanism gives an app a way to interrupt work, but the release notes do not establish how quickly a particular Android phone will stop or whether an app's composer will recover correctly. That also depends on its model runner, backend, and user-interface state.
Batched scheduling matters when a device or service handles more than one request. The new off-graph cache layouts give developers additional control over memory ownership. Neither feature is a blanket promise of lower time to first token. Prompt length, model size, cache allocation, and the selected hardware path still determine what the user experiences. The phone latency guide separates slow prompt processing from slow token generation.
Packaging now reaches beyond Python bindings
Version 1.5 puts a linkable C++ SDK into the Python wheel, alongside packaged TorchAO kernels and libraries for several delegates, including Qualcomm, Core ML, MLX, CUDA, and OpenVINO. The release also adds schema-version checks for model files and clearer failures for missing delegate data. These changes can simplify a build pipeline and make an incompatible export fail explicitly instead of producing a mysterious runtime error.
The backend work is broad but uneven by design. Qualcomm gains language-model quantization-aware distillation and Gemma 4 text-model enablement. Arm adds Ethos-U65 support, Ethos-U85 cache export, and more model workflows. Vulkan gains quantized convolution paths, safer dispatch handling, and external buffer integration. Apple-oriented Core ML and MLX packaging advances separately. A feature on one delegate does not make it available on every Android device.
The release lists new Qwen3.5 MoE, Muse Glimmer, Supertonic, and Voxtral workflows. Here, “workflow” means a documented or supported route through export and execution, not proof that all of those models fit a phone or meet a particular speed target. A large checkpoint can still exceed a device's memory even when the runtime recognises its architecture.
Upgrade against a real app and model
For an Android developer, the useful first test is an existing qualified model on a known phone. Compare a cold launch, a short prompt, a longer prompt, and repeated generation before and after the upgrade. Stop during generation, send a second prompt, and check that the app releases resources and restores its send control. Measure memory and heat as well as visible output timing.
The 1.4 release already expanded Kotlin and Qualcomm deployment options. Version 1.5 adds serving and packaging work on top of that foundation. It is a reasonable upgrade candidate for teams that need those specific features, subject to backend compatibility and regression testing. PyPI confirms that the 1.5.0 package was published on September 16; the release notes and tag document the actual feature set.
There is no independent same-device benchmark for this release in the sources reviewed. A claim that ExecuTorch 1.5 makes local chat faster on Android would need measurements from a real app, model, and phone.