Huawei MACaQ: Open Quantization Tool Handles 122B in 64 Minutes
MACaQ is an Apache 2.0 quantization framework from Huawei's Computing Systems Lab that converts both weights and activations to 4-bit values, one linear layer at a time. The paper reports quantizing a 122B-parameter model in 64 minutes with 32.5 GB of GPU memory. A companion tool, MACaQ-R, refines existing GGUF files while staying compatible with standard llama.cpp.
On this page
What MACaQ does differently from other quantizers
Researchers at Huawei's Computing Systems Lab in Switzerland posted the MACaQ paper to arXiv on 7 November 2026 and released the code under Apache 2.0. The name stands for Model Agnostic Calibrated Quantization. It is a post-training method, meaning it compresses a finished model using a small set of calibration data rather than retraining it.
Most 4-bit files that local users download quantize only the weights. MACaQ targets W4A4: 4-bit weights and 4-bit activations, using the MXFP4 number format. Activations are the intermediate values a model computes as it runs, and squeezing them is harder because errors compound from layer to layer.
The method treats every linear layer on its own. At inference time it applies a per-channel scale and a small block-Hadamard transform to the layer's inputs, and folds the matching inverse into the weights ahead of time, so the layer still computes the same operation before rounding. Nothing around the layer needs to change, which is why the authors call it model agnostic. A second step, Flip-Refinement, then nudges individual 4-bit codes to neighbouring values wherever that lowers the measured error.
The speed and memory numbers Huawei reports
Because layers do not depend on each other, they can be processed in a stream with only one in memory at a time. That is the basis for the headline result: a 122B-parameter model quantized in 64 minutes using 32.5 GB of GPU memory. The paper abstract does not name that model, and the figure is the authors' own measurement.
Against two recent research methods, DuQuant++ and MR-GPTQ, the paper reports lower perplexity, and on Llama-3-8B it matches their best perplexity in 11 times less quantization time. The runtime cost of the added transforms is measured at 2.3% over plain MXFP4 on Qwen3-32B in vLLM.
The repository gives one end-to-end example. On Qwen3.8-27B in vLLM, the W4A4 version raises perplexity from 2.8463 to 2.8994 compared with BF16, while prefill runs 2.33 times faster and decoding 2.58 times faster. The project blog cites a 2.74 times decode speedup for the same model on llama.cpp.
What local GGUF users can use right now
The part most relevant to people running models at home is MACaQ-R. It takes a GGUF file that is already quantized and adjusts its weight codes, while keeping the native formats and block scales, so the result still loads in standard llama.cpp. In the repository's example, a Qwen3.6-35B-A3B file at Q4_K_XL stays at 22.36 GB while its KL divergence from the FP16 model drops from 0.015731 to 0.013367, about 15% closer to the original at the same size.
The framework also produces 2-bit, 3-bit, and 4-bit GGUF files and supports CPU execution through llama.cpp. Beyond text, it covers image, video, audio, and speech models through Diffusers and PyTorch, with examples including FLUX.2 klein 4B, Wan2.2, and Qwen3-TTS.
Caveats before you requantize your model library
The full W4A4 path is not a drop-in file. Workflows that transform activations rely on a llama.cpp runtime bundled with the project, so only MACaQ-R output runs on an unmodified install. Setup expects Linux, Python 3.12, a separate environment for each backend, and a CUDA toolkit for llama.cpp.
The comparisons are also narrower than the headline suggests. The media demos are compared only with simple round-to-nearest rounding, and the blog states that its 3D and audio examples are not speed or accuracy benchmarks. Quantized speech can shift in timing and intonation even with the same seed.
Huawei has its own stake in the result: the repository links W4A4 kernels submitted for its Ascend chips, although running a whole model there still needs integration with an Ascend inference runtime. For everyone else, the practical test is simple. Refine a GGUF you already use, then compare its answers on your own prompts before replacing the original.