What is BitNet b1.58 (1-bit LLM)?

BitNet b1.58 is a language model architecture whose weights are ternary, meaning each one can only take the value -1, 0 or 1, which needs just log2(3) ≈ 1.58 bits per parameter instead of the 16 or 32 bits of a conventional model. The substantial difference from classic quantization is that here the reduced precision is not a compression applied after training to an already full-precision model: the model is trained natively at this precision from the first step, which according to the source paper preserves the model's quality instead of merely compressing it after the fact. It was first described by Ma et al. in "The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits" (Microsoft Research and University of Chinese Academy of Sciences, February 2024, arXiv:2402.17764), and made available through a published model, bitnet-b1.58-2B-4T, and an open source inference framework, bitnet.cpp.

Concrete numbers, not promises

The savings are not theoretical: a 100-billion-parameter BitNet b1.58 model runs on a single CPU, no dedicated GPU needed, at a speed comparable to human reading. The paper also reports that at 3 billion parameters BitNet b1.58 matches an FP16 LLaMA of the same size on perplexity and zero-shot accuracy, with 3.55x less GPU memory and 2.71x higher speed. The advantage grows on CPU inference: the official microsoft/BitNet (bitnet.cpp) framework measures, per the benchmarks published in the repository, a 2.37x to 6.17x speedup on x86 CPUs and a 71.9% to 82.2% energy reduction; on ARM CPUs the speedup ranges from 1.37x to 5.07x with energy reduced 55.4% to 70.0% (the 100-billion-parameter case above runs at 5-7 tokens per second). The published model, microsoft/bitnet-b1.58-2B-4T, is downloadable and verifiable.

It should not be confused with an SLM: the latter cuts costs by reducing parameter count, while BitNet does so by reducing weight precision, an approach orthogonal to model size. The paper itself demonstrates this by scaling from 700 million to 3.9 billion parameters, and bitnet.cpp runs one at 100 billion parameters on CPU: a BitNet model can be small or large depending on what the task needs.

Why it matters for decision-makers

High-volume cloud inference, the kind that pays per GPU call, has a cost that scales linearly with traffic. A model that runs on standard CPUs at usable speed shifts that cost from variable to nearly fixed, and that is the concrete lever for two scenarios: edge or IoT device deployment where a dedicated GPU is not an option, and cutting the cloud bill for applications that currently rent GPU capacity just to serve inference. It remains an early field: the published model at scale is still limited (2B parameters) and the tooling ecosystem around bitnet.cpp is less mature than for established quantization formats, so it deserves a targeted pilot, not adoption as an automatic replacement for existing GPU infrastructure.