What is Apple's MLX?

MLX is an open-source array and machine learning framework released by Apple in December 2023 under the MIT license, designed around the architecture of the M-series chips rather than as a generic port. The difference from PyTorch or TensorFlow is not in the API, which MLX deliberately mirrors to make the transition familiar, but in a structural assumption: on a Mac with an M-series chip, the CPU and GPU do not have separate memory pools, they share the same physical RAM. MLX builds the framework on top of that property, with arrays that live in shared memory and move from a CPU operation to a GPU one with no explicit copy, and with lazy computation that materializes an array only when its value is actually needed. The project covers bindings for Python, Swift, C and C++, with higher-level packages for neural networks and optimizers modeled on PyTorch's conventions.

Why unified memory changes the sizing math

On a machine with a discrete GPU, the constraint is the video card's own memory, VRAM, separate from system RAM: a model that does not fit in VRAM has to be split, offloaded, or dropped, and the question driving the purchase is always "how much VRAM do I have." With unified memory that question changes shape: there is no copy of data between system memory and card memory, because it is the same memory, so the limit becomes the machine's total RAM. A laptop with a large amount of unified memory can therefore load models that a comparable-tier consumer video card could not fit at all. The other side of the coin is that memory bandwidth and compute power remain those of a chip built for a laptop, not those of a dedicated accelerator: more model fits, but it does not run any faster. This relationship between available memory and generation speed is the same calculation covered in general in the entry on running an LLM locally; here only the side of the equation that unified memory shifts changes, not the arithmetic itself.

An ecosystem has grown around MLX: many open model families already circulate in an MLX-converted variant, and the most aggressive quantizations, the ones that push footprint below 16 GB, are often the only practical way to fit a usably sized model into a laptop's memory. A compression technique built on top of MLX, aimed at the attention cache rather than the weights, is described in the entry on TurboQuant.

An enterprise example

A company with a fleet of Macs already issued to its developers or analysts has, without realizing it, hardware it has already paid for that can run local inference on documents that internal policy forbids from leaving the company network, without buying a single dedicated GPU. A model converted for MLX runs directly on those laptops, and a prototype for classification or extraction over sensitive data can start with hardware the team already holds. The limit needs the same honesty: it works for one person's job and for a prototype, not for serving a team under concurrent requests, because MLX does not solve concurrency scaling any more than any other runtime built for a single machine does.

Why it matters for decision makers

The point the enthusiasm for "free AI on the company Mac" tends to skip is that unified memory shifts the bottleneck, it does not remove it: a company planning hardware purchases for local inference on MLX needs to size on the machine's total RAM, not on the video card, but it also has to accept that response speed stays that of a laptop chip, not that of a data-center accelerator. It is a real lever for lowering the cost of entry into experimentation, especially where the fleet of machines already exists, not a shortcut for avoiding infrastructure investment altogether once the use stops being individual.

Sources: the official ml-explore/mlx repository and the project page on Apple Open Source.