What is AI accelerator lock-in?

AI accelerator lock-in is the cost of moving an inference workload from one family of accelerators to another: from one vendor's GPU to another's, or to a dedicated chip designed by whoever sells you the cloud. It sits one level below vendor lock-in, which measures the exit cost on data, business logic and skills: here the constraint is not in the contract, it is in code optimized for a compiler. The confusion comes from the fact that model weights really are portable, and can be copied elsewhere without rewriting anything. What does not move is the system serving them: the kernels written for one platform, the toolchain that compiles them, the runtime libraries and the cloud instance type the whole configuration was measured and accepted on. The model travels, the optimizations stay where they were born.

What actually ties a workload to an accelerator

Four layers do the tying, and none of them appears in a contract. The kernels, the routines that run attention and multiplications on the chip: hand written for one platform, or generated by a language such as Triton, which its own documentation describes as "a language and compiler for parallel programming" for writing kernels that run "at maximal throughput on modern GPU hardware". The compilation toolchain, which turns a graph into an engine already specialized for that chip and that numeric precision. The runtime libraries the engine calls for every operation. The instance typeeverything was tuned on, with its memory and its bandwidth. Abstraction layers exist and deserve naming without promising they solve it. ONNX Runtime exposes the same API across dozens of execution providers, stating that it abstracts "the details of the hardware specific libraries that are essential to optimize the execution": the API is one, the library underneath is not. Source translators are even more explicit, and are worth reading literally rather than trusting the headline: the HIPIFY documentation warns that it cannot translate libraries with no equivalent and that code optimized for performance on one GPU "might require additional rework" to optimize performance on the other. Translating the syntax is the easy part, recovering the performance is the project.

An enterprise example

A company puts contract document classification into production on its own inference service: quantized model, engine compiled for the chosen accelerator, batching tuned to the latency the business accepts. After a year it signs a multi-year spend commitment in exchange for a discount. Halfway through, a different accelerator shows up at a lower cost per token. The bottleneck is not contractual: it is that the engine has to be recompiled, the quantization re-verified against real accuracy, latencies re-measured and the whole output quality accepted again by legal. The honest comparison is at system level, not chip level, which is why MLPerf Inference publishes results per full hardware and software configuration rather than per accelerator in the abstract.

Why it matters for decision makers

It matters because it sets your negotiating leverage. A workload that runs only where it was born has no credible alternative at renewal, and the vendor knows it before you do. Three questions belong in the room before signing, and they are technical: which parts of our serving stack are specific to this accelerator, how many weeks it costs to bring the same service up on another platform at equal latency and accuracy and who pays for that work if we switch. The answer gets written down, not promised. The rest of the bill is governed elsewhere: cost per token belongs to AI tokenomics, and bringing workloads back in house is cloud repatriation. Here the only thing to preserve is the option: keep an exit route you have measured at least once, instead of one you have assumed.