What is data debt?

Data debt is the debt that accumulates not in code but in the data itself: undocumented dependencies between datasets, pipelines nobody remembers the reason for, the same metric defined differently across departments, columns nobody dares delete because it is unclear who still reads them, copies of the same data in different systems that slowly drift apart. It is distinct from technical debt: the latter lives in the code and in architectural choices, and a review uncovers it by reading the source; data debt lives outside the code, in tables, joins and definitions, which is why it almost never surfaces in a code review. It is the silent reason an AI project stalls right at the data preparation stage: the team only then discovers nobody agrees on what "active customer" or "net revenue" means.

How it accumulates

The accumulation mechanism is almost always the same: a shortcut is taken to ship a report or a model quickly, and the shortcut outlives the reason it was created for. An analyst copies a table into another schema to speed up a query, and five years later that copy still feeds an executive dashboard with nobody sure whether it is still in sync with the original. A data scientist recomputes a metric with logic slightly different from finance's, and the two versions coexist for years until an audit forces them to collide. In 2015, Sculley and colleagues at Google, in a paper that has become a field reference, showed that in machine learning systems the most insidious debt is not in the model but in the data dependencies feeding it: unmaintained inputs, features nobody dares remove, glue code holding fragile pipelines together.

An enterprise example

An industrial group with several plants has three different definitions of "machine downtime": one in the maintenance system, one in the commercial data warehouse, one in the controller's Excel sheet. Each made sense as a choice at the time it was created, but nobody ever reconciled them. When the company tries to build a predictive maintenance model, the team discovers the three figures cannot be reconciled without weeks of archaeological work on pipelines written by people no longer at the company. The AI project does not stall on an algorithm problem: it stalls because nobody can say with certainty what "a downtime event" means in the available data.

How you pay it back

Data debt is not repaid with a one-off intervention, but with the same practices the rest of the glossary already covers: measuring the dimensions of data quality instead of assuming them, tracking where data has been with data lineage, monitoring pipelines continuously with data observability, and formalizing what a system promises to whoever consumes its data with a data contract. The pragmatic first step is almost always the same: inventory who reads each table and each column before touching it, because data debt hides exactly in not knowing who depends on what.