The ontology illusion: a buzzword for not talking about data

Getting an ontology today takes one of two shortcuts. Take a data catalog and call it one. Or run an LLM over a pile of documents and out comes a graph with nodes and labels. The result looks right: circles, arrows, a nice drawing. But it's still a drawing that resembles a map: representing something isn't the same as sharing its meaning, and the difference decides who can trust the answer.

The same difference decides whether a project holds up in production or deflates the first time someone checks it seriously.

The problem isn't new, it just gets renamed every year

It used to be called "big data". Then "data lake". Now "ontology". Every cycle repackages the same promise with a different vocabulary: bring your data together and intelligence will follow. The bottleneck hasn't moved an inch: dirty, duplicated, ownerless data with no shared definitions (that's data governance, precisely). Gartner on data governance predicts that 80% of data governance initiatives will fail by 2027, often because they start without a real business crisis behind them. And per the 2025 DATAVERSITY data management trends survey, only 15% of organizations report mature governance today. For that 15%, ontology makes sense. For the remaining 85%, it arrives before the governance it would need.

Representation is not meaning

"Ontology" comes from the Greek ón, óntos (being) and lógos (study): literally, "the study of being". The term was born in philosophy, centuries before any AI lab. Calling an afternoon's worth of auto-generated graph by that name misuses a word carrying all that weight.

There's a distinction the knowledge-representation community understood clearly decades ago (the classic definition is Tom Gruber's, from 1993) and is now losing in the rush to call any graph an "ontology": an ontology, in the proper sense, is a shared conceptualization of a domain, not a graph or a schema: it's the outcome of a process, often slow and negotiated, where different people agree on what their company's words mean and which distinctions actually matter.

A language model that reads thousands of documents and extracts entities and relations doesn't do that work. It produces a statistical projection of the text: plausible, often useful, but negotiated by nobody. Calling it an ontology is like calling the first draft of a machine translation an "official translation": it can be a great starting point, but selling it as the finished product is an illusion.

The real hidden cost: it's not the concepts, it's the data underneath

An ontology, even a well-built one, doesn't survive a minute if the data underneath is a mess. If the "customer" field is a numeric ID in one system, a tax code in another, and a hand-typed name full of typos in a third, no amount of "semantics" stitches that gap back together.

The ontology describes relations between clean concepts; if the underlying data is dirty, duplicated or unreconciled, the ontology inherits the same mess, just with a fancier vocabulary on top.

Gartner on AI-ready data predicts that, through the end of 2026, 60% of AI projects lacking AI-ready data will be abandoned for exactly that reason: lack of data, not lack of an ontology. The problem sits upstream, in the foundations, and an ontology that arrives before the data is in order only pushes it further down the line.

A more recent Gartner survey looks at a narrower, different population than the studies above, I&O leaders: of 782 surveyed between November and December 2025, only 28% of AI use cases actually meet ROI expectations, 20% fail outright, and 38% of those who hit a setback point to poor data quality or limited data availability as the direct cause.

The stakes rise when the ontology, beyond making a chatbot talk well, has to make an AI agent act: book something, invoice something, update a contract. Text-similarity search, on its own, returns plausible text, with no guarantee it matches the authoritative state of the systems.

An agent acting on your systems can't rely on data that might be stale, contradictory, or from the wrong customer. The ontology is supposed to be the layer that grounds the agent in verified, structured truth. Here too, that only holds if the structured truth is actually clean at the source, exactly like it does for multiple AI agents that need to coordinate with each other.

ORAG: when even the technique admits the problem

Research already has a name for this attempted fix: ORAG, Ontology-Guided Retrieval-Augmented Generation. Originally developed to cut hallucinations in entity-typing tasks, the idea is to use an ontology's structure to guide what a system retrieves and how it interprets it, instead of relying on text similarity alone. It's a real improvement over flat RAG: constraining retrieval to a category structure cuts down the coarsest errors.

The paper measures the gain over flat RAG; the damage a bad ontology does is something neither of these works set out to measure. What it takes for granted instead, and states plainly, is an assumption: ORAG is only as good as the ontology guiding it, and the ontology is only as good as the data it was built from. A shallow ontology, rushed together over dirty data, doesn't make RAG more reliable: it just makes it confidently wrong. An incorrect map sometimes guides a system worse than no map at all, because the answer looks more grounded than it is.

Heading in the same direction by a different road, Microsoft Research has published OG-RAG, Ontology-Grounded RAG, built on hypergraphs instead of plain text-similarity retrieval. ORAG and OG-RAG are two independent pieces of work, with different authors and different tasks, alike only in the acronym. And OG-RAG is the one that says the most useful thing for anyone who has to make the call, and it says it in its own limitations section: a domain ontology is not always available, nor always comprehensive enough. Its results come from an ontology verified by crop scientists over documents prepared by agronomists, so the human cost sits in preparing the experiment. That is the line no vendor puts in the quote when selling you ontology-guided retrieval. OG-RAG's stated limitations don't say that "pure" RAG (vectors only, similarity only) is disappearing. In my reading, it's increasingly becoming an umbrella term under which agentic and ontology/graph-grounded variants coexist (GraphRAG among them). Naive RAG, with no structure or governance behind it, is increasingly being complemented or replaced by versions that start from a model of the domain.

Why "almost nobody has an ontology" (and the big vendors first)

Look closely at what major vendors sell today under the "ontology" label and a pattern jumps out: most of what gets called that is, strictly speaking, a governed property graph, or a business glossary with typed relations. Useful, often well built, but without the formal semantics the term promises: axioms, description logic, an explicit stance on what the system can and can't infer.

Even products bootstrapped from existing business-intelligence models with no-code tools, or engines that define objects, links and actions but no formal description logic, fall into this category. Even when a property graph follows a shared structural standard (like ISO/IEC 39075, the GQL query language), it still lacks the formal semantics the word "ontology" promises: verifiable axioms and inference rules. An OWL ontology, by construction, has that semantics: in its OWL 2 DL profile it's a decidable subset of first-order logic (the OWL Full profile isn't).

When an ontology governs an AI agent's autonomous decisions, that difference determines whether the system reasons on verifiable rules, or just improvises plausibly, with no guarantee on what it can and can't conclude. That said: by that same criterion, shared and negotiated agreement, a property graph built this way can qualify as an ontology. What it lacks isn't legitimacy. It's the formal inference guarantee, and that missing piece is exactly why, in the fuller sense of the term, almost nobody actually has one.

Ontology is not a fancier dictionary

There's an even more common mistake: confusing an ontology with a slightly fancier company glossary. A shared dictionary ("customer" means this, "supplier" means that) is already an important step, and on its own resolves a lot of misalignment between teams. But an ontology, in the stricter sense of the term, is the formalization of the fundamental categories through which a domain makes sense, not a list of types and labels optimized for machine learning. An OWL ontology, for instance, also encodes inference rules: if "an active contract implies a current customer" is written once, the system deduces on its own consequences nobody entered by hand. Without clean, consistent data to start from, those rules produce inferences nobody can check. Which is why inference has to stay separate from validation: OWL infers, and checking that the data has the declared shape is the job of SHACL.

In my experience on these projects, most companies ignore that domain knowledge almost always comes from an agreement between people across different teams. A complex domain needs multiple legitimate viewpoints that stay coherent with each other.

Palantir knows this, and doesn't skip the step

Even the systems that made the word "ontology" popular in the enterprise, like Palantir Foundry (on why Palantir as a company is polarizing, I cover that separately), never start from the graph. They start from integrating and cleaning the source data, and only then build the ontological layer on top. It's a detail that disappears in the demo but stays ninety percent of the actual work. If a vendor offers you "your ontology" before you've put your source systems in order, they're selling you the shortcut not even their own reference example would take.

What to actually do

The sequence is always the same, whatever the platform: data first, ontology second. It's how I work. Concretely:

• Fix the source before you model the domain. Reconciliation, deduplication, data ownership, written and shared definitions: without this, any semantic layer on top inherits the same reconciliation errors.
• Start small and verifiable. A handful of entities that genuinely matter to the business. An ontology covering three concepts correctly beats one covering three hundred that falls apart under scrutiny.
• Treat an LLM's output as a draft, not a result. A graph auto-generated from text is a great starting point for a discussion among people who know the domain, and that discussion still has to happen.
• Accept that multiple local ontologies will coexist. The realistic goal is the ability to compare and align different representations when it matters.

In short

"Ontology" has become a word spent to sell before the actual problem it's meant to solve gets fixed: dirty, unreconciled data with no shared definitions. A graph generated by a language model is a plausible representation, not an agreement on meaning, and the difference matters the moment someone needs to trust the answer. The right sequence has stayed boringly the same all along: the data foundation first, the ontological layer that reasons over it second.