The ontology illusion: why data comes first

Getting an ontology today takes one of two shortcuts. Take a data catalog and call it one. Or run an LLM over a pile of documents and out comes a graph with nodes and labels. The result looks right: circles, arrows, a nice drawing. But a drawing that resembles a map isn't a map: representing something isn't the same as sharing its meaning, and the difference decides who can trust the answer.

It isn't an academic nicety: it decides whether a project holds up in production or deflates the first time someone checks it seriously.

The problem isn't new, it just gets renamed every year

It used to be called "big data". Then "data lake". Now "ontology". Every cycle repackages the same promise with a different vocabulary: bring your data together and intelligence will follow. The bottleneck hasn't moved an inch: dirty, duplicated, ownerless data with no shared definitions (that's data governance, precisely). Gartner on data governance predicts that 80% of data governance initiatives will fail by 2027, often because they start without a real business crisis behind them. And per the 2025 DATAVERSITY data management trends survey, only 15% of organizations report mature governance today. For that 15%, ontology makes sense. For the remaining 85%, it's paint over an unsolved problem.

Representation is not meaning

Worth starting from the word itself. "Ontology" comes from the Greek ón, óntos (being) and lógos (study): literally, "the study of being". It isn't a term born in an AI lab, but in philosophy, centuries ago. Calling an afternoon's worth of auto-generated graph by that name isn't just imprecise: it's misusing a word carrying all that weight.

There's a distinction the knowledge-representation community understood clearly decades ago (the classic definition is Tom Gruber's, from 1993) and is now losing in the rush to call any graph an "ontology": an ontology, in the proper sense, is a shared conceptualization of a domain. It isn't a graph. It isn't a schema. It's the outcome of a process, often slow and negotiated, where different people agree on what their company's words mean and which distinctions actually matter.

A language model that reads thousands of documents and extracts entities and relations doesn't do that work. It produces a statistical projection of the text: plausible, often useful, but negotiated by nobody. It's a representation, not an agreement. Calling it an ontology is like calling the first draft of a machine translation an "official translation": it can be a great starting point, but selling it as the finished product is an illusion, not a minor overstatement.

The real hidden cost: it's not the concepts, it's the data underneath

Here the argument turns operational, and it is the part that decides whether AI holds up in production: an ontology, even a well-built one, doesn't survive a minute if the data underneath is a mess. If the "customer" field is a numeric ID in one system, a tax code in another, and a hand-typed name full of typos in a third, no amount of "semantics" stitches that gap back together.

The ontology describes relations between clean concepts; if the underlying data is dirty, duplicated or unreconciled, the ontology inherits the same mess, just with a fancier vocabulary on top.

Gartner on AI-ready data predicts that, through the end of 2026, 60% of AI projects lacking AI-ready data will be abandoned for exactly that reason. Not for lack of an ontology. For lack of foundations. An ontology that arrives before the data is in order accelerates nothing: it moves the problem further down the line.

A more recent Gartner survey looks at a narrower, different population than the studies above, I&O leaders: of 782 surveyed between November and December 2025, only 28% of AI use cases actually meet ROI expectations, 20% fail outright, and 38% of those who hit a setback point to poor data quality or limited data availability as the direct cause.

The stakes rise, too, when the ontology isn't just there to make a chatbot talk well, but to make an AI agent act: book something, invoice something, update a contract. Text-similarity search, on its own, returns plausible text, not authoritative state.

An agent acting on your systems can't rely on data that might be stale, contradictory, or from the wrong customer. The ontology is supposed to be the layer that grounds the agent in verified, structured truth, not in text that merely resembles the right answer. Here too, that only holds if the structured truth is actually clean at the source, exactly like it does for multiple AI agents that need to coordinate with each other.

ORAG: when even the technique admits the problem

Research already has a name for this attempted fix: ORAG, Ontology-Guided Retrieval-Augmented Generation. Originally developed to cut hallucinations in entity-typing tasks, the idea is to use an ontology's structure to guide what a system retrieves and how it interprets it, instead of relying on text similarity alone. It's a real improvement over flat RAG: constraining retrieval to a category structure cuts down the coarsest errors.

Watch what the paper actually proves, and what it just assumes: it measures the gain over flat RAG, not the damage a bad ontology does, which neither of these works set out to measure. The assumption is stated plainly all the same, and it is exactly this article's argument: ORAG is only as good as the ontology guiding it, and the ontology is only as good as the data it was built from. A shallow ontology, rushed together over dirty data, doesn't make RAG more reliable: it just makes it confidently wrong. Guiding a system with an incorrect map isn't better than leaving it without one; sometimes it's worse, because the answer looks more grounded than it is. The technique evolves, the bottleneck stays the same: data first, then the structure that reasons over it.

Heading in the same direction by a different road, Microsoft Research has published OG-RAG, Ontology-Grounded RAG, built on hypergraphs instead of plain text-similarity retrieval. It is not a variant of ORAG: they are two independent pieces of work, with different authors and different tasks, alike only in the acronym. And OG-RAG is the one that says the most useful thing for anyone who has to make the call, and it says it in its own limitations section: a domain ontology is not always available, nor always comprehensive enough. Its results come from an ontology verified by crop scientists over documents prepared by agronomists, so the human cost sits in preparing the experiment. That is the line no vendor puts in the quote when selling you ontology-guided retrieval. Careful not to read more into that than it says: "pure" RAG (vectors only, similarity only) isn't disappearing. In my reading, it's increasingly becoming an umbrella term under which agentic and ontology/graph-grounded variants coexist (GraphRAG among them), not one monolithic technique. It's not that "RAG is dead"; it's that naive RAG, with no structure or governance behind it, is increasingly being complemented or replaced by versions that start from a model of the domain. Which, again, brings everything back to the starting point: what model, built on what data.

Why "almost nobody has an ontology" (and the big vendors first)

Look closely at what major vendors sell today under the "ontology" label and a pattern jumps out: most of what gets called that is, strictly speaking, a governed property graph, or a business glossary with typed relations. Useful, often well built, but without the formal semantics the term promises: axioms, description logic, an explicit stance on what the system can and can't infer.

Even products bootstrapped from existing business-intelligence models with no-code tools, or engines that define objects, links and actions but no formal description logic, fall into this category. The difference isn't cosmetic: even when a property graph follows a shared structural standard (like ISO/IEC 39075, the GQL query language), it still lacks the formal semantics the word "ontology" promises: verifiable axioms and inference rules. An OWL ontology, by construction, has that semantics: in its OWL 2 DL profile it's a decidable subset of first-order logic (the OWL Full profile isn't).

It isn't a philosopher's footnote. When an ontology governs an AI agent's autonomous decisions, that difference determines whether the system reasons on verifiable rules, or just improvises plausibly, with no guarantee on what it can and can't conclude. That said: by that same criterion, shared and negotiated agreement, a property graph built this way can qualify as an ontology. What it lacks isn't legitimacy. It's the formal inference guarantee, and that missing piece is exactly why, in the fuller sense of the term, almost nobody actually has one.

Ontology is not a fancier dictionary

There's an even more common mistake: confusing an ontology with a slightly fancier company glossary. A shared dictionary ("customer" means this, "supplier" means that) is already an important step, and on its own resolves a lot of misalignment between teams. But an ontology, in the stricter sense of the term, is the formalization of the fundamental categories through which a domain makes sense, not a list of types and labels optimized for machine learning. An OWL ontology, for instance, also encodes inference rules: if "an active contract implies a current customer" is written once, the system deduces on its own consequences nobody entered by hand. Without clean, consistent data to start from, those rules don't deduce anything sensible: they produce inferences nobody can check. Which is why inference has to stay separate from validation: checking that the data has the declared shape is the job of SHACL, not of OWL.

There's also a point we see most enterprise projects ignore in our own work: domain knowledge almost always comes from an agreement between people across different teams, not from a single isolated observation. A complex domain needs multiple legitimate viewpoints that stay coherent with each other, not one flattened schema pretending to fit everyone.

Palantir knows this, and doesn't skip the step

Worth noting: even the systems that made the word "ontology" popular in the enterprise, like Palantir Foundry (on why Palantir as a company is polarizing, I cover that separately), never start from the graph. They start from integrating and cleaning the source data, and only then build the ontological layer on top. It's a detail that disappears in the demo but stays ninety percent of the actual work. If a vendor offers you "your ontology" before you've put your source systems in order, they're selling you the shortcut not even their own reference example would take.

What to actually do

The sequence that works isn't glamorous, but it's the right one: data first, ontology second. It's how I work. Concretely:

• Fix the source before you model the domain. Reconciliation, deduplication, data ownership, written and shared definitions: without this, any semantic layer on top inherits the same reconciliation errors.
• Start small and verifiable. A handful of entities that genuinely matter to the business, not a giant schema generated in an afternoon. An ontology covering three concepts correctly beats one covering three hundred that falls apart under scrutiny.
• Treat an LLM's output as a draft, not a result. A graph auto-generated from text is a great starting point for a discussion among people who know the domain, not a substitute for that discussion.
• Accept that multiple local ontologies will coexist. The realistic goal isn't a single company-wide truth, but the ability to compare and align different representations when it matters, not pretending they don't exist.

In short

"Ontology" has become a word spent to sell before the actual problem it's meant to solve gets fixed: dirty, unreconciled data with no shared definitions. A graph generated by a language model is a plausible representation, not an agreement on meaning, and the difference matters the moment someone needs to trust the answer. The right sequence has stayed boringly the same all along: the data foundation first, the ontological layer that reasons over it second. Whoever skips the first step doesn't have an ontology: they have a graph nobody has validated.