Data design · 2026-08-24 · Go Kyono

Ontology design in practice: the knowledge structure that decides retrieval accuracy

Almost every conversation that starts with we deployed RAG and it does not answer well ends in the same place. The search algorithm is fine. The model is fine. The knowledge being searched was never structured, and no amount of model quality repairs that.

Short answers first

If you only came for one definition

What is ontology design?: Writing down, in a form a machine can act on, the concepts a business uses and the relationships and constraints between them. Deciding once and explicitly what product, part and model number refer to, and how they connect, so the meaning stops drifting from person to person and system to system.
How is it different from a knowledge graph?: The ontology is the specification; the knowledge graph is the instance built to it. The ontology says a company employs a person. The graph says TechJapan employs Kyono. Build the graph without the specification and contradictions accumulate in proportion to how much data you load.
What does it have to do with retrieval accuracy?: Retrieval depends on the distance between the words in the question and the words in the documents. When the same thing is called something different in each department, the embeddings treat those as different things and the right passage never enters the context. Controlled terminology and defined relations let retrieval reach the same object through different words.
Where do you start?: With a terminology inventory for one scoped process — never a company-wide dictionary. Pull the nouns out of that process’s documents and bind synonyms and abbreviations to one canonical name. That alone removes the largest single cause of retrieval misses. Hierarchies and constraints can wait.

What an ontology actually is

An ontology is an explicit specification of the concepts in a domain, together with the relationships and constraints between them. The word comes from philosophy; in information science it means something narrower and much more practical — an explicit specification of a conceptualization, written so that knowledge can be shared and reused.

That sounds abstract until you write one. For a manufacturer, an ontology is a set of decisions like these:

Concept definitions. What exactly counts as a product, a part, a process step, a machine, a defect.
Terminology control. Are item number, model number and part number the same thing, or three different things that happen to look alike?
Relation definitions. A product is composed of parts. A process step uses a machine.
Constraints. A defect report attaches to exactly one process step, never zero and never two.

The whole exercise is writing down what everyone already tacitly agrees on. The knowledge that is connected inside an experienced employee’s head gets transcribed into a form a machine can traverse. That is unglamorous, and it is finite, and it is the step almost everybody skips.

Why this became urgent when LLMs arrived

Retrieval-augmented generation searches for documents related to a question and has the model answer from them. In that arrangement, if retrieval does not surface the right document, nothing downstream can recover. A more capable model produces a more fluent wrong answer.

Three patterns break retrieval, and we see all three repeatedly.

2.1 Terminology was never unified

The same organization is a customer in one document, an account in another, a counterparty in a third. Embedding vectors absorb some of this. They do not absorb in-house abbreviations, part-number formatting drift, or a code that changed meaning when a product line was renamed. If the asker says account and the document says customer, the similarity score falls by exactly enough to drop the passage below the cut.

2.2 Relationships between concepts are undefined

To answer which process step produced this defect on this product, you have to follow a chain: product to part to process step to defect. If that chain is not written anywhere, vector search cannot follow it. It returns documents that resemble the question, which is a fundamentally weaker operation, and it does so with no signal that it has done something different from what you asked.

2.3 Departments have forked the meaning

Two systems each have a field called status. They mean different things. Each department is locally correct and has been for years. Indexed together, they produce answers that are wrong in a way that survives review, because every individual document is accurate.

Five steps that fit inside a real project

  1  INVENTORY     what exists, and where it lives 
      |                                            
      v                                            
  2  TERMINOLOGY   one term per concept. synonyms  
      |                mapped, homonyms split apart
      v                                            
  3  STRUCTURE     classes, and the relations      
      |                between them                
      v                                            
  4  CONSTRAINTS   what must hold. what cannot     
      |                co-exist                    
      v                                            
  5  MEASURED      build the graph, then score it  
                     against a fixed gold set.     
                     improvement becomes a number

Step 1 — Inventory

What knowledge exists, where it lives, and who maintains it. Scope this to one process. A company-wide inventory is a project that ends in a binder, and the binder is how these efforts die.

Step 2 — Terminology control

Extract the nouns from that process’s documents. Bind synonyms and abbreviations to one canonical name; expand abbreviations; split homonyms that were quietly sharing a word. This is the highest-yield step and the least technical one, and it is normally enough on its own to move retrieval measurably.

Step 3 — Structure

Classes, and the relations between them. Keep the relation set small and named after what the business actually says: is composed of, uses, supersedes, is reported against. A relation nobody in the business would say out loud is a relation nobody will maintain.

Step 4 — Constraints

What must hold, and what cannot co-exist. Constraints are what make the graph checkable: they turn a data-quality problem from an argument into a failing test.

Step 5 — Build it, then measure it

Load the graph, then fix a gold set: representative questions with known-correct passages. Score retrieval against it before and after. This is the step that converts the whole exercise from taxonomy into engineering, because it produces a number that either moved or did not.

Without step 5 you cannot tell the difference between an ontology that helped and an ontology that was merely tidy. We have seen both, and they look identical in a slide deck.

Three things worth knowing before you start

Chunking is a separate problem, and it is downstream

How documents are split into indexable units interacts with everything above. Our measured comparison is in A comparative study of text chunking methods. Fixing chunking while terminology is still inconsistent produces a well-cut index of ambiguous text.

The graph is not the deliverable. The maintenance process is

Vocabulary drifts because businesses change. An ontology with no owner is accurate on the day it ships and wrong within a year. Naming who maintains it, and how a new term gets added, matters more than the initial modeling.

Structure beats scale, and it keeps beating it

A larger context window lets you put more ambiguous text in front of the model. It does not make the text less ambiguous. Structural work compounds; model upgrades reset.

Bring the questions that are failing.

A list of twenty real questions your system gets wrong is a better starting point than a requirements document. The first consultation is free.

Start a conversation The service page