How do you build a text-to-SQL agent for SAP without hallucinated queries?
Josue Ramirez
Last updated: August 2026
The first version of our agent answered "show me the top 10 materials by revenue" with a confident, well formatted, completely fictional SQL query. It joined a table called ORDERS to a table called PRODUCTS and summed a column that does not exist anywhere in SAP.
Neither of those tables exists. The real field is VBAK.NETWR, and getting from a material to its revenue means going through VBAP, which the model never mentioned.
The query was syntactically perfect. That is what makes this failure mode dangerous. A broken query throws an error and everyone moves on. A plausible query returns a number, and the number ends up in a board deck.
We stopped trying to make the model better at guessing. The problem was that we were asking it to guess at all.
Why should the model be a compiler, not an inventor?
A raw SAP schema is not something you hand to a language model. Thousands of tables, names like VBAK and VBAP and MARA and EKKO, joins shaped by decades of performance decisions rather than by readability, and business meaning that lives in tribal knowledge rather than in metadata.
So Onibex ASK, short for Agentic Semantic Knowledge, does not point the model at the schema. It points the model at a curated semantic layer, and the model's only job is to map the question onto structures that already exist.
"What is our revenue by material" resolves like this. The word revenue maps to VBAK.NETWR through a declared synonym. The path from the sales order header to its items comes from the join topology declared in the data product, not from the model's intuition. The generated SQL touches only the tables the selected data product allows.
The model still does the hard linguistic work. It just does not get to invent structure. That single constraint is the difference between a demo and something a controller will sign off on.
What does the medallion model look like applied to SAP?
The semantic layer is organized in three layers, borrowed from the medallion pattern and adapted to how SAP actually stores things.
Bronze holds raw SAP tables with their columns and primary keys. VBAK, VBAP, MARA. This layer exists so the system knows what physically exists, nothing more.
Silver merges Bronze tables into coherent business concepts, with the complete join topology and a declared role for every field: measure, dimension, identifier, timestamp. This is where a sales order stops being two tables and becomes one queryable thing.
Gold holds denormalized, pre-aggregated entities so that the most common business questions resolve in a single scan instead of a multi table join.
The routing is simple, and deliberately so. If a Gold entity already covers the question, query it directly. If not, fall back to Silver and plan the joins. Gold covers most of what people actually ask, and that is where the latency budget goes.
This is also the concrete answer to a claim that gets made loosely in this industry: raw tables are not a semantic layer. Bronze is not a semantic layer either. Silver is where meaning starts.
Why three engines instead of one?
We shipped three query modes instead of one, which sounded like over-engineering until we watched how differently people use the same agent.
Flash. One LLM call, about 15 seconds. Free text search over the schema, single shot SQL generation, no join verification. This is the exploration mode, for cheap iteration and for questions where you are going to sanity check the answer yourself anyway.
Smart. Two LLM calls, about 40 seconds. The model selects the data product from a compact catalog, and everything after that selection is deterministic, including join planning. This is the default in production and it carries most of the traffic.
Precise. Three LLM calls, about 60 seconds. It extracts a Semantic Plan IR, runs hybrid kNN and BM25 retrieval with medallion re-ranking, plans joins with Dijkstra, then audits the generated SQL against the allowed table set and retries when the audit fails. This is the mode for numbers that leave the building.
The interesting part is not the latency. It is where the determinism sits.
| Engine | Data product selection | Join planning | SQL audit |
|---|---|---|---|
| Flash | Free text schema search | Not verified | None |
| Smart | Model picks from a compact catalog | Deterministic | None |
| Precise | Deterministic retrieval | Deterministic | Audited against the allowed table set, retried on failure |
Join planning runs Dijkstra over a declared relationship graph. Not because shortest path is glamorous, but because the resulting join is explainable. When someone asks why the query joined those three tables, there is an answer that does not begin with "the model decided."
What happens when the question is ambiguous?
Ambiguity is the second place these systems hallucinate, and it needs its own handling rather than a better prompt.
Level one: the term maps to exactly one thing and resolves silently.
Level two: the term maps to multiple entities across different SAP modules, so the agent asks. "Order" is the honest example. In SD it is a sales order. In MM it is a purchasing document in EKKO. Rather than pick, the agent asks which one you meant.
Level three: the term maps to nothing. The agent says so and points at the administrator who can add it. This is the level that took the most discipline to build, because saying "I do not know that term" feels like a product failure and is actually the whole point.
The dictionary behind this holds canonical field labels, SAP column mappings, synonyms, context clues, and disambiguation hints per module, indexed in OpenSearch for hybrid retrieval and maintained through a configuration app rather than by editing files in production.
Where do the reports come from?
Once the semantic layer exists, generating documents is almost a side effect. The agent can run several queries, write narrative against the actual results, and produce an Excel workbook with the report, the data, and the SQL on separate sheets.
The SQL sheet matters more than it sounds. It means the recipient can see exactly what was asked of the database, which is the difference between a document you trust and a document you hope about.
What makes this work is that artifacts obey the same semantic layer as chat queries. There is no separate reporting model to keep in sync, so a number in a report and the same number in a chat answer cannot drift apart.
What did we learn?
Schema quality is the product. Every measurable improvement in answer quality came from better data products: more synonyms, sharper disambiguation hints, join descriptions written for a reader. The engineering is infrastructure. The semantics are the product.
Determinism is a feature, not a constraint. We expected users to want the model to have more freedom. They wanted the opposite. An auditable join beats a clever one every time the output has someone's name on it.
Three engines is not over-engineering. Mode selection also turned into a diagnostic. If Precise answers correctly and Flash does not, the problem is retrieval. If both fail the same way, the problem is the schema definition.
Governance needs ceremony. Schema changes promote from dev to prod through an explicit flow, because the failure we most wanted to avoid was a well meaning definition change silently breaking a query an executive runs every Monday.
Frequently asked questions
What is Onibex ASK?
Onibex ASK — short for Agentic Semantic Knowledge — is an agentic semantic layer over SAP data products that translates a plain-language question into governed SQL. It maps the question onto declared data products rather than onto the raw SAP schema, so the generated query can only reference tables, fields, and joins a human already approved.
Why can't a language model just read the SAP schema and write the SQL?
Because the schema does not carry business meaning. VBAK.NETWR is net value, but nothing in the table says that this is revenue or that materials connect to it through VBAP. Given thousands of cryptic tables and no semantics, the model fills the gap by inventing, which produces syntactically valid and factually wrong SQL.
How does a semantic layer prevent hallucinated columns and joins?
Three mechanisms. Synonyms resolve business words to real fields; join topology is declared in the data product and planned deterministically with Dijkstra over a relationship graph; generated SQL is audited against the set of tables the selected data product allows, and rejected queries retry rather than ship.
Which engine should you use in production?
Smart, at two LLM calls and about 40 seconds, because it keeps join planning deterministic while letting the model pick the data product. Move to Precise when the numbers will be published or audited. Use Flash for exploration.
Do you need SAP data streaming in place before this works?
You need the data somewhere queryable and reasonably fresh. Onibex One Connect moves business events out of SAP ECC and SAP S/4HANA through Apache Kafka into the destination where these queries run. The semantic layer determines whether the answer is correct. The streaming layer determines how current it is.
Does this approach only apply to SAP?
No. The Bronze, Silver and Gold structure and the governed SQL pattern are not SAP specific, so most of it transfers to any schema that is too large and too cryptic to hand to a model directly. SAP is simply the hardest common case.
The semantic layer specification, the platform code, and the manual are on GitHub, so if you are building something like this on a schema of your own you can start from ours: github.com/Onibex/agentic-semantic-knowledge-ask