Data

Why ERP data breaks the textbook semantic layer

Most writing on semantic layers assumes the modelling problem upstream is already solved. ERP data is where that assumption falls over, and where the reports go wrong without a single word being used loosely.

Written for CFO COO CIO

The short versionAbout a minute

This is part two of two. Part one covers what a semantic layer is, and why your chart of accounts is already one.

Somewhere in your reporting there may well be a number several times too large that nobody has noticed, because it foots, it ties to the ledger, and it looks entirely reasonable. It got that way without anyone using a word loosely.

That’s the problem this piece is about. Part one made the case that a semantic layer is the place a measure gets defined once, so every report inherits the same answer rather than deriving its own, and that the closest thing most businesses already own is the chart of accounts over the ledger. That’s a good working definition. It’s also where most explanations stop, and where ERP data starts causing trouble. A chart of accounts settles ambiguous words. Two of the more expensive failure modes in ERP reporting involve no ambiguous words at all.

Why does the report multiply?

A report can carry perfectly sensible calculations, with every measure defined exactly as the business agreed, and still return a number several times too large. Not because anyone used a word loosely, but because the records underneath it exist at different levels of detail.

The tables and ledgers differ by system, but the pattern is common to operational software: related transactions get stored at different levels of detail, because they were recorded for different operational reasons. Business Central is the cleanest example, so it’s the one to use here. In Microsoft Dynamics 365 Business Central the quantity of an item movement and its cost live in separate ledgers, and one quantity entry can carry several cost entries: an initial cost, an expected cost, a revaluation, a later adjustment once the true cost is known. Join the two so a report can show quantity and cost together, and each quantity repeats once per cost entry. The total multiplies, and the report looks entirely reasonable.

No definition of a measure prevents that. The relationship between the two tables was modelled wrongly, and everything built on top inherits the error.

That’s a grain problem. Grain is the level of detail the records sit at, and the model has to preserve it, along with the relationships between tables, so that a sensible question produces a sensible answer. It gets less attention than defining measures does, and it’s the reason a warehouse can land both tables perfectly, reconciled to the source, and still let every report built over them be wrong.

Syspro reaches a related problem from the other side. The costing method is set per warehouse, so one stock code can be valued at standard cost in one warehouse and at average cost in another. Both figures are correct. A report that adds them together is summing two different kinds of number, and nothing in the result says so.

Why the report multiplies. One movement, three cost entries. Joined without accounting for that, the movement comes back once per entry. The calculation isn’t wrong. The relationship is.
Nobody used a word loosely. The report is simply multiplying.

Why did last October change?

The second problem is time, and it’s the one that erodes trust quietly.

A chart of accounts describes the present, and so does ERP master data. Item category, customer territory, sales rep and standard cost all get overwritten in place. Recategorise twelve items in February and every historical report that joins transactions to master data restates itself, so last October’s product mix reads differently than it did in October. Retroactive cost adjustment does the same to a closed period, correctly, after you have already presented it.

Both versions are legitimate questions. What did we report at the time, and what does it look like on today’s structure? Finance usually wants the first and the commercial team usually wants the second, and neither is wrong.

Keeping the state as it was is warehouse work rather than model work, and it doesn’t happen simply because a warehouse exists. It has to be designed in, by capturing master data as it changes rather than overwriting it. Getting one of the two answers by accident, and not knowing which one you got, is the problem.

What changes when software starts answering?

For as long as a person sat in the middle of this, the gap was survivable. They know to be careful at cut-off, when goods that left the dock on the 31st get invoiced on the 2nd and operations and finance count them in different months. They know the operations meeting wants ship date and the board pack wants invoice date. They will ring somebody when a number looks odd. That hedging is invisible, unpaid, and largely the reason nothing has gone badly wrong yet.

Point an assistant at the same data and the checkpoint isn’t there. Ask what margin was last quarter and it will answer. It has no way of knowing that four candidate costs exist, or that finance and the commercial team have been using different ones for years, so it works with what it has and presents the result with the same composure it brings to everything else. The definition was imprecise the whole time. It got expensive when something started answering without asking.

This is also why the useful question isn’t whether to use an assistant. Connectivity is getting cheaper and more standardised, and the direction of travel is toward agents reaching operational data directly. What doesn’t commoditise on that path is knowing what the data means. The same assistant is considerably more useful pointed at a governed model than at raw tables, which is an argument for having the definitions in a form software can read before the software arrives rather than after.

The stakes also rise as these systems move from answering to acting. A wrong answer in a report is embarrassing. A wrong answer that triggers a purchase order or a credit hold is a decision.

There’s a governance point alongside this that finance readers tend to raise before IT does. If the definitions live in the model, the access rules can live there too, so the same restriction applies whether somebody opens a dashboard, refreshes an Excel pivot or asks a question in plain language. That holds for anything reading through the model. Somebody querying the warehouse directly sits outside that, which matters when you design the access.

When you don't need one, and what it won't do

Two situations where this is premature and worth naming. If you are mid-implementation on an ERP still being reconfigured week to week, you would be modelling something that won’t exist in three months. And if you are testing whether an idea has legs, an export and a couple of hours will tell you faster than any model.

What won't it decide for you?

The more important limitation is what a semantic layer doesn’t do even once you have one, because it’s the thing most likely to stall a project halfway. It does not adjudicate. Where three teams quote three fill rates, all three are usually defensible. Line fill, order fill and case fill measure genuinely different things and different people need different ones. On-time against the requested date and on-time against the confirmed date are both real measures of something. A model can hold both, label them, and make it obvious which one is on the slide. What it can’t do is decide which one the business runs on. That decision is yours, it takes a meeting, and any vendor implying the software makes it for you is setting up a disappointment.

Nor does it make your numbers correct. It makes them consistent and traceable, which is narrower and more useful. A definition everybody inherits can still be the wrong definition. The difference is that it becomes one thing you can find, argue about and change, rather than five things nobody can compare.

Where Zap fits

This is the problem Zap has spent more than fifteen years on, and specifically the ERP half of it.

Some of this work is genuinely the same at every company running the same system and version. Which tables carry the transactions, how the ledgers relate, where the grain sits, how the dates diverge. That part can be built in advance rather than rediscovered per customer, and it is: pre-built models for Sage 100, Sage 300, Sage X3, Sage Intacct, SAP Business One, Syspro, Microsoft Dynamics 365 Business Central and Finance and Operations, and Xero.

The rest is configured, and that part is read from your system rather than shipped with it. Your fiscal calendar, your dimensions, your chart, what your own code values mean. Pre-built is a speed claim, not a we-decide-for-you claim. Each source you add extends the model, and the definitions your business argues about stay yours to set.

Underneath, the warehouse is built to keep the history the ERP overwrites. Over the top, the model carries the calculations and hierarchies, and dimensional security travels with them, so the same rules apply to a dashboard, an Excel pivot and a question asked in plain language.

On AI we would rather be precise than exciting. Zap AI answers from your governed model, applies the same dimensional security, and links every number back to its source report. What is available today, and what is still to come, is set out on our AI innovation roadmap, and we would rather point you at that than describe things here as though they had already landed.

None of it writes the business decisions for you. The durable asset was never the reporting tool, and it’s not the model of the month either. It’s your definitions, your calendar, your entity structure and your history, recorded once, in a form the next tool and the next assistant can both read. Those will change. Your understanding of your own business shouldn’t have to be rebuilt each time they do.

FAQ

Common questions

Do I need a semantic layer if I only have one ERP?

Often yes, and the single-system case is less reassuring than it sounds. One ERP still holds several defensible dates on a sales order, several candidate costs behind margin, and more than one way to read what is available to sell. The ambiguity comes from the richness of the source rather than from the number of sources. A second system adds the problem of reconciling two entity models on top, which raises the value of a shared layer rather than creating the need for one.

Who maintains the definitions?

The business does, and that's the point rather than a caveat. A vendor can pre-build the structural part, how the ERP is organised, how the ledgers relate to one another, where the grain sits, because that's the same at every company running the same system and version. Deciding whether on-time is measured against the requested date or the confirmed one is a business decision and it usually takes a meeting. What changes is that the decision gets recorded once instead of being remade every time somebody asks.

Does a semantic layer make AI answers correct?

It makes them consistent and traceable, which is a smaller claim and a more useful one. An assistant reading a governed model applies the definition of margin your business agreed rather than deriving one on the spot, and a good implementation links the answer back to the report it came from so somebody can check it. It doesn't make the underlying definition right, and it doesn't remove the need for a person to judge whether an answer looks sensible.

Why does joining quantity and cost give the wrong total?

Because they usually sit at different grains. In Microsoft Dynamics 365 Business Central an item movement has one quantity entry but can carry several cost entries: an initial cost, an expected cost, a revaluation, an adjustment once the true cost is known. Join them naively and the quantity repeats once per cost entry, so the total comes out several times too high. Nothing about the definition of the measure is wrong. The relationship between the tables is.

Why does an old report change when nobody edited it?

Usually because it joins transactions to master data, and master data describes the present. Item category, customer territory, sales rep and standard cost are all fields that get overwritten. Recategorise a dozen items and every historical report that reads those categories restates itself, so last October's product mix quietly reads differently than it did in October. Keeping the state as it was has to be designed in, by capturing master data as it changes rather than overwriting it.

What are the requirements for a semantic layer?

Five things, and the first three are the ones textbook explanations tend to skip. Definitions the business has agreed and somebody owns, recorded once. A model that keeps the grain of the underlying records, so quantity and cost held at different levels of detail don't multiply when they're joined. History captured as master data changes rather than overwritten, so last October still reads the way it did in October. One place every tool reads from, so Power BI, Excel and an assistant inherit the same answer. And access rules that live in the model, so the same restriction applies however the question is asked.

See it on your own ERP data.

A governed warehouse and semantic layer for your ERP, live in days rather than months.