Finance can’t deal in “probably”

For twenty years, business software has done one thing so reliably that nobody thinks about it. Run a saved report today, run it again tomorrow on the same data, and you get the same numbers.

An AI chatbot that builds the report itself doesn’t work that way. You can see it for yourself in five minutes, and it’s worth doing before anyone puts a chatbot’s numbers in front of the board.

Try it: one table, two chats

You don’t need your own data for this, and you shouldn’t paste company numbers into a public tool.

Open ChatGPT, Claude, Copilot or Gemini. Start a new chat, paste the prompt below, and note the answer. Then start a second new chat and paste it again. For a stronger test, try a second assistant as well. The table is invented, but every row in it is the kind of thing your ledger contains.

“Don’t ask me any questions” is there on purpose. A careful assistant might ask what you mean, which is the right instinct. Most people asking for a board pack draft don’t want a conversation, and they take what comes back.

Now put the answers side by side and compare three numbers.

Then read the notes, if there are any. Better assistants list their assumptions. Look closely: the assumptions themselves are likely to differ from one chat to the next, and assumptions are the first thing cut when a draft becomes a board paper.

What changed between the two chats?

Before a chatbot can add anything up, it has to make accounting decisions. The prompt doesn’t say how, so it decides for itself, and it doesn’t have to decide the same way twice.

Every switch below is one of those decisions. Each option is a reading a reasonable person could defend. Flip them and watch the board pack change.

The same data, three defensible board packs

None of these chats made an arithmetic mistake. Each one added up correctly from the rules it chose. That’s what makes the problem hard to spot: every version looks finished, confident and internally consistent. Two of them even agree on the best region, for different reasons.

What a wrong number costs

A hypothetical, but not an unusual one. The board pack names West as the best-performing region, as Chat C did. The business acts on it: two sales hires go to West, and next year’s marketing budget follows them. Two quarters later, finance rebuilds the quarter under the agreed rules. West’s lead was a pro forma quote counted as a sale and New Zealand dollars added as if they were Australian. North was ahead all along.

By then the hires are in place and the budget is spent. Fixing the number takes an afternoon. Unwinding the decision made on it takes far longer.

The question for a CFO isn’t whether the AI can add up. It’s who decided what counts.

Why does this happen?

There are two causes, and the second one matters more.

The chatbot writes every answer fresh

An AI assistant builds its reply from predictions about what text should come next. Those predictions can come out differently from one run to the next, so wording, layout and emphasis vary. The technical word is probabilistic. On its own, this is a nuisance more than a risk: you can read past different wording.

Your data doesn’t carry its own meaning

The table records what happened. It doesn’t say that your financial year starts in July, that a pro forma is a quote, that INV-0822 was re-issued, or that support contracts report as services. Those rules live in your accounting policies, your chart of accounts and the heads of your finance team.

An AI given raw data has to fill those gaps every time it’s asked. Because it writes each answer fresh, it can fill them differently each time. Your real ledger has hundreds of these gaps, against the six in this table.

Won’t a better model fix it?

A better model makes fewer slips and is more likely to list its assumptions. It still can’t know your fiscal calendar or your revenue rules, because they aren’t in the data. More intelligence applied to an undefined question gives you a more articulate guess.

Can’t we just put the rules in the prompt?

You can fix one rule that way. Write “Q3 means January to March” and that gap closes. A complete set of reporting rules runs as long as your accounting policies, every person would write it slightly differently, and nothing stops the next person leaving it out. A template fixes the layout of a report. It doesn’t decide which transactions count.

Where does “exactly” come from?

From deciding the rules once, writing them down where every report and every assistant reads them, and letting the AI ask for a defined number instead of composing one.

That foundation has two parts, and they do different jobs.

A governed data warehouse holds what happened. Transactions from your ERP and the systems around it, cleaned up once: a re-issued invoice is one document, a quote is marked as a quote, and history is kept, so last quarter’s report can be reproduced as it stood at the time.

A semantic layer holds what it means. Which months make up Q3. What net sales includes. Which exchange rate applies, and for which period. Which product lines count as services, and which region each customer belongs to. Each rule is decided once by the people accountable for it.

With both in place, the AI’s job changes. It no longer decides what counts. It works out which defined measure you are asking for, the platform calculates it under your rules, and the AI explains the result.

Two further things follow, and both matter in a finance function.

When a number changes, there’s a reason you can point to. A late posting, a correction, a new month’s exchange rate. The figure moves because the data moved, never because the question was read differently this time.

When something isn’t defined, a well-built system says so. If nobody has decided whether a new product line is services, the right answer is “that isn’t defined yet”, rather than a confident guess. An assistant that answers everything is worse than the report it replaces, because the report at least fails visibly. Hold any AI to that standard.

Where is “probably” fine?

Wherever a person is going to read, check and rewrite the result anyway.

A governed foundation doesn’t take AI out of the right-hand column. It makes the right-hand column safe for AI: the assistant brings the speed and the explanation, and the platform brings the numbers.

Where Zap fits

The Zap AI Data Platform is that foundation, built from your ERP.

Zap Data Hub brings the data from your ERP and the systems around it into one governed data warehouse, with pre-built models for Sage, Syspro, Microsoft Dynamics, SAP Business One and Xero. On top of the warehouse sits the semantic model: your measures, your calendar and your hierarchies, defined once. We’ve been building that for more than fifteen years.

Everything reads from the same model. Zap Analytics reports, Excel through Zap Live, Power BI and Zap AI all use the same definitions under the same access rules. So when someone asks Zap AI for Q3 sales, it works from the measure your finance team defined, rather than a rule it improvised for the occasion.

The AI layer will be open as well. When MCP access is released, Zap will expose the same governed tools to other assistants through the Model Context Protocol (MCP), so an assistant your business already uses can ask for the defined measure in the same way Zap AI does.

What the assistant writes around the figures will still vary from run to run. That’s the draft, and it’s yours to read. The figures underneath won’t vary, because nobody is improvising them.