Every money figure Moola's chat shows you is calculated in code and checked against your own projection before it reaches the screen. The model writes the sentence; it is never allowed to produce the number.
That distinction is the whole of this post. Most financial chat products hand a language model your data, ask it a question, and print what comes back. The number in the answer is then a plausible number, which is not the same thing as your number. Over the past fortnight we closed the last routes by which a figure could reach you without having been computed, and we want to show you the actual failures that got us there, because they are more convincing than the principle.
The £70,000 that should have been £80,000
On 13 October our nightly first-run quality run flagged four replies. Triage had already looked at them once and marked them "not reproducible", which is the most dangerous verdict a number can get.
Here is the first one. A user was looking at a £400,000 property. The reply read:
Deposit: £70,000 (assuming the same 20% deposit)
Twenty per cent of £400,000 is £80,000. The sentence contains its own refutation, and it still went out, because nothing in the pipeline was reading the arithmetic the sentence was asserting.
It now does. Every reply is scanned for relations of the form "P% of B is R" and "P% deposit (D)", each one is recomputed exactly, and a wrong amount is corrected in place. The mortgage figure built on the wrong deposit is corrected with it. This runs on the dashboard, in first-run, in the greeting, and on the WhatsApp, taxes, investing and accounts surfaces. The accounts surface is now buffered specifically so the check completes before you read the reply rather than after.
The model writes the sentence. The engine owns the number.
The other three were the same shape in different clothes. A reply priced "bump it up to 10%" as though the whole ten per cent were new, rather than pricing the change from today's rate. A reply said "now that you've adjusted to age 60, everything aligns" when nothing had been modelled at age 60 at all. And a reply answered "why did my net wealth change just now?" without naming the growth slider the user had moved a moment earlier. Each now has a guard that holds the reply to figures the engine produced, and replaces the sentence with one built from those figures when it does not.
Age 48, or age 49
This one is subtler, and it is the reason we no longer trust a reply that merely looks checked.
A transcript review on 13 October found a dashboard reply, with no tool call behind it, that said you were "currently projected to reach financial freedom at age 48". Recomputed from that user's saved rows, 48 was the figure including an unsaved what-if they had drawn one turn earlier. Their saved position alone gave 49.
The guard that exists to catch exactly this had been running the whole time. It missed, because the bold markers around the number broke every pattern it matches on. A reply in markdown emphasis was invisible to the checker.
So the checker now strips emphasis before it reads, and on a turn with no fresh calculation it is handed the structured figures from the screen you are actually looking at. A claimed age has to be your headline age, your partner's, or the saved-only age underneath a what-if you have drawn. Anything else is replaced. If the figure came from an unsaved what-if, the reply now says so and gives you your saved number alongside it.
Six languages, or it does not count
Moola chats in English, Italian, Spanish, French, German and Brazilian Portuguese. All of the above was written in English and read only English, which meant that for a fortnight the guards were decoration for five sixths of the people they were meant to protect.
"Anticipo: 70.000 € (lo stesso anticipo del 20%)" on a 400.000 € price passed every check. So did "ahora que lo has ajustado a los 60, todo cuadra", and "du könntest mit 58 aufhören".
Fixing it meant teaching the checkers to read money the way each locale writes it: 1.234,56 and 1 234 and 1'234, symbols before the number and after it, "R$ 80 mil" and "1,2 Mio. €". A corrected figure keeps the grouping the reply used, so a German reader does not suddenly get a British comma. One of the bugs was a single regular expression that never matched the Italian "perché", because a word boundary does not fire after an accented letter.
The test file for this holds 125 cases. 79 of them failed against the code as it stood.
Eight hundred imaginary households a night
Guards catch a wrong number on its way to you. They do not tell you whether the engine behind it is right. That is a separate exercise, and it runs nightly.
We generate synthetic households and push them through the real path, from questionnaire to projection, with the engine's monthly ledger recorded as it goes. The personas span the United Kingdom, Switzerland, Germany, the United States, Italy and Ireland, salaries quoted gross and as take-home, solo and couples in both partner shapes, pensions and ISAs and Lifetime ISAs and general accounts and cash, and positions in surplus, in shortfall, at the annual allowance, at the ISA cap and in retirement.
Seven invariants have to hold for every one of them. Money is conserved. The ledger agrees with the display. A what-if with nothing changed equals the saved plan. Paying in more never leaves you worse off. Accounts respect their caps. A number you stated is still the number we show. When one fails, the persona is shrunk to the smallest version that still breaks and printed as a fixture.
Twelve personas run on every pull request. Eight hundred run each night against a random seed.
The detail we are most pleased with: the tax law in those tests is written out as literal figures with their sources, rather than read from the engine's own constants. A test that imports the thing it is testing can only ever confirm that the engine agrees with itself.
That exercise currently has fifteen open findings against our own engine and converter. They are listed in the repository as failing tests, pinned, with notes.
What Moola deliberately does not do
We do not let the model do arithmetic. Not as a prompt instruction, which is a request rather than a constraint, but structurally: the figure is computed, the reply is checked against it, and a reply that disagrees is replaced before you see it.
We do not hide which model answered you. The chat says "Powered by OpenAI and Anthropic models", and every single turn records which model produced it, so when a pattern of bad answers appears we can tell you where it came from instead of guessing.
We do not claim the engine is never wrong. Fifteen known violations is not zero, and we publish the number rather than the aspiration. The useful claim is narrower and we will stand behind it: a figure that reaches you has been computed from your data and checked, and when we find one that was not, the finding is written down.
We do not turn a checked number into advice. Moola tells you what your position is and what a change would do to it. It does not tell you to make the change.
Why this is the thing worth building
There is a version of this product that is much easier to ship. Pass the user's accounts to a capable model, write a careful prompt, and the answers are right most of the time. The trouble with most of the time is that you cannot tell which time you are in, and neither can the person reading it. A plausible £70,000 and a correct £80,000 look identical on a phone screen at eleven at night.
A number you are going to make a decision on has to come from somewhere you can point at. That is what we have been building, and it is why some of these fixes took sixteen rounds of review for a change you will never notice.
If you want to see it, open the chat and ask it something with a percentage in it.
