TMADD Complete — Lattice Scoring, Audit Trail & Seed Rules

The Full TMADD Implementation

This release completes the True Mathematical Architecture Design Document (TMADD) — a four-phase overhaul of SpeyBooks’ transaction categorisation engine. What was a simple “first match wins” system is now a deterministic, explainable, auditable matching engine built with the same engineering discipline used in safety-critical systems.

Every categorisation decision now produces typed results, scores rules on a six-dimension vector, logs the decision with full evidence, and ships with sensible defaults for UK contractors. Six versions shipped in a single session: v4.3.0 through v4.6.0.

Rules Management Page (v4.3.0)

A dedicated Rules page where you can view, edit, delete, and reorder your categorisation rules.

• Full CRUD interface — table view with name, field, operator, value, target account, priority, and active toggle for every rule.
• Test panel — enter any transaction description and see which rule matches, why it matched, and the score breakdown. Essential for debugging categorisation before you import.
• Priority ordering — drag rules up and down or set explicit priority numbers. Auto-assigned priorities use gaps of 10 so you always have room to insert.
• Sidebar integration — “Rules” appears in the main navigation alongside Transactions and Import.

Phase 1: Typed Results & Evidence (v4.4.0)

Replaced the old “account or nothing” return with a proper typed result. Every categorisation now returns either Known (with the matched account and evidence) or Unknown (with a typed reason explaining why).

• Four failure reasons — NoMatch (no rules matched), AmbiguousTopScore (multiple rules tied with different accounts), InvalidRule (matched rule references a deleted account), and MissingField (transaction has no data in the target field).
• Evidence on every decision — the test panel shows exactly which field matched, which operator was used, what the normalised input looked like, and how many alternative rules were considered.
• Server-side rule creation — rule creation during import moved from the frontend to a dedicated API endpoint (v4.4.1). Rules are created atomically alongside the categorisation decision.

Phase 2: Lattice Scoring (v4.5.0)

Replaced sequential “first match wins” with a lexicographic lattice scorer. When multiple rules match the same transaction, the engine scores each one on six dimensions and picks the mathematical maximum.

The Six Dimensions

• OperatorStrength — equals (5) beats starts_with (4) beats contains (3). An exact match is inherently more specific than a substring match.
• FieldStrength — reference and contact fields score higher than description, which scores higher than amount. More informative fields produce more reliable matches.
• PatternSpecificity — longer match values are more specific. “COSTA COFFEE” (12 chars) beats “COSTA” (5 chars).
• ValueFit — how much of the input the rule covers. A rule matching the entire description scores higher than one matching a substring.
• Priority — your explicit priority number. When you deliberately set a rule to priority 100, it wins. Priority resolves intent; specificity resolves ambiguity.
• TieBreak — earliest creation date, then lowest ID. Guarantees absolute determinism under all conditions.

What This Means in Practice

If you have a contains "UBER" rule pointing to 7320 Travel and a contains "UBER EATS" rule pointing to 7460 Subsistence, the engine automatically picks the more specific rule for food deliveries — without you touching priority numbers. The score vector is visible in the test panel so you can see exactly why a rule won.

Regex rules include a specificity safeguard: literalChars − 2 × wildcards. A lazy .* pattern always loses; a precise TESCO.*LTD pattern scores comparably to equals.

Phase 3: Audit Trail (v4.5.1)

Every categorisation decision is now persisted with full evidence, enabling “why was this categorised here?” queries and regulatory audit readiness.

• Score vector stored — the six-dimension vector is persisted as an integer array alongside every decision, not just displayed in the test panel.
• Evidence as structured data — the full evidence object (matched field, operator, normalised input, rule count, reason) is stored as structured data for each decision.
• Row Level Security — audit entries are tenant-isolated using the same security model as all other SpeyBooks data.

Phase 4: Enhanced Logging, Corrections & Seed Rules (v4.6.0)

The final phase adds three capabilities that close the loop on categorisation intelligence.

Enhanced Match Logging

• Normalised description captured — the audit trail now records what the engine actually matched against after normalisation. “CARD PAYMENT TO SHELL PETROL” becomes “SHELL PETROL” — you can see exactly what the normaliser stripped.
• Bank name resolution — each decision records the canonical bank name (NatWest, Starling, Monzo, etc.) for bank-specific analytics. No more fragmentation from “NatWest” vs “NATWEST” vs “Nat West”.
• Outcome consistency enforced — Known decisions cannot have a failure reason; Unknown decisions must have one. The database enforces this constraint.

Correction Tracking

• User overrides logged — when you change an auto-categorised account during import preview, or recategorise a Suspense transaction later, the correction is recorded with the original decision, the new account, and the source of the change.
• Append-only history — multiple corrections on the same transaction are valid. The latest is authoritative; earlier corrections are training data for future rule improvements.
• Survives log maintenance — if older audit entries are purged, corrections remain intact with a null reference to the original decision rather than being cascade-deleted.

Seed Rules for New Accounts

New organisations are created with 26 high-confidence categorisation rules across 9 groups, tailored for UK sole traders and contractors.

• The Seed 9 — Fuel (Shell, BP, Esso, Texaco → 7300), Telecoms (BT, Vodafone, EE, O2, Three → 7540), Post (Royal Mail, Post Office → 7550), Parking (NCP, APCOA, RingGo, PayByPhone + catch-all → 7310), Subsistence (Greggs, Costa, Pret, Starbucks, Uber Eats → 7460), Travel (TfL, Uber, Trainline → 7320), Companies House (→ 7900), and ICO (→ 7900).
• Operator precision — BT uses starts with "BT " (not contains) to avoid matching DEBT, SUBTOTAL, or ABT. O2 uses starts with "O2 " to avoid CO2. UBER EATS has priority 110 vs UBER at 90 for explicit separation.
• Low regret only — we deliberately exclude HMRC (could be VAT, PAYE, SA, or Corp Tax), supermarkets (ambiguous between subsistence and office supplies), SCREWFIX/TOOLSTATION (context-dependent), AMAZON (massively ambiguous), and utility providers (sole traders claim flat-rate use-of-home, not actual bills).
• Marked as defaults — seed rules show a “(default)” badge in the Rules page. You can edit or delete them freely — they’re yours to customise.

The Complete Picture

No other accounting platform — Xero, FreeAgent, QuickBooks, or otherwise — provides typed categorisation failures with machine-readable reasons, evidence-backed decisions, specificity-aware matching, or full audit trails on every categorisation decision. This is safety-kernel engineering applied to financial data.