Checking numbers, data and code

Numbers and code feel objective, which makes AI errors in them especially easy to miss. Here's how to check them efficiently.

5.1 Numbers: sanity-check first

Before checking any figure in detail, ask whether it's even plausible:

• Order of magnitude: Is it in the right ballpark? (A UK café with £2bn revenue? No.)
• Units: £ vs $, thousands vs millions, per month vs per year, % vs percentage points
• Totals: Do parts add up to the whole? Do percentages sum to about 100%?
• Consistency: Does the same figure appear the same way throughout?
• Direction: If sales "rose 20%", is the later number actually higher?

5.2 Recalculate what matters

For any number that will drive a decision or be published, recompute it yourself — in a spreadsheet, a calculator, or by asking the AI to show the code it used and checking that the code does what it should. Watch especially for:

• Averages of averages (often wrong when groups are different sizes)
• Percentage change vs percentage-point change
• Filters that silently dropped rows
• Dates and time zones

5.3 Data analysis checks

5.4 Checking AI-generated spreadsheet formulas

1. Test the formula on a few rows where you know the right answer.
2. Check cell references — absolute ($A$1) vs relative (A1) — when copied down.
3. Test edge cases: blanks, zeros, text in number columns, very large values.
4. Ask the AI to explain the formula step by step, then check the explanation matches what the formula does.

5.5 Checking AI-generated code

AI coding assistants are powerful, but code that looks right can contain subtle bugs, security holes or invented functions and packages.

• Read it — never run code you don't understand on important systems or data
• Run tests — including edge cases; ask the AI to write tests, then check the tests make sense
• Check dependencies — AI can suggest packages that don't exist; attackers have registered such names to distribute malware. Confirm packages are real, maintained and the ones you intended
• Look for security issues — hard-coded passwords, unvalidated inputs, overly broad permissions
• Use normal code review — AI-written code deserves the same review as human-written code

# A quick way to test an AI-written function in Python
def vat_inclusive(price, rate=0.20):
    return round(price * (1 + rate), 2)

# Check known answers and edge cases before trusting it
assert vat_inclusive(100) == 120.00
assert vat_inclusive(0) == 0.00
assert vat_inclusive(9.99) == 11.99
print("All checks passed")

Key takeaways

• Sanity-check magnitude, units, totals, consistency and direction.
• Recompute numbers that drive decisions.
• Check row counts, missing values, definitions and causal claims.
• Test formulas on known answers and edge cases.
• Read, test and review AI code — and confirm packages are real.