Why AI Hallucinations Are the Real Problem Behind the AI Adoption Numbers

A new survey covered by CPA Practice Advisor puts small business AI adoption at 66%, up from 55% a year ago. The same survey, run by Thryv, found 92% of AI users say it saves them time, and a third are now spending more on AI tools than they were 12 months ago. It also found that 70% of owners say they need more training to use AI effectively.

That last number is the one getting the headlines, and I think it is being read backwards.

Training is not the bottleneck. Reliability is.

The framing in most of the coverage is that AI adoption is rising but staff skills have not caught up, so the fix is more training. Ask a business owner what that training is actually for, though, and the honest answer is usually learning to spot when the tool is wrong. Checking a generated summary against the source, or re-reading an AI-drafted clause because the tool invented a term that was never there. That is not a skills gap. That is a business quietly building a manual QA layer on top of a piece of software, and calling it training. In South Africa, where most SMEs have no data science team to absorb that QA work, it falls on whoever is already stretched thin, usually the owner, eating into the returns that extra AI spend is supposed to produce.

The uncomfortable trade-off: the AI tools that are easiest to adopt, the generic ones bolted onto an existing workflow in an afternoon, are also the ones that push the reliability problem onto your staff. A 2024 study from Stanford's RegLab and Vanderbilt found that general-purpose legal AI tools hallucinated in 58% to 82% of queries about court rulings, even with retrieval-augmented setups meant to ground them in real case law. No amount of staff training changes the underlying failure rate of the model. It only changes how fast someone catches the mistake.

Building reliability in, rather than training staff to catch failure

The alternative is to treat reliability as an architecture decision, not a user skill. We ran into exactly this problem doing embedded CTO work for an EdTech platform that generates geometry diagrams for a children's learning app. An AI model asked to draw a triangle with specific angles will occasionally draw one that is geometrically impossible, confidently and without any indication it got the maths wrong. For a maths app, that teaches the wrong lesson, not just a rendering glitch. Our fix was to stop asking the AI to draw anything at all: a neural-symbolic architecture where the language model's only job is reasoning out the structural parameters (the angles, the lengths, the coordinates), which are then handed to a separate, deterministic rendering pipeline that constructs the image with ordinary code. The unreliable part of the system never touches the final output, so the diagram is mathematically correct by construction, regardless of how confident the AI's reasoning step was.

The same principle applies to architecture decisions AI cannot make for you: whether its output can be trusted is one of them. The same failure mode shows up when a prospective customer asks an AI search tool what your business actually offers: if the model hallucinates or misreads unclear content, getting the basics of AI search visibility right for a South African business becomes a reliability problem, not just a marketing one.

That is the shift I would want SA founders evaluating AI reliability software to look for. Not "does the vendor claim high accuracy," but "what happens architecturally when the model is wrong." A tool where a wrong output reaches your customer or your books unfiltered needs a human minding it constantly. A tool where the AI's judgement is checked, constrained, or handed to deterministic logic before anything ships is one your team can rely on without babysitting it, the same architecture-first thinking behind every custom software project we build.

None of this means avoid AI. The 92% of users saving time in the Thryv survey are onto something real. It means asking a sharper question before you adopt: is this tool reliable because of how it was built, or is it reliable because someone on my team is quietly checking its work? If you cannot get a straight answer to that question from a vendor, treat the 70% training figure as a warning label, not a to-do list.