Why AI fails on poorly managed data

AI projects rarely fail by producing nothing. More often, they quickly produce something that sounds plausible, based on data that nobody has checked, and the organisation acts on it before anyone notices.

Most organisations starting with AI ask some version of the same question: which use should we start with? It is the wrong first question, because it assumes that ideas are what is holding them back. In almost every organisation we assess, the real problem is that the underlying records are not reliable enough to support a confident answer. Adding AI does not fix that. It removes the delays and caveats that used to make the problem visible.

When a person produces a figure from messy data, the process is slow and full of caveats. They know which source they do not trust, and they say so in the meeting. When an AI model produces the same figure, it arrives instantly, in fluent language, with no caveats at all. The uncertainty in the data is exactly the same. The only difference is that the warning signs have disappeared.

AI does not make bad data worse. It makes bad data more convincing, and that is worse.

Three things that must be true first

Before an AI project is safe to start, three questions need real answers. They should describe how things are today, not how a strategy document hopes they will be.

Ownership

Each important set of data has one named person who is responsible for whether it is correct. That means a person, not a department or a committee. Without an owner, nobody is responsible for noticing that a field stopped being filled in back in March.

Quality

You can say, with figures, how complete and consistent each set of data is. Those figures come from checks that run regularly, not from an exercise someone did once. They include duplicate rates, how often important fields are empty, and where records in different systems fail to match.

Access

You know who can read each set of data. That is enforced by a system rather than by habit, and it still applies when the data is copied to wherever the AI runs. Most governance failures in AI projects are not caused by the AI. They are caused by a copy of sensitive data turning up somewhere new.

What a two-week health check looks at

A data health check is not a strategy exercise, and it should not produce a maturity score. It produces a list of specific problems, each with a cost attached. For a medium-sized organisation, this is achievable in two weeks, and the results are usually uncomfortable in a useful way.

• Every system that holds a copy of a customer, product or transaction record, and which one the business treats as correct
• How many duplicate and near-duplicate records exist for customers, products and transactions, measured rather than estimated
• Fields that are filled in by habit rather than by rules, and are therefore filled in inconsistently
• Where systems that should agree do not, and by how much
• Where personal or commercially sensitive data is actually stored, including in exports and copies used for reporting
• Which questions your data can answer confidently today, and which it cannot

Clients usually find the last item the most useful. It changes the conversation from what we would like AI to do, to what our data can currently support. Those two lists are very different, and the gap between them is the real project.

The order that works

The order we recommend is not exciting, but the order is what matters. Each step makes the next one last.

1. Assign owners

   This comes first, because without a named owner, nothing stays fixed. Every improvement slowly slips back to where it started.

2. Measure quality

   You cannot improve what you have not measured, and a figure makes a stronger case for funding than an opinion.

3. Limit access

   As soon as data becomes useful, people start copying it. Access needs to be controlled before the data is moved, not afterwards.

4. Choose one narrow use

   It should be narrow enough that the results can be checked against an independent source. If you cannot check the results, you learn nothing.

This is not an argument for delaying AI until your data is perfect. No organisation’s data is perfect, and those that wait for it tend to wait forever. It is an argument for knowing which parts of your data are reliable enough to build on. A business with three reliable sets of data, and an accurate description of the rest, is in a far better position than one with forty sets of data and no idea which to trust.

The governance question that gets skipped

There is one more problem that poorly managed data causes, and it tends to appear late. When an AI answer turns out to be wrong, and a decision was made based on it, someone will ask what data the AI was using. If nobody recorded where the data came from, that question cannot be answered, and the organisation has to explain a decision it cannot reconstruct.

Recording where data comes from is not difficult if it is designed in from the start, but it is almost impossible to work out afterwards. That is the main argument for doing the unglamorous work first. The same is true of audit trails, access controls and other structural decisions. You are not paying for tidiness. You are paying for the ability to answer questions that you cannot yet predict.