Listen to this article:
A 2020 report published by DAMA in the Netherlands found more than 60 different data quality dimensions. Traditionally, most practitioners recognize the “Core 8”: accuracy, validity, completeness, integrity, uniqueness, timeliness, reasonability, and consistency.
The one I want to focus on here is timeliness.
Actually, just a portion of timeliness, because until recently timeliness encompassed both freshness and latency, combining “How old is the data and is that acceptable?” with “Did the data arrive when it was needed?” The revised DMBOK 2.0 now separates currency from timeliness. Guess it’s now the “Core 9.”
Data should come with a USE BY or a FRESHNESS GUARANTEED THROUGH date. You wouldn’t drink milk that expired six months ago. You wouldn’t use a customer profitability report that was generated five years ago to make decisions about customers today. Or would you?
We don’t seem to give currency a second thought; about whether we’re consuming old expired data sets. Most likely we don’t know how old they are or whether they have expired (or should have expired). You’re going to take some action today and you need to have data relevant to the decision made today.
Notice that I didn’t say that you needed data that was generated today. That’s timeliness and that’s something different. Today’s data might not be needed, and it might not be wanted. For example, corporate processes might be geared around annual customer evaluations. Doing it more often than that might cause too much churn.
I’ve talked over and over again about the importance of Data Governance, Information Management, and Data Quality. Most of us will never have the responsibility of managing data that has life or death impact. But some of us do.
According to the preliminary findings of a Pentagon investigation, in the early hours of Operation Epic Fury one of the U.S. missiles hit a civilian target. My purpose here is not to comment on the conflict, its objectives, or the way in which it is being carried out, but solely on the data.
A New York Times article about the incident began, “Outdated targeting data may have resulted in mistaken missile strike.” It goes on to say that the building hit used to be part of an adjacent military base but had been converted to a school. Officers at the U.S. Central Command created the target coordinates for the strike using Defense Intelligence Agency data.
Again, the findings are still preliminary, but it seems that despite a well-defined, multi-departmental process for collecting, evaluating, validating, and presenting this data, the coordinates were not accurately assessed before they were used. At one point the data was valid, but apparently not any longer.
And, you know, some things never change. The Times article goes on to cite an incident in 1999 during the Kosovo war where outdated maps led to another incidence of erroneous targeting data. The Chinese Embassy in Belgrade was wrongly labeled as the headquarters of a Yugoslav arms agency. George Tenet, CIA director at the time, told a congressional committee, “Database maintenance is one of the basic efforts of our intelligence effort, but it is also one that has suffered in recent years as our work force has been spread thin.”
Even in 1999.
Some things never change.
Maybe the stakes aren’t life and death, but somebody is depending on that data that you manage or produce or provide or own or steward or host, and they’re depending on it to be correct. What would happen if that data is incorrect? Be honest about your answer. Ask your consumers. Maybe you’ll discover that the answer is nothing. If that’s the case, then you have to ask the next hard question: Why does the company need that data at all? It probably doesn’t. Let it go and reassign everybody to something more consequential.
Finally, are you even looking at the data to see if it’s correct? Most companies aren’t. Do you have data quality processes, and if you have them, how do you ensure that they are being followed? Most companies don’t. They remain intentionally ignorantly blissful. If you don’t have good answers to those questions, start there.
Featured Image: Steve Jurvetson, “AIM-9B Sidewinder heat-seeking missile Head.jpg” CC BY 2.0, via Wikimedia Commons.
