A few years ago, while visiting back home in Pittsburgh, a friend and I spent the day at Kennywood Park, a 125-year-old amusement park about 8 miles from downtown. It has one of the country’s best steel roller coasters, but my favorites are the wooden ones built in the 1920s. At the end of the night, we said our farewells and I got into my rental car, entered the address of my hotel into the GPS, and started out. After a few minutes of winding block by block through street-lit row house neighborhoods, it became clear that the GPS had selected the shortest distance route, not the shortest time route. Block by block. Stop sign by stop sign. The shortest time route would have been several miles longer, but on main roads and the Parkway. I just trusted the GPS to take me the “best way,” but I didn’t review the route first. I should have.
In the 1995 movie The Net, Sandra Bullock plays Angela Bennett, a computer programmer whose life is turned upside down when her online identity is erased and replaced with that of a destitute and mentally unstable criminal. The key theme running throughout the film is that “our whole lives are on the computer” and that people trust the computer more than the person standing in front of them.
The computer tells the grocery store checkout whether or not to accept your credit card. The computer tells the car dealership whether or not to offer you a loan and specifies the terms. Picnic plans, hurricane evacuations, and policies impacting national economies are made in response to computer model calculations. A recent study found that 54% of US adults say that they get news from social media sometimes or often. The computer said it; therefore, it must be true. This bias can be so strong and so pervasive that you can lose your job, be removed from social media, and be ostracized from the community for merely expressing skepticism.
But as Abraham Lincoln famously said, “Don’t believe everything you see on the internet.”
I first encountered the term Automation Bias in a conversation with Dave Loshin and my Rock Bottom Data Feed podcast partner, John Ladley. They were discussing root causes of organizational risk associated with the use of data. Dave cited Automation Bias:
We are predisposed to believe whatever the computer tells us.
Like my GPS.
Like an executive that arrives at the meeting armed with THE REPORT. They have an air of authority, regardless of whether or not the data in THE REPORT is correct. And when not properly governed, the data in THE REPORT may not be correct.
Yet, we are blind to that risk.
But it’s worse than that.
Since our bias, conscious or not, is that computers are correct, it follows that there’s no need for a Data Governance or Data Quality practice. The computer is always right.
This mindset subverts attempts to establish Data Governance and Data Quality practices.
We can argue the benefits of Data Governance and Data Quality programs forever, but until this underlying bias is reversed, we will continue to not be successful.
A colleague recently shared the story of a meeting for which he prepared a report that cast one particular department in a negative light. After he presented his findings, the department head challenged the data content, saying that it was wrong and that the report was wrong and that his department was not performing poorly. It’s easy to dismiss that criticism as a reflexive, protective measure in response to being embarrassed by the negative report.
And while that’s probably true, I applaud that mindset.
At about the same time, coincidentally, a friend who was having data quality challenges suggested that analytics data should be approached from a Zero Trust perspective. In finance, records must balance correctly and could be audited at any time. Their correctness is never assumed and must always be proven. Intrigued, I asked him if I could run with the idea. He replied, “Go for it. It sure would help me.”
The Zero Trust concept is borrowed from cybersecurity. It is a model designed to help protect sensitive data and systems from cyber threats. The National Institute of Standards and Technology (NIST) defines it as, “a collection of concepts and ideas designed to minimize uncertainty in enforcing accurate, least privilege per-request access decisions in information systems and services in the face of a network viewed as compromised.”
In other words, never trust. Always verify. Deny access by default. We can apply this concept to data content.
A Zero Trust Data Content strategy assumes that in the absence of sufficient evidence to the contrary, data is always assumed to be incorrect.
This totally contradicts the way that people interact with data. It is the opposite of Automation Bias.
Zero Trust was a response to the need for a mindset shift in computer security. The original design of the internet and its foundational protocols lacked built-in security. After all, its purpose was to enable communication between a small, trusted group of researchers, government agencies, and academics. As the internet expanded with more public and commercial use, access and authentication methods had to be developed.
A similar mindset shift from Automation Bias to Zero Trust is required for data.
In other words, never trust. Always verify. Assume that the data is incorrect by default.
Returning to the story of the doubting department head, my colleague responded with a detailed explanation of how the report was generated, the calculations, the source systems, and the way that those source systems collected the data. The data was assumed to be incorrect. He provided evidence of its reliability. That’s the way these conversations should go.
In future articles I’ll develop this Zero Trust Data Content approach further, exploring evaluation mechanisms and the use of Data Products to provide a framework for reliability.
Photo Credit: Serena Epstein, “GPS fail.” Flickr.com. Some rights reserved.
0 Comments