Listen to this article:
So much of corporate behavior is driven by beatings; or more accurately, by the avoidance of beatings.
Research consistently shows that people are more motivated by avoiding negative outcomes than achieving positive ones. This tendency, is referred to as negativity bias, and is especially evident in gambling and investing. Many will pass on the opportunity to gain ten to avoid the risk of losing one. This dynamic can even be found in the Bible in the Parable of the Talents, where the third servant’s inaction and failure are driven by fear of losing the master’s money entrusted to him.
Most companies have a forum for reviewing system failures. The website goes down for a half hour, and you have to show up in front of the Audit Committee or Risk Committee or IT Executive Committee or whoever to explain what happened, what you did to fix it, and how it’s not going to happen again. Nobody wants to have to do that.
One day, the metric that your team generates has a problem. An incorrect answer was given to a customer, and the next thing you know you’re summoned to explain yourself in front of the executive committee.
Your next two days are consumed with preparation. All hands on deck. You’re not in a very good mood. Your team is walking on eggshells, and nobody is happy. On Thursday you show up, present your remediation plan, and take your beating.
It turns out that the data problem originated in an operational source system error that was thought to have been corrected but wasn’t. Or at least wasn’t fully. The error probably could have been caught if someone was looking for it. The data was considered to be good enough for operations, and besides, nobody knew what it was supposed to contain anyway.
Since the root cause of the error wasn’t discovered until shortly before the meeting, the source system management did not have to appear. After the meeting, the source system management got some “second-hand beating.” Perhaps a sharply worded email or an uncomfortable conversation. But to quote Aldo Raine in Inglorious Basterds, “I’ve been chewed out before.”
This is how data silos are born, and I totally understand it. Management is incented to not rely on any other team; not to leverage existing artifacts. I need to own everything I need. After all, if I’m going to get a beating, it’s going to be for something that my team did and that I can control.
It is the natural consequence of misaligned beatings, and from a data perspective it is totally wrong.
This is where Data Products are particularly valuable. With Data Products, responsibility, accountability, and ownership are established up front. Understanding the data, especially its expected content, becomes standard operating procedure. Data Quality is evaluated continuously. And while problems should be caught before making it all the way to the customer, we have clear defect identification, escalation, and remediation procedures.
The entire team, business, technical, and data, stands together, takes their beatings together, and moves forward together.
