Not long ago I was asked to analyze a data set for a company of which I knew nothing, in an industry about which I was completely ignorant, for a team of people I had never met.
To add an extra layer of complexity, their B2B structure was tiered and convoluted, and the dataset—though not large—required extensive manual cleaning. They sent me the data, pseudonyms providing anonymity to their sources where necessary, with little to no additional context.
For most intents and purposes, I went into it blind.
Hours of work later, I sent them my slide deck and offered to present to the team via Zoom. After the presentation, the most common feedback I received was a little disappointing at first: I had told them many things they already knew (along with a few additional insights that they hadn't considered).
But it soon dawned on me that they couldn't have paid me a higher compliment. I, who had no knowledge of the company, its clients, or its processes, had been able to use a simple dataset to relate back to them information that they took for granted but about which I had been completely ignorant previous to that interaction.
Though not uncommon in my line of work, the process of forming those particular insights from the data was exhilarating to me. I had uncovered some underlying truths about the organization in the course of my analysis. But two questions have stayed with me ever since:
Can we ever really discover "truth"?
Does data truly aid us in that discovery?
While my simple answer to these questions is "Yes!", my reasoning is slightly more nuanced.
The Philosophical Debate
Truth. Fact. Reality.
All of these terms connote an underlying understanding and assurance of things as they really are.
We like to believe that truth is, whether we like it or not. That behind every decision, every situation, there are inherent laws that cannot be broken.
The law of gravity, for example, is that two masses exert forces upon each other. While we may use our understanding of other forces to keep airplanes aloft here on Earth, truly "defying” gravity is not a feat we've yet achieved (though still a great song), and one that we subsequently believe to be impossible. In other words, the truth of gravity (as explained by its law) is that this planet exerts a force upon us while we're close enough to it, whether we like it or not.
And that truth cannot NOT be. Right?
Physical Truth vs. Theoretical Truth
Physical Truth
The effects of gravity are an excellent example of what I will call physical truth, or truth manifest (and observed) in the physical world. The event of dropping a glass vase of flowers on the concrete inherently involves a number of physical laws and forces that are used to explain why you're left with a dangerous mess to clean up instead of a last-minute gift for your wife to save a forgotten anniversary.
However, physical events such as this are also subject to the viewpoints of those involved.. We can't deny that the vase did break (though I won't get into defining terms such as "break," as that's a discussion for another time), that the water spilled out, and that the roses scattered all over the sidewalk. That indeed happened. However, when we dive deeper into the details of the event, that's where we run into challenges.
Arguments could arise around the "intent" of the event—were you just being careless? Were you distracted? Did you throw the vase down in anger or let it slip from your fingers in shock? Even your own view of your intentions may be skewed by your desire to be perceived a certain way or maintain status as a good and thoughtful husband. Similarly, the "impact" of the event is also subjective—dropping a vase of flowers intended for your wife has a different impact on one husband than on another, on the same husband but on a different day or in a different mood, or on a flower delivery boy who already delivered the flowers to the wrong house first. Trying to piece all these things together (pun intended), even as the vase dropper yourself, can be challenging.
Even the underlying laws that are used to explain physical phenomena are under constant review. As we get new information, we adjust. In the meantime, we do the best with what we have, and we all try to understand where we are in that process—more on that below.
Theoretical Truth
Unlike physical events, what I will label theoretical truth exists independent of human subjectivity. Various mathematical principles illustrate this well: If we add 2 + 2 (integers), the result is a specific amount (we've labeled that amount "4"). We will not get more, we will not get less. It's simply impossible. If we subtract 1, we will get 3. If we multiply that by 2, we will get 6.
Simple, easy, objective truth. (And, it seems, generally numeric in nature.)
The Problem
Unfortunately, we live in a subjective, physical world, where our experiences and our perspectives shape how we view and define events. Our brains collect data that is then sifted through those filters. Our organizations and our high-tech data collection systems exist and operate in the physical world, and as a result may have those underlying biases built in, intentionally or unintentionally.
And, worst of all, for all the ever-increasing amounts of data that we collect and analyze as people and companies, there is no guarantee that it will lead us to truth.
So Why Even Try?
If we operate in a physical world full of physical truths, and if those "truths" are so often subjective, why even attempt to ascertain them? If our data inherently incorporates some degree of bias or error, then what's the point of even capturing it?
I would posit at least two reasons:
Data tells us much about ourselves
Even a little insight is impactful
Data Tells Us Much About Ourselves
I've written mostly in theoretical terms or in the abstract, but let's go back to the story of my project for the company about which I new nothing.
What they were acknowledging by saying that I had shown them things they already knew wasn't necessarily that I had discovered "truth" at its purest and most objective form.
Rather, they were acknowledging that I had told them much of what they already knew or believed about themselves. I showed them that their data reflected their processes and relationships, not solely insights into financials, etc.
Data can be extremely instrumental in helping us uncover our own subconscious biases and subjectivity (the "intent" and "impact"), even if the processes and tools that we utilize to collect and analyze that data may have some inherent bias built in.
That said, I highly recommend bringing in a third party (shameless plug for Insyyt's services here), and one that can be as objective as possible. Be willing to receive feedback, and be willing to trust that the data might be saying something about you or your organization that you may not like.
Even a Little Insight is Impactful
I also do not mean to argue that we can't gain valuable insights into the marketplace, internal operations, personal finances, etc. There are some things that we do an excellent job of capturing or are, by nature, difficult to be biased or skewed—the amount of money in a bank account, for example.
In a world teeming with complexity and ambiguity, we often find ourselves (individually or as organizations) at a crossroads, needing to move forward. When we've done everything we can to get our data and processes into a place of trust (as imperfect as it likely will still be), then even the smallest insight from that data can mitigate much of the doubt and anxiety that come from difficult decision making.
Again, nothing will be perfect. And no amount of data can make your decisions for you, as much as we humans have an irrepressible need to make sense of the world around us. Fortunately, we have made great leaps in progression because we chose to move forward regardless of ambiguity, doing our best to utilize the limited resources available to us.
The Ongoing, Messy Journey
Ultimately, it comes down to this: The world is not a straightforward place, and each of us has a unique set of experiences that informs how we view and interact with it. There are rarely "right" answers, and while I have a personal belief in certain indisputable truths, there is a multitude of things about which I am simply unsure and probably will always be.
But I believe that the undeterred, human need to make sense of the world is a noble endeavor. If we waited for everything to be perfect before making decisions, we would likely be an extinct species.
When we understand that we can learn much about ourselves through (albeit imperfect) data, and that even a little insight can be incredibly beneficial, we can help acknowledge and remove implicit biases, and feel more confident pushing forward with our decisions.
__
As always, I welcome your comments and feedback!
Comments