Metadata
- Author: Benn Stancil
- Full Title:: The Design Flaw at the Heart of Every Data Product
- Category:: 🗞️Articles
- URL:: https://benn.substack.com/p/the-design-flaw-at-the-heart-of-every
- Finished date:: 2024-05-18
Highlights
What should Ava do? Which of the four dozen noisy graphs should she drill into? Who knows. Nobody knows. Some charts are up. Some charts are down. (View Highlight)
There is only noise, and a useless chatbot that nobody wants. The dashboard is a horoscope, capable of telling whatever story its reader is looking for. (View Highlight)
Well, Ava thinks, it was like this yesterday, and we’re still here. Ava moves on. (View Highlight)
Most of us are Red Eye Entertainment, plagued by messy data, erratic metrics, and graphs with too many series. This is why most data products never look as good as the mocks:3 It’s not the products that are the problem; it’s the data we put underneath. (View Highlight)
That, I think, is a gross misunderstanding of what most businesses need. Their problems are more like Red Eye’s: There are needles everywhere, and they all halfheartedly point in different directions. Not only does the chart wiggle, 44 of them wiggle. Some wiggle up. Some down. Some wiggle across 20 product SKUs. Some have too many null values to wiggle. Some just say, “Error while reading data, error message: Failed to parse JSON: Unexpected end of string; Unexpected end of string; Expected key.” (View Highlight)
In other words, we aren’t detectives looking for the single strand of hair that explains the entire case. We’re Benoit Blanc, investigating a murder full of contradictory clues where everyone has a motive. Interpreting data is hard because everything is inconclusive—MQLs are up, ACVs are down, NRR had its worst quarter ever and DAUs have never been higher—and we don’t know how to add it all up.5 (View Highlight)
All of this, however, is often counterproductive. More clues don’t help us know what to do; they just create more confusion. Instead, we need help making sense of what we already know. We need help with deduction, not discovery. (View Highlight)
The elephant in the room From Julie Zhao:
A famous Indian parable describes five blind men encountering an elephant for the first time. They each decide to touch the elephant to understand what the animal is like.
“An elephant is smooth and hard, like bone,” says the man who grasped its tusk
“Nonsense, an elephant is soft like leather,” says the man who felt its ear.
“You’re both wrong; an elephant is rough like a tree,” says the man who touched its leg.
The five men begin arguing vehemently, each convinced that he is right. None of them manages to convince the other of anything, except that everyone else is untrustworthy. (View Highlight)