Metadata
- Author: Benn Stancil
- Full Title:: The Rapture and the Reckoning
- Category:: 🗞️Articles, Data Analysis
- URL:: https://benn.substack.com/p/the-rapture-and-the-reckoning#footnote-anchor-12-99275606
- Finished date:: 2023-04-07
Highlights
Tristan Handy, the CEO of dbt Labs’, said that a meaningful percentage of their customers are using dbt to create more than 5,000 tables (View Highlight)
And there aren’t often simple answers—the average query in Mode is 75 lines and 2,700 characters long. (View Highlight)
Balaji Srinivasan—who may well be a chatbot himself, trained exclusively on backwater crypto subreddits, the University of Austin’s “curriculum,” and Future, Andreessen Horowitz’s techno-optimism content marketing program (View Highlight)
the average Fivetran customer, for example, is extracting data from about 25 different services; (View Highlight)
A few years ago, Narrator, a product inspired by tools inside of WeWork, proposed a design pattern that they called the activity schema. If nothing else, I give it points for audacity: The entire blueprint is one giant event table. It’s as if you took a star schema, joined every dimension table onto its corresponding fact table, and then unioned all of those fact tables together into one bottomless list of events. (View Highlight)
For ChatGPT, however, this could be trivially easy—far easier than stepping through a convoluted series of joins (View Highlight)
As the skeptics of the chatbot analyst correctly pointed out, it’s the relational nuances that confuse AI models. But we could flip that. If AI-powered analytical tools have enough potential—which, seeing what ChatGPT can do, it seems like they do—would it not make sense to favor schema design patterns that enable the big-brained computers more than they enable us feeble-brained apes? (View Highlight)
Analysts would either become less relevant, or would finally be freed up to work on more interesting things. (View Highlight)
The implications of this would be profound. For decades, we’ve thought about building data tools and processes for people. They’ve been our best analysts, our best query writers, and our best decision makers. But I’m not so convinced that’s true anymore—and the only thing that’s keeping us on top is we’ve rigged the game in our favor. If that changes, if we design our worlds for AIs, if we give our power and strength unto the beast, we might be able to go much further than we can today. A new heaven and a new earth will come, and the first heaven and the first earth will pass away. (View Highlight)