Metadata
- Author: Matt Arderne
- Full Title:: The Way of Ways
- Category:: 🗞️Articles
- Document Tags:: data culture, Data culture,
- URL:: https://groupby1.substack.com/p/the-way-of-ways
- Finished date:: 2023-08-22
Highlights
(ELT, synonymous with MDS, came under cost pressure. People realised that the work was pretty predictable and so cheaper, better, faster options became abundant. (View Highlight)
dbt democratised that:
Did we achieve more collaboration on an analytics code base? ✅
Did we achieve more leverage through reusable and modular code? ✅
Did we also buy more complexity, resulting in longer maintenance and debugging cycles? Unfortunately, also ✅ 🤓
Turns out the price of enabling people to build a more complex code base is… a more complex codebase, and everything that comes with that. (View Highlight)
the following “MDS categories”. (View Highlight)
(Business Intelligence tools continued to underwhelm, primarily because of the split between traditional reporting and exploratory analytics. (View Highlight)
Traditional BI just don’t move the needle in the same way that newer tools like Hex do. (Hex described this paradigm in deleted article, they now position themselves as a data tool that does reporting too). I use Hex daily. It is relatively cheap, it works very well and has sufficient depth to replace a fair chunk of MDS and technology infrastructure too. (View Highlight)
The data systems are largely good enough and so the bottleneck becomes what to do with the data. Data engineering was the bottleneck. In January 2022 I gave the opinion that Data Engineering was no longer the primary bottleneck to delivering insights/value/whatever. (View Highlight)
This is weird because I would say this is still Data Engineering
The challenges are now more subtle: 8 Track 1: “All I want is to know what’s different” Track 2: “The emotionally informed company” Track 3: “The truth is out there The only thing stopping us from finding it is us” Track 4: “Will we ever have clean data? Probably not, but maybe we can work with messy data” (View Highlight)