Metadata
- Author: Katie Bauer From Wrong But Useful
- Full Title:: Skills and Jobs
- Category:: 🗞️Articles
- Finished date:: 2024-06-17
Highlights
You open up slack, hoping this might be a day where you actually have a little time to think for a change. (View Highlight)
You’d love for folks to be able to self-serve more with SQL but then think nervously about your data warehouse. It’s in a relatively good state but still has a confusing blend of modeled dimension and fact tables, and far more unmodeled clickstream data, replicated application databases, and data from a dozen or so SaaS products brought in by an ingestion tool. There are so many time grains, so many keys that seem like they should connect but don’t, so many entities defined in subtly different ways depending on where the data is coming from… Turning the ops team loose without guidance in your data warehouse feels risky. (View Highlight)
“Your errors are actually coming from violating the assumptions of this type of model,” you tell her, and then give her a high level explanation of how regressions work as you silently thank the libraries authors for building in these checks. “You don’t need someone who knows Python,” you conclude, “you need someone who knows statistics.” (View Highlight)
Some folks observe this trend and make a stronger claim—working with and interpreting data is a part of so many people’s jobs these days that we’ve reached the point where everyone is a data professional. Perhaps data is a skill that other folks have and shouldn’t need to be a job in its own right. (View Highlight)
What feels like the core of a job today could very easily be a vendor or framework used by some other role tomorrow. (View Highlight)
Is a data professional’s job writing SQL, or is it knowing how to unify heterogenous data from multiple conflicting sources to get a useful representation of your business? Is it knowing the APIs of Python libraries, or is it understanding how to frame a problem in a way that allows you to apply a statistical method and get an answer that isn’t falsely rigorous nonsense? Is it knowing how to find and pull data enough times that you finally get the answer to a seemingly unanswerable question, or is it knowing what types of questions quantitative data can actually be used to answer so you can stop digging before you’ve poured weeks of effort into them? (View Highlight)
In a world where tools reduce the barrier to entry, the higher level skills and understanding of when they’re relevant and how to use them becomes more differentiating. (View Highlight)
The thing that makes specialists worth hiring tends to be the size and complexity of the problems that specialists are good at solving. (View Highlight)
It only follows then, that the need for data jobs is driven by the data parts of other jobs becoming complex and time-consuming enough that they need to be spun out into a separate role. Tech and tooling advances change where that line is drawn, but it’s hard to imagine a world where the line disappears entirely. (View Highlight)