Metadata
- Author: Airbyte
- Full Title:: The Rise of the Semantic Layer
- Category:: 🗞️Articles
- Document Tags:: metrics Layer,
- URL:: https://airbyte.com/blog/the-rise-of-the-semantic-layer-metrics-on-the-fly
- Finished date:: 2023-06-12
Highlights
But in the other hand, queries may be super slow/expensive
In the transformation layer, you must balance low and high granularity. What level do you aggregate and store (e.g., rollups hourly data to daily to save storage), or what valuable dimensions to add. With each dimension and its column added, rows will explode exponentially, and we can’t persist each of these representations to the filesystem. A semantic layer is much more flexible and makes the most sense on top of transformed data in a data warehouse. Avoid extensive reshuffles or reprocesses of large amounts of data. Think of OLAP cubes where you can dice-and-slice ad-hoc on significant amounts of data without storing them ahead of time. (View Highlight)