Computation
-
Batch ones are easy: just use dbt.
(https://www.tecton.ai/blog/why-real-time-data-pipelines-are-hard/)
- Interesting interplay between dbt and ML for Scalable computing of features: dbt + Machine Learning: What makes a great baton pass? | dbt Developer Blog
- Why not with https://www.terality.com/? It may be the easier way out!
Notes
- If we plan on using BigQuery, we need to think of the costs that will have, compared to a DB that does not charge per query or files.
- The popular arch reference from GCP: https://www.datacouncil.ai/talks/building-a-feature-platform-to-scale-machine-learning
