How We Made Our Reporting Engine 17x Faster

Metadata

• Author: Quentin Fernandez
• Full Title:: How We Made Our Reporting Engine 17x Faster
• Category:: 🗞️Articles
• Document Tags:: Adtech data lake alternatives
• URL:: https://medium.com/teads-engineering/how-we-made-our-reporting-engine-17x-faster-652b9e316ca4
• Finished date:: 2023-04-15

Highlights

(View Highlight)

For example, for the creative_type dimension we can now group directly in BigQuery without joining, instead of pulling all creatives and checking their type from the dimensions data (View Highlight)

we were able to reduce the cost of the reporting engine by a factor of 10 by removing the EMR cluster and only focusing on a small service that forwards query and stream results. (View Highlight)