A database change management (DCM) tool is critical for managing your Snowflake objects and implementing CI/CD pipelines.
The traditional approach has been to try and manage all of your database objects with a single DCM tool
some objects don’t fit well into this model (like users, roles and permissions for example).
Snowflake supports SCIM (or System for Cross-domain Identity Management), which is “an open specification to help facilitate the automated management of user identities and groups (i.e. roles) in cloud applications using RESTful APIs.” In particular Snowflake provides SCIM support for both Azure Active Directory and Okta.
Like with users and roles, trying to manage privileges with a traditional DCM tool is troublesome and error prone.
This is especially true for imperative style DCM
The traditional approach has been to try and manage all of your database objects with a single DCM tool
some objects don’t fit well into this model (like users, roles and permissions for example).
Snowflake supports SCIM (or System for Cross-domain Identity Management), which is “an open specification to help facilitate the automated management of user identities and groups (i.e. roles) in cloud applications using RESTful APIs.” In particular Snowflake provides SCIM support for both Azure Active Directory and Okta.
Like with users and roles, trying to manage privileges with a traditional DCM tool is troublesome and error prone.
This is especially true for imperative style DCM
Managing the privilege definitions in code and using an open source tool to deploy them. See snowflake-rbgm (Rules Based Grant Management) by James Weakley or permifrost by the GitLab Data Team for two similar approaches.
objects which contain or represent sets/tables of data, such as tables, external tables, views, and streams for Snowflake.
In my experience something like 70–80% of data engineering time is spent managing these dataset objects and the plumbing that goes along with them.
dbt is the first tool I’ve seen to completely turn the data transformation process on its head.
The key thing to understand is that the dbt materializations take care of managing the dataset objects and plumbing code for you.
Managing the privilege definitions in code and using an open source tool to deploy them. See snowflake-rbgm (Rules Based Grant Management) by James Weakley or permifrost by the GitLab Data Team for two similar approaches.
objects which contain or represent sets/tables of data, such as tables, external tables, views, and streams for Snowflake.
In my experience something like 70–80% of data engineering time is spent managing these dataset objects and the plumbing that goes along with them.
dbt is the first tool I’ve seen to completely turn the data transformation process on its head.
The key thing to understand is that the dbt materializations take care of managing the dataset objects and plumbing code for you.