Launched: dbt Cloud Integration

Highlights: Synchronizing your metadata from dbt to Holistics is easier than ever with dbt Cloud integration.

Our team would like to proudly anounce that, dbt Cloud Integration is available on Holistics 4.0 (Analytics As-Code). Please reach out if you want early access to Holistics 4.0.

Now, Holistics supports two ways to integrate with dbt. These are:

  • dbt Cloud Integration
  • dbt Core Integration (refer to this document)

How dbt Cloud Integration works?

  • dbt Service Account Token: This token enables secure authentication with the dbt Cloud API given that you grant each token a narrow, controlled set of permissions. You can learn more about it here.
  • dbt Cloud job: a set of dbt commands to execute against a data source. This is created in dbt Cloud.
  • dbt Integration job: an operation in Holistics to ensure that the metadata between dbt models and Holistics models are in-sync. By linking a dbt Cloud job with a data source in Holistics, we can get the metadata after dbt commands are run on that data source.

Why dbt Cloud Integration?

  • Results: After the dbt Integration Job executes successfully, all the metadata (descriptions, dependencies graph) in dbt will be synced automatically with related Holistics Table Models via the Data Source name and Source Table Name. This is the same with dbt CLI integration.
  • Automatic metadata sync: As for dbt Cloud Integration, Integration job triggers every 5 minutes or manually. As for dbt CLI Integration, if you have updated your dbt’s metadata, you will need to re-push your latest manifest.json to Holistics. Once done, the dbt’s metadata in Holistics is automatically updated.

Resources

  • See this tutorial for more information on how you can set up automatic metadata sync between dbt Cloud and Holistics.

Let us know if you have any questions or feedback in the comments below!

7 Likes