Development workflow//testing

[Khai_To]

Hi @dacou,

Thanks for your detailed explanation.

@dacou

For now, the models and dataset preview feature are useful for the first controls :

1. “model preview” : detect formatting issues, visual check of measures
2. “dataset preview” : control relationship + sandbox for dataset first checks

When these first checks are done, we publish the dataset in order to start building dashboard, reports and filters. At this stage, the dataset is already merged in the master branch but some major changes can still occur on to implement features that could not be detected before

I totally understand that our current version doesn’t support this use case well.

The only (upcoming) option that you can use is our Preview Reporting, but we haven’t released it yet due to several performance issues. Our team is actively working on the improvement so that we can release it as soon as possible.

---

@dacou

For a new dataset, these first steps happen with models and datasets querying a staging database.
When it’s done, we publish the new tables to our production database and switch the datasource of our models and datasets to production.

With this workflow it is not easy to have at the same time multiple versions of the same models and datasets (one production version and one+ version for dev/test).

I think that this use case is totally valid. We’re discussing internally how to best solve this, and one of the options we’re thinking of is environment variable where we allow users to customize the behavior of your project depending on where the project is running (either Production, Development, or a specific Branch).

For example, you can define the dataset

Func ds_name () {
  if (env.IS_PRODUCTION == true) {
    env.PROD_DB_NAME
  } else {
    env.DEV_DB_NAME
  }
}

Dataset name { 
  label: 'Name'
  description: 'Something here'
  data_source_name: ds_name()
  // Other configurations
}

and the .env file

IS_PRODUCTION=true
PROD_DB_NAME=prod_db_name
DEV_DB_NAME=dev_db_name

Do note that it’s just the drafted solution . The final one could be different.

---

@dacou

One ideal solution would be to have one single holistics env but have “branch support” included in the URL for both reporting and modeling so that we can share with our key users a dataset or dashboard for review before merging to master :
https://eu.holistics.io/{release-xxx}/dashboards/v3/…

With this use case, basically, you want to share a version of your Data (dashboard) to another Data Analyst so that he/she can check review it before Deploying to Production.
With the release of our upcoming Preview Reporting, you can basically Commit and Push your changes to a particular branch and ask the reviewers to checkout that branch and go to the Dashboard to validate the data.
But of course, with the branch name included in the URL, it would be much easier for you to quickly share the Dashboard with the reviewers.
I will note this down and consider supporting it in the future.

Do let me know if you have any questions.

Thanks,