What is the correct way to use FullPersistence?

Ask the Community
1

Hello Team,

I have a report that include a lot of calculations and statements. I’m using MySQL and I tried to use like WITH statement but still the report is very large and heavy and needs a lot of time to run.

I have found in Holistics something can help me called “Persistence” and we do have some kinds in Holistics. I tried to use it but the problem is when I set up the schedules.aml file I put like this.

const schedules = [
  // Schedule orders model to run every 10 minutes
  Schedule { models: [my_module], cron: '0,10,20,30,40,50 * * * *' }
]

So it works great and the report now run faster. and I put like this in my module.

persistence: FullPersistence {
    schema: 'my database name'
    view_name: 'randome name' => and this name appears later in my db
  }

So the problem is that each 10 or 15 minutes I found a new table in the DB start with the view_name and some numbers and letters after … For example:: randome_name__H562bc0_T2209242

This is a big issue and I don’t want the table to be duplicate each 10 minutes … If I have any changes in the report I need to update the same table that Holistics generate and don’t create another one because in this case the DB will down.

2

Hi @Hamza_Rashed1,
Since FullPersistence creates a new table every time it’s triggered, there’ll be a lot of temporary tables created as your cronjob runs every 10 minutes. While we do clean up these tables daily, creating one every 10 minutes is not ideal.

For your case, we have some possible recommendation:

  1. Adjust the cron schedule: Depending on your use case, if you don’t need strict real-time data, you should modify the cronjob to run every few hours instead of every 10 minutes. This will help reduce both system load and database storage usage.

  2. Use IncrementalPersistence: instead of rebuilding a new table completely, this option will append to the existing table instead. Note that it requires additional configuration, you can read more on that here. We still recommend reducing the cron job frequency if possible.

I hope this helps! Let us know if you have any further questions.