Data Modelling Schedule

Updated by Andreea Oprea

This help document provides an overview of the process needed to get updated data in Dreamdata and insights into the types of data available in real-time versus delayed time.

What are the steps of getting my data updated in Dreamdata?

  1. Dreamdata starts pulling data for all connected sources except the primary CRM once every day at 2 am UTC.
    For insights into when your data was last synced for each source, navigate to Home --> Dashboard and check the Sources Status Overview. The timestamp of your last synchronization is displayed on the right, as shown in the snippet below.
  2. Primary CRM data is extracted once or twice daily depending on the selection displayed in the schedule visible in the Data modelling schedule tab in the App.
  3. Data Modelling operations run once or twice daily depending on the schedule set in Dreamdata. The time it takes to build the data model depends on how much data you have and your model configurations. You can see an estimated run time and process end time in the Data modelling schedule tab. The estimation is calculated based on runs from the last 7 days and can change if there are significant changes in your CRM data or Dreamdata setup.
    For Team, Business and Enterprise plans you can select when you want your data modelling process to start based on the settings visible in the Data modelling schedule tab. The choices available be are between 4 AM and 10 PM UTC to allow Dreamdata to pull the data from all the other sources and be able to include it in data modelling. Some other options may become unavailable based on the estimation calculated for each individual case, as schedules should not overlap.
  4. The Data Warehouse (BigQuery, Cloud Storage, S3) is updated immediately after the completion of data modelling.

You can use the run time estimations provided in the Data modelling schedule tab if you want to know when to set up an automation, such as Reverse ETL or BI Reporting.

  1. Syncs are triggered immediately after the completion of data modelling. The time of the last sync is visible in each syncs overview page in Dreamdata.

You can also verify the live status of your tracking. Just go to our debugger tool found under Data platform --> Debugger.

You'll see all of your tracking data within a few minutes of the completed event.

Note: While the data appears in the debugger in real time, it may take up to 24 hours before your reports are fully populated.

We are always here to help so please reach out to friends@dreamdata.io with any questions or requests!

Thank you!

How did we do?