Skip Navigation
Expand
Dataminer aggregation of knowledge foundation tables
Answer ID 5922   |   Last Review Date 01/08/2019

How is the data in the Knowledge Foundation tables aggregated? 

Environment:

All sites on the November 2011 version or more recent

Resolution:

The dataminer utility is responsible for aggregating the data from the 'clickstreams' tables into the Knowledge Foundation tables.  The data is aggregated on a per interface bases depending on the type of activity performed during a given time interval. It is important to note that Dataminer runs on end-user data that is 4 hours old. Therefore, the newest end-user statistics that are available are 4 hours old.

Hourly (window = 0):
Initially, dataminer rolls the data into one hour intervals.  This roll-up happens at the end of the next hour that dataminer executes meaning the Xth hour will be processed during the first dataminer run after the clock strikes X+1.  For example, the data from the 12:00:00 to 13:00:00 hour will be rolled into the 12:00:00 hour the next time dataminer runs after the clock hits 13:00:00. 

Daily (window = 1):
The daily roll-up occurs when dataminer runs at the end of the next day.  Therefore, the data for 08/12/16 will roll into a daily window during one of the dataminer runs at the end of the day on 08/13/16 or the beginning of the day on 08/14/16.   

Monthly (window = 2):
The monthly roll-up occurs when dataminer runs at the end of the next month meaning the data for July 2016 will roll into a monthly window during one of the dataminer runs the last day of August 2016 or the first day of September 2016. 

Yearly (window = 3):
The yearly roll-up occurs when dataminer runs at the end of the next year.  Thus, the data for 2015 will not roll into a yearly window until dataminer runs on 12/31/16 or 01/01/17.

Available Languages for this Answer:

Notify Me
The page will refresh upon submission. Any pending input will be lost.