Aggregation of Data

Aggregation means that you collect separate data in a summarized form. Aggregation can dramatically reduce the need for data storage space and the time it takes to query large sets of data.
PerformanceGuard aggregates the data that PerformanceGuard agents collect. Basically, this means that you will have very detailed performance data from recent points in time and more summarized data about historical performance.

Let’s take a look at how PerformanceGuard aggregates data:

Caution

The following is based on the default data retention periods (called keep periods) in PerformanceGuard. You can adjust the keep periods to suit your organization’s exact needs.

Agent Performs the Very First Aggregation

PerformanceGuard agents store collected data at regular intervals, determined by a setting called report interval. The report interval is 60 seconds, so an agent will store collected data once every minute. During those one-minute intervals there may be a lot of identical combinations of server IP address, port and protocol data. Such identical combinations of data are automatically summarized by the agent into one line of data that covers the average for the whole report interval.

This is the very first step in the aggregation of data.

Important

Can I change the report interval?  If you change the report interval, it can have a significant impact on the size of your database. Because of this you can’t change the report interval directly in the PerformanceGuard web interface. Contact a CapaSystems consultant if you want to change the report interval.

Note that the agent report interval isn’t that same as the agent delivery interval. The report interval determines how often the agent stores its collected data, whereas the delivery interval determines how often the agent sends the data to the PerformanceGuard system. Unlike the report interval, you can change the delivery interval setting directly in the PerformanceGuard web interface (ADMINISTRATION > Setup > Parameters, then select the Status tab).

From Detailed Data to Trend Data

When the PerformanceGuard system receives data from agents, it keeps the data in three different levels of aggregation:

• as detailed data
• as aggregated data (in a more summarized form)
• as trend data (in an even more summarized form)

Keep periods determine how long data is kept on each of the three levels. The more summarized the data is, the longer it’s kept. You can adjust the keep periods to suit your organization’s exact needs.
PerformanceGuard keeps the detailed, aggregated and trend data side-by-side, not in a waterfall mode where data is summarized and transferred to the next level of aggregation at the end of each keep period:

Important

Why is it good to have aggregation levels side-by-side?  Because PerformanceGuard keeps the three aggregation levels side-by-side, it’s able to respond quickly even if you want to view trend data from a recent period of time, for example the last three days. If PerformanceGuard used the waterfall approach, it would have to go through all the detailed data (which typically has a resolution of one minute) and then remove a lot of unnecessary data in order to meet your trend data request (for which you only require a resolution of one hour) before it could dsiplay the data.

Let’s take a closer look at the three aggregation levels:

•  Detailed data: Detailed data is kept in the form in which it was received from agents, that is with a resolution that corresponds to the report interval (see Agent Performs the Very First Aggregation in the previous). Data from individual agents is stored by agent ID, so you can view data from individual computers. Detailed data is by default kept for 5040 minutes (that is a keep period of three and a half days), with the exception of protocol transaction data which is by default kept for 10080 minutes (that is a keep period of one week).

Important

Can I change how long detailed data is kept?  Yes, but bear in mind that keep periods for detailed data are specified per Manage Agent Configuration Groups: In the PerformanceGuard web interface, select ADMINISTRATION > Agent Configuration > Configurations, then click the Edit link next to the required agent configuration group, and then select the Advanced tab, and specify required keep periods in minutes for the various types of data. You can only do this if you’re a PerformanceGuard administrator.

•  Aggregated data: The data is also stored summarized into aggregated data. Basically, PerformanceGuard removes information about individual agents so that data is kept for groups of agents rather than for individual agents. Agents are typically grouped based on the subnets that computers with PerformanceGuard agents are connected to. Because most computers at a location are typically on the same network, the agent groups will often reflect your organization’s locations. Even though aggregated data is kept for groups of agents rather than for individual agents, data is still kept with a resolution that corresponds to the report interval (see Agent Performs the Very First Aggregation in the previous). Aggregated data is by default kept for 1488 hours, that is a keep period of two months.

Important

Can I change how long aggregated data is kept?  Yes. In the PerformanceGuard web interface, select ADMINISTRATION > Setup > Parameters, then select the Status tab and adjust the KEEP_PERIOD_AGGREGATE_DATA setting. You can only do this if you’re a PerformanceGuard administrator.

Tip

While PerformanceGuard automatically creates agent groups based on the subnets that computers with PerformanceGuard agents are connected to, you can manually create more groups as required. A computer can easily be a member of multiple groups, so you can create your own groups for computers at specific locations or for computers across locations, for example a group for all computers that use a particular ERP application. Read more in Grouping of Computers.

•  Trend data: The data is also stored even further summarized into trend data. Trend data is summarized per hour. With trend data you lose some amount of detail, but you are still able to identify trends in performance over time—hence the name. Trend data is by default kept for 470 days, that is a keep period of a little less than one and a half years.

Important

What happens at the end of the trend data keep period? Data that has exceeded the trend data keep period is deleted.

Important

Can I change how long trend data is kept?  Yes. In the PerformanceGuard web interface, select ADMINISTRATION > Setup > Parameters, then select the Status tab and adjust the KEEP_PERIOD_TREND_DATA setting. You can only do this if you’re a PerformanceGuard administrator.

Caution

PerformanceGuard uses more keep periods than those mentioned here. To view the full list of keep periods in the PerformanceGuard web interface, select ADMINISTRATION > Setup > Parameters, then select the Status tab.

Special Keep Periods for Custom Counters

[[Custom counters]] have their very own keep periods. You can specify five different keep periods for custom counters to suit the exact needs of your organization:

• A keep period for data with a resolution of less than 60 seconds. By default PerformanceGuard will keep such data for 120 hours (that is five days).
• A keep period for data with a resolution of one minute. By default PerformanceGuard will such keep data for 240 hours (that is ten days).
• A keep period for data with a resolution of 15 minutes. By default PerformanceGuard will keep such data for 720 hours (that is 30 days).
• A keep period for data with a resolution of one hour. By default PerformanceGuard will keep such data for 1440 hours (that is 60 days).
• A keep period for data with a resolution of one day. By default PerformanceGuard will keep such data for 17520 hours (that is a little less than two years).

To change the keep periods for custom counters, select ADMINISTRATION > Setup > Parameters, then select the Status tab and adjust the settings KEEP_PERIOD_CUSTOM_COUNTER_1 to KEEP_PERIOD_CUSTOM_COUNTER_5. You can only do this if you’re a PerformanceGuard administrator.

Important

What if I don’t need so many keep periods for my custom counters? In that case simply specify a value of 0 for the keep periods that you don’t require. You can only do this if you’re a PerformanceGuard administrator.