Setting up and Using Rolling Baselines

Consistent response times and service levels have become increasingly important for Web systems. It is the nature of Internet that performance can vary significantly over short periods of time. These variations may represent random changes in the number of client requests, bandwidth usage from file downloads, or they may indicate a systemic problem in the infrastructure. In order to better document service level and response performance it is sometimes useful to consider a set of previous performance metrics from a monitor to establish expected performance ranges. When the monitor's performance values vary beyond a certain range the monitor can signal an error or warning. The activity involved in determining the expected performance characteristics is called baselining.

Sitescope includes two baselining options. One is the Rolling Baseline discussed here which is currently only applicable to the URL and NT Performance Counter Monitors. The other option is the Static Performance Baseline available for most monitor types.

This section describes:

About Rolling Baselines
Enabling Rolling Baselines
Disabling Rolling Baselines

About Rolling Baselines

In contrast to the Fixed Performance Baseline, which is calculated one time over an interval measured in days, the Rolling Baseline is calculated for an interval measured in monitor runs and the baseline value is updated after each monitor run. The URL Monitor and NT Performance Counter Monitors are distinguished from other monitors by the addition the Baseline Interval configuration parameter and rolling baseline related criteria in the error, warning, and good threshold settings. The steps for enabling the Rolling Baseline are described below.

Baselining a URL Monitor and NT Performance Counter Monitor involves using monitor performance measurements over a given interval to calculate and update a performance average and a standard deviation. The average establishes the baseline and when the performance reported by the monitor exceeds that baseline, either by a multiple of the standard deviation or a percentage difference from the baseline, the monitor may be considered in error.

The acceptable performance range of a monitor is determined by how far the current performance is from the baseline (deviation). This range of the variance is based on percentage deviation from the baseline average. For example, suppose a baseline for a monitor's round-trip time is currently 2 seconds and the percentage deviation is set for 30%. The monitor will remain in a good condition as long as it is round-trip time does not exceed 2.6 seconds.

Once a rolling baseline has been enabled and calculated, the text: (rolling baseline) is added to the status string of the monitor. This can be viewed in the group detail page to which the monitor belongs. It can also be used as a filter criteria for Match Status box in the Monitor Browser.

Rolling Baseline values are maintained for each monitor across SiteScope restarts.

Enabling Rolling Baselines

There are two main actions that need to be done to setup and use SiteScope Rolling Baselines. These are summarized as follows:

Enable rolling baseline calculation for selected monitors
Edit the configuration settings for the baselined monitors to make use of the baseline data.

once you have configured monitors to use a rolling baseline the monitors will automatically begin reporting status relative to the baseline data after they have run enough times to accumulate the required amount of data for the baseline calculation. Use the following steps to enable Rolling Baselines.

To enable Rolling Baselines for URL monitor or NT Performance Counter monitors

Create or select a URL monitor or NT Performance Counter monitor to be baselined.
In the Add Monitor page or the Edit Monitor page, scroll down the Advanced Options section to find the Baseline Interval box.
Enter an interval value in the Baseline Interval box. The Baseline Interval is the number of monitor runs that should be used to calculate the Rolling Baseline. Valid entries for this setting are positive integers. For example, entering a value of 20 will instruct SiteScope to use an interval of 20 previous monitor runs to calculate the rolling baseline for this monitor.
Edit the error, warning, and good threshold ranges for the monitor to be calculated relative to the baseline data. Use the drop down selection menu to the right of these settings to select the parameter. This will usually be a Deviation Percentage setting. Select the comparison operator and the comparison value for each threshold.
Click the Add or Update button to save the changes and enable the rolling baseline.

After enabling the rolling baseline, allow the monitor(s) to run for a period at least as long as the Baseline Interval you have entered. For example, if you entered a baseline interval of 10, the monitor needs to run ten times in order to accumulate enough run data to calculate the baseline.

After accumulating monitoring data for the indicated baseline interval period, the selected monitors will report their status relative to baseline. The baseline value will be recalculated and updated for each subsequent monitor run.

Disabling Rolling Baselines

You disable the use of Rolling Baselines by editing the applicable monitors. This involves clearing the Baseline Interval value and changing the Error if, Warning If, and Good if settings to use non-baseline related parameters. Use the following steps to disable baseline averaging for a monitor

To disable Rolling Baseline calculation for a monitor

Navigate to the group that contains the monitor(s) that have been enabled for rolling baseline calculation by clicking on the group name in the SiteScope main page or using the Browse Monitors page.
Click the Edit link for the applicable monitor in the group detail page. The Edit Monitor page for that monitor is displayed.
Scroll down to the Advanced Options section of the monitor setup screen and clear the value in the Baseline Interval text box.
Edit the error, warning, and good threshold ranges for the monitor to be set to something other than baseline related data.
Click the Update button to save the changes