Setting up and Using Static Performance Baselines

Often it is desirable to use past performance metrics from a monitor to establish acceptable or expected performance ranges. When the monitor's performance exceeds that range by some value the monitor can signal an error or warning. The activity involved in determining the expected performance characteristics is called baselining.

Sitescope includes two baselining options. One is the Static Performance Baseline discussed here. The other option is the Rolling Baseline available with the URL Monitor and NT Performance Counter Monitor.

This section describes:

About Static Performance Baselines
Understanding Performance Baselining
Enabling Static Performance Baselining for Monitors
Setting Monitor Thresholds Using Baselines
Monitor Properties for Static Baselines

About Static Performance Baselines

Static Performance Baseline feature calculates a one-time, or fixed baseline value based on monitor readings for a time interval you specify. The units of the time interval are expressed in days. Once enough monitor data has been accumulated to calculated the baseline, subsequent monitor results are compared to that baseline value. The baseline value remains constant or static.

Monitors selected for static performance baselining are distinguished from other monitors of the same type by the addition of new, baseline related criteria in the error, warning, and good thresholds for the monitors which have been configured for baselining.

Once a baseline has been calculated, the text: (static baseline, refvalue) is also added to the status string of the monitor. The refvalue displayed is a reference value of the baseline mean value. The meaning of the refvalue will vary depending on the type of monitor (see the table of monitor types below). The status string can be viewed in the group detail page to which the monitor belongs. It can also be used as a filter criteria for Match Status box in the Monitor Browser. The specific values used for the baseline is added to the Advanced Options section of the individual monitor set up page as described below. You use the Edit link associated with the monitor to view this data.

Understanding Performance Baselining

Baselining a monitor involves using the historical performance measurements to calculate a performance average (that is, baseline) and a standard deviation. The average establishes the baseline and when the performance the monitor exceeds that baseline, either by a multiple of the standard deviation or a percentage difference from the baseline, the monitor may be considered in error.

The acceptable performance range of a monitor is determined by how far (variance) the current performance is from the baseline. This range of the variance may be based on multiples of the standard deviation or a multiples of a percentage difference. For example, suppose a baseline for a monitor's round-trip time is 3 seconds and the standard deviation is 1.5 seconds. If the range for variance is selected as one times the standard deviation, then the monitor will remain in a good condition as long as it is round-trip time does not exceed 4.5 seconds. This can be represented as a monitor error condition formula of:

if(round-trip > (baseline + (multiplier * stddev)) then error = true

Substituting the numbers from the example above gives:

if(4.5 > (3 + (1 * 1.5)) then error = true

To take another example. Suppose the baseline for a monitor's round-trip time is 5 seconds. Also suppose the range for variance is selected as 50 percent of the baseline. The monitor will remain in a good condition as long as it is round-trip time does not exceed 7.5 seconds. This can be represented as a monitor error condition formula of:

if(round-trip > baseline + (baseline * multiplier)) then error = true

Substituting the numbers from this second example above gives:

if(7.5 > (5.0 + (5.0 * 0.50)) then error = true

Enabling Static Performance Baselining for Monitors

There are several steps required to setup and use SiteScope Fixed Performance Baselining. These are summarized as follows:

Enable static baselining for selected monitors
Allow the monitors to accumulate data for the baseline period
Edit the configuration settings for the baselined monitors to make use of the baseline data.

A baseline may be set, reset, or extended whenever necessary to update performance expectations based on increased operational experience or to document the effects of infrastructure changes. Once a Static Performance Base has been calculated, it can only be updated by selecting the same monitor(s) and repeating the baseline setup steps.

Use the following steps to enable baselining of monitors with this feature. These steps assume that you have already created one or more monitors that you want to baseline.

To enable static performance baselines for monitors

Click the Manage Monitors/Groups link on the SiteScope navigation menu. The Manage Monitors and Groups page is displayed.
Use the expandable tree menu to select one or more monitors to be baselined. Click the check box to the left of the monitor name in the group and monitor tree menu.
Scroll to the bottom section of the Manage Monitors and Groups page and click the Baseline button. The Baseline Monitors page is displayed.
The monitors you selected for baselining are displayed in the Monitors table list. Enter the baseline data interval you want to use for these monitors in the text box labeled Compute a fixed performance baseline using data from the last N days. The interval should be a positive integer representing a number of days.
Click the Baseline button to enable baselining for the selected monitors.

After you have selected and enabled monitors for baselining, you should allow the monitors to run for a period at least as long as the baseline period you want to use. For example, if you set the baseline interval as 2 days, you can edit the configuration settings for the monitors for which you have enabled baselining to make use of the baseline data.

Setting Monitor Thresholds Using Baselines

Once the baseline has been setup for monitors, each monitor must be individually edited to set the error threshold range from the Edit Monitor page. When a baselined monitor is edited, a area on the page named Baseline Info is displayed that shows the date the baseline was taken, the baseline value (that is, average) and the calculated standard deviation. Also, each baselined monitor then includes the following two thresholds in the Error if, Warning if, and Good if drop-down lists:

# std dev from baseline
% difference from baseline.

The following is an example of the threshold setting section on the Edit Monitor page for a URL monitor for which a static baseline has been created. The Baseline Info is displayed above the Error if box.

Baseline Info

Baseline taken on: 9:09 AM 10/17/03. Average:2.37 sec. StdDev:n/a

Error if