URL Content Monitor

The SiteScope URL Content Monitor is a specialized variation of the URL Monitor that can match up to ten different values from the content of a specified URL. The matched values are displayed with the status of the monitor in the monitor group table and written to the monitor log.

The content values are matched using regular expressions.

Each time the URL Content Monitor runs, it returns a a status and several match values and writes them in the monitoring log file. It also writes the total time it takes to receive the designated document in the log file.

Usage Guidelines

The URL Content Monitor is primarily used to monitor Web pages that are generated dynamically and display statistics about custom applications. By monitoring these pages, these statistics can be retrieved and integrated into the rest of your SiteScope system.

What to monitor

You should use the URL Content Monitor if you need to verify multiple values (up to 10 variables) from the content of a single URL. Otherwise, the standard URL Monitor is normally used. One use of this monitor is to integrate SiteScope with other applications that export numeric data through a Web page. The monitor includes the matched values as part of the monitor status which are written to the log. If the matched values are numeric data, the results can be plotted in a management report.

About scheduling this monitor

The frequency will depend on the statistics being monitored. For more statistics, every several minutes is often enough.

Status

The reading is the current value of the monitor. Possible values are:

OK
unknown host name
unable to reach server
unable to connect to server
timed out reading
content match error
document moved
unauthorized
forbidden
not found
proxy authentication required
server error
not implemented
server busy

The status is returned as good, warning, or error dependent on the results of the retrieval, content match, and the error or warning status criteria that you select.

Completing the URL Content Monitor Form

To display the URL Monitor Form, either click the Edit link for an existing URL Content Monitor in a monitor table, or click the add a Monitor link on a group's detail page and click the Add URL Content Monitor link.

Complete the items on the URL Content Monitor form as follows. When the required items are complete, click the Add Monitor button.

URL: Enter the URL that you want to monitor (for example, http://demo.thiscompany.com). If you are monitoring a secure URL, be sure the URL reflects the correct transfer protocol (for example https://demo.thiscompany.com).
Match Content: Enter an expression describing the values to match in the returned page. If the expression is not contained in the page, the monitor will display "no match on content". A regular expression is used to define the values to match. For example, the expression /Copyright (\d*)-(\d*)/ would match two values, 1996 and 1998, from a page that contained the string Copyright 1996-1998
Update every: Select how often the monitor should check this URL. The default interval is to run or update the monitor once every 10 minutes. Use the drop-down list to the right of the text box to specify another update interval in increments of seconds, minutes, hours, or days. The update interval must be 15 seconds or longer.
Title: Enter a title text for this monitor. This text is displayed in the group detail page, in report titles, and other places in the SiteScope interface. If you do not enter a title text, SiteScope will create a title based on the host, server, or URL being monitored.

Advanced Options

The Advanced Options section presents a number of ways to customize monitor behavior and display. Use this section to customize error and warning thresholds, disable the monitor, set monitor-to-monitor dependencies, customize display options, and enter other monitor specific settings required for special infrastructure environments. The options for this monitor type are described below. Complete the entries as needed and click the Add or Update button to save the settings.

Disable

Check this box to temporarily disable this monitor and any associated alerts. To enable the monitor again, clear the box.

Timeout

The number of seconds that the URL monitor should wait for a page to begin downloading before timing-out. Once this time period passes, the URL monitor will log an error and report an error status.

HTTP Proxy

Optionally, a proxy server can be used to access the URL. Enter the domain name and port of an HTTP Proxy Server.

Retrieve Images

Check this box if you want the status and response time statistics to include the retrieval times for all of the embedded images in the page. Embedded images include those referenced by "IMG" and "INPUT TYPE=IMAGE" HTML tags. Images that appear more than once in a page are only retrieved once.

Note: If the Retrieve Images option is checked, each image referenced by the target URL will contribute to the download time. However, if a image times out during the download process or has a problem during the download, that time will not be added to the total download time.

Retrieve Frames

Check this box if you want SiteScope to retrieve the all frames references in a frameset and count their retrieval time in the total time to download this page. If Retrieve Images is also checked, SiteScope will attempt to retrieve all images in all frames.

Note: If the Retrieve Frames option is checked, each frame referenced by the target URL will contribute to the download time. However, if a frame times out during the download process or has a problem during the download, that time will not be added to the total download time.

Error If Match

Enter a string of text to check for in the returned page. If the text is contained in the page, the monitor will display "no match on content". The search is case sensitive. Remember that HTML tags are part of a text document, so include the HTML tags if they are part of the text you are searching for (for example, "< B> Error < /B> Message"). You may also perform a Perl regular expression match by enclosing the string in forward slashes, with an "i" after the trailing slash indicating case-insensitive matching. (for example, "/href=Doc\d+\.html/" or "/href=doc\d+\.html/i").

Check for Content Changes

Check this box if you want SiteScope to notify you if the contents of this document are changed. SiteScope records a checksum of the document the first time the monitor runs and then does a checksum each subsequent time it runs. If the checksum changes, you will be notified. The new checksum will be recorded as the default after the initial error has been issued.

Authorization User Name

If the URL specified requires a name and password for access, enter the name in this box.

Authorization Password

If the URL specified requires a name and password for access, enter the password in this box.

NT Challenge Response

Check this box if you want SiteScope to use Window's NT Challenge Response authorization when retrieving this Web page.

Proxy Server User Name

If the proxy server requires a name and password to access the URL, enter the name here. Technical note: your proxy server must support Proxy-Authenticate for these options to function.

Proxy Server Password

If the proxy server requires a name and password to access the URL, enter the password here. Technical note: your proxy server must support Proxy-Authenticate for these options to function.

POST Data

If the URL is for a POST request, enter the post variables, one per line as name=value pairs. This option is used to verify that a form is working correctly by performing the same request that occurs when a user submits a form. See also the Match Content box for a way to verify that the correct form response was received. If this item is blank, a GET request is performed.
Advanced: This item can also be used to pass cookies with the request. For example, "Set-cookie: cookieName=cookieValue".

Error If Redirected

Check this box if you want SiteScope to notify you if a URL is redirected. Normally, SiteScope follows redirects without reporting an error.

Show Detailed Measurement

Check this box if you want SiteScope to record a detailed break down of the process times involved in retrieving the requested URL. This includes DNS lookup, connect time, HTTP server response time, described as follows:

DNS time - The time it takes to send a name resolution request to your DNS server until you get a reply.
Connection time - The time it takes to establish a TCP/IP/Socket connection to the Web server.
Response time - The time after the request is sent until the first byte (rather first buffer full) of the page comes back.
Download time - The time it takes to download the entire page.

Verify Error

Check this box if you want SiteScope to automatically run this monitor again if it detects an error. When an error is detected, the monitor will immediately be scheduled to run again once.

Note: In order to change the run frequency of this monitor when an error is detected, use the Update every (on errors) option below.

Note: The status returned by the Verify Error run of the monitor will replace the status of the originally scheduled run that detected an error. This may cause the loss of important performance data if the data from the verify run is different than the initial error status.

Warning: Use of this option across many monitor instances may result in significant monitoring delays in the case that multiple monitors are rescheduled to verify errors at the same time.

Update Every (on error)

You use this option to set a new monitoring interval for monitors that have registered an error condition. For example, you may want SiteScope to monitor this item every 10 minutes normally, but as often as every 2 minutes if an error has been detected. Note that this increased scheduling will also affect the number of alerts generated by this monitor.

Schedule

By default, SiteScope monitors are enabled every day of the week. You may, however, schedule your monitors to run only on certain days or on a fixed schedule. Click the Edit schedule link to create or edit a monitor schedule. For more information about working with monitor schedules, see the section on Schedule Preferences for Monitoring.

Monitor Description

Enter additional information about this monitor. The Monitor Description can include HTML tags such as the <BR> <HR>, and <B> tags to control display format and style. The description will appear on the Monitor Detail page.

Report Description

Enter an optional description for this monitor that will make it easier to understand what the monitor does. For example, network traffic or main server response time. This description will be displayed on with each bar chart and graph in Management Reports and appended to the tool-tip displayed when you pass the mouse cursor over the status icon for this monitor on the monitor detail page.

Depends Condition

If you choose to make the running of this monitor dependent on the status of another monitor, select the status condition that the other monitor or monitor group should have in order for the current monitor to run normally. The current monitor will be run normally as long as the monitor on which it depends reports the condition selected in this option.

List Order

By default, new monitors are listed last on the Monitor Detail page. You may use this drop-down list to choose a different placement for this monitor.

List Order

By default, new monitors are listed last on the Monitor Detail page. You may use this drop-down list to choose a different placement for this monitor.

Error if

By default, SiteScope generates an error if the returned status is anything other than 200, which indicates a successful retrieval. You may choose to have SiteScope generate an error based on any of the following:

tenth content match
content match
age
second content match
third content match
fourth content match
fifth content match
sixth content match
seventh content match
eighth content match
ninth content match
download time
connect time
response time
size
dns time
round trip time

At present, content match values used for error or warning generation must be numeric.

The URL Content Monitor follows redirect codes (301 and 302) to retrieve the actual page before returning the status of the URL retrieval. SiteScope will show a redirect error only if the redirects are more than 10 levels deep - this prevents infinite redirects from being followed, or if the Error On Redirect check box is selected..

Warning if

By default, SiteScope does not generate warnings for URL Content monitors. You may choose to generate a warning based any of the options listed under Error if. Enter the lowest value that should generate a warning.

Good if

By default, SiteScope returns an OK status if a 200 status is returned, but you may choose to base an ok status on any of the options listed under Error if. Enter the value that SiteScope should consider to be a good response.