SiteScope User's Guide

URL Sequence Monitor

The SiteScope URL Sequence Monitor simulates a user's access across several pages. This is particularly useful for monitoring and testing multi-page e-commerce transactions and other interactive online applications. For example, you can instruct SiteScope retrieve a login page, enter an account name via a secure Web form, check an account status for the page that is returned, and then follow a sequence of links through several more pages. URL Sequence Monitors are also useful for checking pages that include dynamically generated information, such as session IDs, that are embedded in the Web pages via dynamic links or hidden input items.

This section describes:

• Usage Guidelines
• Using the Add URL Sequence Wizard
  • Beginning a New URL Sequence Monitor
  • Defining the Next Step of the URL Sequence
• Completing the URL Sequence Monitor Form
  • Advanced Options

A URL Sequence begins with a specific URL acting as the starting point for the sequence. This can then be followed by additional URLs, or more commonly, links or form buttons that a user would be required to select in order to complete a specific transaction. By default, SiteScope allows you to define up to twenty sequence steps. For each step you may specify a content match or error string to search for, enter a user name and password if required, define custom POST data, as well as other optional criteria for that step.

See the primer "Web Technologies and URL Sequences" for an overview of the most widely used technologies for supporting transactions.

Usage Guidelines

URL Sequence Monitors provide you with end-to-end verification that multiple-page transactions are working properly. This is especially important for e-commerce sites and sites that give users the ability to complete forms or perform searches on-line. You can use the URL Sequence Monitor Tool to see what is returned at every step of a URL sequence, making trouble-shooting easier to complete.

What to monitor

You should monitor any multi-step sequence system that you have made available to general users because you are much less likely to hear about problems with these applications. Web site visitors often assume that any problems they encounter are due to user error rather than system error, especially if they're not familiar with your application. By using this monitor to perform sequence testing, you will know that users are able to successfully complete transactions.

Status

The status reading is the current value of the monitor. Since the monitor executes multiple steps, if an error is detected, the step on which the error occurred is included with the status message. Possible status values for the URL Sequence Monitor include:

• OK
• unknown host name
• unable to reach server
• unable to connect to server
• timed out reading
• content match error
• document moved
• unauthorized
• forbidden
• not found
• proxy authentication required
• server error
• not implemented
• server busy

If the status returned is good or OK, the total time for the sequence, the number of steps completed, and an indication of the data transferred will be displayed. If an error is returned during the sequence, the URL in the sequence where the problem occurred is listed as part of the reading.

Using the Add URL Sequence Wizard

Use the Add URL Sequence Wizard to quickly and easily create a URL Sequence Monitor. The wizard automatically opens when you click the Add URL Sequence Monitor link on the SiteScope Add Monitor page. It guides you through creating and testing a URL Sequence Monitor. When you are finished, click the Add Monitor button to exit the wizard and add the new monitor to SiteScope.

Beginning a New URL Sequence Monitor

1. Enter the starting URL for the sequence in the Other text box under the Enter the initial URL address selection button box for Step 1.
2. Use the Update Every options to specify how often you want SiteScope to run this monitor.
3. Type a descriptive title in the Title text box.
4. Click the Add Step button.

The URL Sequence Wizard form will refresh, showing whether or not SiteScope was able to complete the first step of the sequence. The Step and its related action or URL is displayed in a list form at the top of the form. The HTTP response header and the content of the URL are appended to the bottom of the page. If SiteScope was unable to complete the step, an error screen will be displayed with information about the error. Click the Back button in the browser window to return to the first step form. Check the form for errors, make corrections as needed, and then click the Add Step button to continue.

Note: Using the Back 1 step button causes SiteScope to reset all information for the current steps, allowing you to start over from the last successful step. Do not use this button as a method to view previously successful steps.

If the first page of this sequence requires that a User ID and Password be entered, or if you want to check for specific content on this page, scroll down to the Advanced Options section of the wizard form and enter that data in the appropriate items. Read more about how to complete these items in the next section, titled Completing the URL Sequence Monitor Form.

When you have filled in the necessary information to complete the first step of the sequence, click the Add Step button in the upper portion of the form. SiteScope will run the monitor to check the step defined and then display the form for adding the next step to the sequence.

Defining the Next Step of the URL Sequence

When you have completed the first step successfully, you are ready to move on to the subsequent steps. This will be a repetitive process depending on the number of Web pages and actions that need to be taken to complete the sequence. The Add URL Sequence Wizard makes this easy by automatically showing you the available sequence related elements on the current Web page. This includes buttons, hyperlinks, form elements, and so forth. You will use the Wizard to create each subsequent sequence step separately. Most sequence steps involve one of the following elements:

• Selecting a Hyperlink (including image maps)
• Selecting a Form Button
• Selecting a Frame within a Frameset
• Following a META REFRESH Redirection
• Go to Another URL Manually

The figure below is an interactive example of the upper portion of the URL Sequence Wizard page. This is a hypothetical example created to illustrate how the different sequence elements are presented by the URL Sequence Wizard. The HTML FORM elements on this help page simulate the interaction of the actual sequence Wizard page within SiteScope.

Note: SiteScope does not parse or interpret embedded scripts or other client-side program code such as Javascript (ECMAscript). Web page content that is generated or controlled by client-side code will usually not appear in the URL Sequence Wizard. See the URL Sequences and Client-side Programs help page for more information on dealing with Web page scripts.

Selecting a Hyperlink

SiteScope parses the content of the URL in the current step and creates a list of hyperlinks that are found. This includes links that are part of an image map. Any links found on this page of the sequence can be displayed in the drop-down list box to the right of the "Link" radio selection button. Use the following steps to add a link step to the sequence:

1. Click the round selection button to the left of the list of links.
2. Click the arrow on the right of the box to display all available links and then click the link that you want SiteScope to follow. If you know a link is available on the subject page but it does not appear in the drop-down list, see the Note: below on client-side programs.
3. (Optional) See the Advanced Options for the step you are currently defining and enter any string match or user name and password information you want SiteScope to use.
4. Click the Add Step button to move to the next step.

Selecting a Form Button

SiteScope parses the content of the URL in the current step and creates a list of form elements of the type "Submit". If SiteScope finds any HTML forms on the current page of the sequence, they will be displayed in a drop-down list.

The listings are in the following format:

For example, the Search button on a company's search page might be listed as:

1. Click the round selection button to the left of the list of submit buttons.
2. Click the arrow on the right of the box to display all of the available form buttons and then click the one that you want SiteScope to select. If you know a form is available on the subject page but it does not appear in the drop-down list, see the Note: below on client-side programs.
3. Directly below the list of submit buttons is a box that contains a listing of all of the input items available for this page. Locate the one(s) that pertain to the submit button you selected and type the appropriate post data after the = sign. You may need to run through the sequence in a separate browser window to determine the format and expected values for the post data.
4. (Optional) See the Advanced Options for the step you are currently defining and enter any string match or user name and password information you want SiteScope to use.
5. Click the Add Step button to move to the next step.

Selecting a Frame within a Frameset

Complete the following steps if the initial URL for this sequence contains a FRAMESET and you need to access a hyperlink, form, or form button that is a page displayed in one of the frames to proceed with the sequence.

1. Click the round selection button to the left of the Frame text entry box.
2. Click the arrow on the right of the box to display all available filenames displayed in the current FRAMESET and then click the file that you want SiteScope to retrieve.

Following a META REFRESH Redirection

If the page for this step of the sequence is controlled by a <META HTTP-EQUIV="Refresh" CONTENT="timedelay; URL=filename.htm"> tag, you can instruct SiteScope to retrieve the specified file as the next step. This sort of construct is sometimes used for intro pages, splash screens, or pages redirecting visitors from an obsolete URL to the active URL.

1. Click the round selection button to the left of the Refresh text entry box.
2. Click the arrow on the right of the box to display all available Refresh filenames and then click the file that you want SiteScope to retrieve.

Go to Another URL Manually

Where the sequence uses the Common Gateway Interface (CGI) for data transmission between the client and the server, it may be useful to specify a particular URL and name-value pairs. You can enter the URL you want to request along with any name-value pairs needed to get to the next sequence step even if those values are available through some other page element (such as a form). This option also allows you to copy URL and CGI strings directly from the location or address bar of another browser client that you may be using to step through the sequence you are building.

Complete the following steps if you want to direct SiteScope to go to another URL that .

1. Click the round selection button to the left of the URL text entry box.
2. Type the URL you want SiteScope to go to in the text entry box which contains http://.
3. See the Advanced Options for the step you are currently defining and enter any string match or user name and password information you want SiteScope to use.
4. Click the Add Step button to move to the next step.

Displays HTML in a Floating Window.

Check this option within the Transaction Wizard to have a copy of the most recently retrieved HTML displayed in a separate window. This can be very helpful in constructing URL sequences as it gives you a visual correlation between the elements displayed in the Transaction Wizard and the text that appears in the user's browser. When you check this option, the HTML view window is displayed as part of the next step.

Note: For effective use of this feature it necessary to change the settings in the browser you use to view SiteScope. For Internet Explorer, you must change the Temporary Internet files settings to "Check for newer versions of stored pages: Every visit to the page". Use the menu bar and select Tools->Internet Options->General:Temporary Internet files->Settings.

Completing the URL Sequence Monitor Form

Once you have selected an element or action for the current step of the sequence, SiteScope sends the request to the specified server which returns the result. The result is usually another Web page which includes other sequence elements or text to confirm the progress of the sequence so far. Continue using the URL Sequence Wizard to select the next sequence action or element and continue until you have created the steps that will complete the sequence. It is important to build end-to-end sequence monitors to be sure that you will know if users will be able to complete transactions.

The following items are also included on the URL Sequence Wizard form:

Update every

Within the Add URL Sequence Wizard, only the value entered for the final step of the sequence is remembered. The value is applied to the entire monitor.

Title

Enter the text for the title of this monitor that will be displayed in the SiteScope Monitor Detail Table. Within the Add URL Sequence Wizard, only the value entered for the final step of the sequence is remembered. You can change the title after adding the monitor via the Edit URL Sequence feature.

Advanced Options

The Advanced Options section presents a number of ways to customize monitor behavior and display. Use this section to customize error and warning thresholds, disable the monitor, set monitor-to-monitor dependencies, customize display options, and enter other monitor specific settings required for special infrastructure environments. The options for this monitor type are described below. Complete the entries as needed and click the Add or Update button to save the settings.

Disable

Check this box to temporarily disable this monitor and any associated alerts. To enable the monitor again, clear the box.

Like the sequence step type-reference pairs displayed in the upper portion of the form, the Advanced Options section displays the list of options for each step. Each step includes:

1. Step n POST Data
2. Step n Match Content
3. Step n Error If Match
4. Step n User Name
5. Step n Password
6. Step n Delay
7. Step n Title

The following describes these options:

Step n POST Data

If the URL at this step is for a POST request, enter the post variables, one per line as name=value pairs. This option is used to verify that a form is working correctly by performing the same request that occurs when a user submits a form. This item is most commonly used with the Form Reference Type. When the form is submitted, SiteScope fills in any items that are not specified with data here with the same defaults as a browser would have chosen.

Step n Match Content

Enter a string of text to check for in the returned page for this step. If the text is not contained in the page, the monitor will display no match on content for this step's URL. The search is case sensitive. Remember that HTML tags are part of a text document, so include the HTML tags if they are part of the text you are searching for (for example, "<B> Hello</B> World"). You may also perform a Perl regular expression match by enclosing the string in forward slashes, with an i after the trailing slash indicating case-insensitive matching (for example, /href=Doc\d+\.html/ or /href=doc\d+\.html/i).

Retaining and Passing Values Between Sequence Steps

One important feature of the Match Content capability in SiteScope URL Sequence Monitor is the ability to match, retain and then reference values from one URL sequence step for use as input into a request of a subsequent step. Using one or more sets of parentheses as part of a Match Content regular expression instructs SiteScope to "remember" the values matched by the pattern inside the parentheses. These values can then be referenced using the syntax described in the following example:

Example

Suppose you create a URL Sequence Monitor and include a Match Content expression for the first step to capture some session information. The Step 1 Match Content expression could be in the form of

/[\w\s]*?(pattern1)[\/\-\=]*?(pattern1)/

The two sets of parentheses in this expression instruct SiteScope to retain the two values matched by pattern1 and pattern2. To use these values as input to the next step in the URL sequence, use the syntax {$valuenum}. In this example, the string {$1} references the value matched by pattern1 and {$2} will reference the value matched by pattern2.

Use the above syntax for passing the referenced values to the URL sequence step immediately following the step in which the content match was made (step 1 to step 2 in our example). You can retain and pass matched values from one step to any other subsequent step by using a compound syntax of {$$stepnum.valuenum}. If, in our example, you want to use the value matched by pattern1in step 1 as input in a FORM or URL request in step 4 of the URL sequence, you would include the syntax {$$1.1} in Step 4. To reference the value matched by pattern1, use the {$$1.2} syntax.

Step n Error If Match

Enter a string of text to check for in the returned page for this step. If the text is contained in the page, the monitor will display no match on content for this step's URL. The search is case sensitive. Remember that HTML tags are part of a text document, so include the HTML tags if they are part of the text you are searching for (for example, "<B> Error </B> Message"). You may also perform a Perl regular expression match by enclosing the string in forward slashes, with an i after the trailing slash indicating case-insensitive matching. (for example, /href=Doc\d+\.html/ or /href=doc\d+\.html/i).

Step n User Name

If the URL specified at this step requires a name and password for access, enter the name in this box.

Step n Password

If the URL specified at this step requires a name and password for access, enter the password in this box.

Step n Delay

(Optional) Enter how long SiteScope should wait before executing the next step of the sequence.

Step n Title

(Optional) Enter the text for the title of this step within the sequence monitor. The title will only be displayed in the Edit URL Sequence form.

Advanced Options

The advanced options for each step of a URL sequence are listed in sequential groups. The following advanced options apply to the entire transaction.

Verify Error

Check this box if you want SiteScope to automatically run this monitor again if it detects an error. When an error is detected, the monitor will immediately be scheduled to run again once.

Note: In order to change the run frequency of this monitor when an error is detected, use the Update every (on errors) option below.

Note: The status returned by the Verify Error run of the monitor will replace the status of the originally scheduled run that detected an error. This may cause the loss of important performance data if the data from the verify run is different than the initial error status.

Warning: Use of this option across many monitor instances may result in significant monitoring delays in the case that multiple monitors are rescheduled to verify errors at the same time.

Update Every (on error)

You use this option to set a new monitoring interval for monitors that have registered an error condition. For example, you may want SiteScope to monitor this item every 10 minutes normally, but as often as every 2 minutes if an error has been detected. Note that this increased scheduling will also affect the number of alerts generated by this monitor.

Schedule

By default, SiteScope monitors are enabled every day of the week. You may, however, schedule your monitors to run only on certain days or on a fixed schedule. Click the Edit schedule link to create or edit a monitor schedule. For more information about working with monitor schedules, see the section on Schedule Preferences for Monitoring.

Monitor Description

Enter additional information about this monitor. The Monitor Description can include HTML tags such as the <BR> <HR>, and <B> tags to control display format and style. The description will appear on the Monitor Detail page.

Report Description

Enter an optional description for this monitor that will make it easier to understand what the monitor does. For example, network traffic or main server response time. This description will be displayed on with each bar chart and graph in Management Reports and appended to the tool-tip displayed when you pass the mouse cursor over the status icon for this monitor on the monitor detail page.

Timeout

The number of seconds that the URL Sequence Monitor should wait for the entire sequence to complete before timing-out. Once this time period passes, the URL Sequence Monitor will log an error and report an error status.

Timeout is per Step

Check this box if you want to use the value entered for the Timeout above as the Timeout for each step of the sequence rather than for the entire transaction. If the step takes more than this time to complete, the URL Sequence Monitor will log an error and report an error status.

HTTP Proxy

Optionally, a proxy server can be used to access the URLs in the sequence. Enter the domain name and port of an HTTP Proxy Server.

Retrieve Images

Check this box if you want the status and response time statistics to include the retrieval times for all of the embedded images in the page. Embedded images include those referenced by "IMG" and "INPUT TYPE=IMAGE" HTML tags. Images that appear more than once in a page are only retrieved once.

Note: If the Retrieve Images option is checked, each image referenced by the target URL will contribute to the download time. However, if a image times out during the download process or has a problem during the download, that time will not be added to the total download time.

Retrieve Frames

Check this box if you want SiteScope to retrieve the all frames references in a frameset and count their retrieval time in the total time to download this page. If Retrieve Images is also checked, SiteScope will attempt to retrieve all images in all frames.

Note: If the Retrieve Frames option is checked, each frame referenced by the target URL will contribute to the download time. However, if a frame times out during the download process or has a problem during the download, that time will not be added to the total download time.

Proxy Server User Name

If the proxy server requires a name and password to access the URLs in the sequence, enter the name here. Technical note: your proxy server must support Proxy-Authenticate for these options to function.

Proxy Server Password

If the proxy server requires a name and password to access the URLs in the sequence, enter the password here. Technical note: your proxy server must support Proxy-Authenticate for these options to function.

NT Challenge Response

Check this box if you want SiteScope to use Window's NT Challenge Response when retrieving the URL in this step.

Resume at step n if error

You use this option to specify a URL sequence step to execute in the case that a URL Sequence results in an error. This is useful when a URL sequence involves a user or customer login which would result in problems if the sequence were aborted without logging out. Use the drop-down list to select a URL sequence step to jump to in the case that any step in the sequence returns an error.

Execute resume step and remaining steps

If the Resume at step is option is selected and executed, selection of this option causes SiteScope to execute that step and continue executing the other, subsequent steps until it reaches the end of the sequence.

Show Detailed Measurement

Check this box if you want SiteScope to record a detailed break down of the process times involved in retrieving the requested URL. This includes DNS lookup, connect time, HTTP server response time, described as follows:

• DNS time - The time it takes to send a name resolution request to your DNS server until you get a reply.
• Connection time - The time it takes to establish a TCP/IP/Socket connection to the Web server.
• Response time - The time after the request is sent until the first byte (rather first buffer full) of the page comes back.
• Download time - The time it takes to download the entire page.

List Order

By default, new monitors are listed last on the Monitor Detail page. You may use this drop-down list to choose a different placement for this monitor.

Setting URL Sequence Monitor Status Thresholds

SiteScope URL monitor types allow you to set threshold conditions based on part or all of a Web page retrieval or transaction to determine the status reported by each monitor. Monitor status can be set based on the roundtrip, DNS, connect, or response times needed to negotiate a Web request or transaction.

The roundtrip time is one of the most commonly used metrics for URL monitoring. Roundtrip time includes the time to complete the following:

1. Name lookup (DNS)
2. Connect to the server socket
3. Send the HTTP request
4. Download the entire page

If the server hosting SiteScope is heavily loaded, either with other applications or with a high SiteScope monitoring load, then it can contribute to slower times, especially for HTTPS URLs, which are more CPU intensive. Usually though, the time waiting for the network is a much greater factor than the CPU usage.

Set the monitor status thresholds for each step of the URL sequence as described below.

Error if

By default, SiteScope generates an error if the returned HTTP status is anything other than 200 ("OK"), which indicates a successful retrieval. You can choose to have SiteScope report an error status based on any of the following measurements:

• round trip time - the total time for the entire request, in milliseconds
• DNS time - the amount of time to translate the host name to an IP address, in milliseconds
• connect time - the amount of time to make the connection, in milliseconds
• response time - the amount of time before the first response was received, in milliseconds
• download time - the amount of time to receive the page contents, in milliseconds
• age -- the amount of time between the current time and the last-modified time for the page, in seconds
• content match
• total errors
• overall status

Choose a comparison operator from the drop-down list, and enter a value for the comparison in the text box.

The URL Monitor follows HTTP redirect codes (301 and 302) to retrieve the actual page before returning the status of the URL retrieval. SiteScope will show a redirect error only if the redirects are more than 10 levels deep - this prevents infinite redirects from being followed, or if the Error On Redirect check box is selected.

Warning if

By default, SiteScope does not generate warnings for URL Sequence Monitors. You may choose to generate a warning based on round trip retrieval time. Enter the shortest retrieval time (in milliseconds) that should generate a warning.

Good if

You can base a good status on the round-trip time for any one step if you want. Select the step from the drop-down list and set the threshold.

Offline

Check this option to build the URL Sequence without actually making the HTTP request for each step. You can use this option to create a URL Sequence "offline" without verifying each step during creation. After you create the sequence and save or update the monitor, the monitor itself will run as normal and check that each URL can be retrieved and each step can be verified.

Each time that you define a URL sequence step and press the Add Step button, SiteScope reruns all of the defined steps in the sequence as well as the step you have just added.

When you have successfully defined each step in the sequence that you want SiteScope to monitor, click the Add Monitor button. The URL Sequence Wizard will close and the new monitor will appear on the Group Detail page. Click the Edit button on this page to make changes to the URL Sequence Monitor.

Copyright © 2004 Mercury Interactive Corporation.
All rights reserved.

Documentation comments or suggestions?
Please send feedback to documentation@merc-int.com