SiteScope User's Guide

URL Monitor



The SiteScope URL Monitor is one of the most versatile and powerful Web monitoring tools available to Webmasters. Its core function is to attempt to reach a specified Web page to verify that it can be retrieved, but it can also be used to do the following:

When the URL Monitor retrieves a Web page, it retrieves the page's contents. A successful page retrieval assures you that your Web server is functioning properly. The URL Monitor doesn't automatically retrieve any objects linked from the page, such as images or frames. You can, however, instruct SiteScope to retrieve the images on the page by selecting the Retrieve Images or Retrieve Frames box located in the Advanced Options section of the Add Url Monitor Form.

In addition to retrieving specific Web pages, the URL Monitor can help you verify that CGI scripts and back-end databases are functioning properly. Just input the complete URL used to retrieve data from your database or trigger one of your CGI scripts, and the URL monitor will verify that the script generates a page and returns it to the user. For example, you can verify that your visitors are receiving a thank you page when they purchase something off of your site. The URL monitor's string matching capability even allows you to verify that the contents of the page are correct.

Each time the URL Monitor runs, it returns a reading or status and writes it in the monitoring log file. It also writes in the log file the total time it takes to receive the designated document. This status value is also displayed in the SiteScope Monitor tables and is included as part of alert messages sent via e-mail.

Usage Guidelines

Why should I use this monitor?

The SiteScope URL Monitors provide you with end-to-end verification that your web server is running, serving pages correctly, and doing so in a timely manner. Because it tests end-to-end, it is also able to determine whether back-end databases are available, verify the content of dynamically generated pages, check for changed content, and look for specific values from a page.

What should I monitor?

We suggest that you create URL monitors to watch pages that are critical to your web site (such as your home page), pages that are generated dynamically, and pages that depend upon other applications to work correctly (such as pages that utilize a back-end database). Your goal is to monitor a sampling of every type of page you serve to ensure that everything is working properly. There's no need to verify that every page of a particular type is working correctly -- one or two will do.

When you're choosing which pages to monitor, try to pick pages with the lowest overhead. For example, if you have several page that are generated by another application, monitor the shortest one with the fewest graphics. This will put less load on your server while still providing you with the information you need.

How should I schedule my monitors?

Each URL Monitor puts no more load on your server than someone accessing your site and retrieving a page, so in most cases you can schedule them as closely together as you like. Keep in mind that the length of time between each run of a monitor is equal to the amount of time that can elapse before you're notified of a possible problem.

A common strategy is to schedule monitors for very critical pages to run every 1 to 2 minutes, and then schedule monitors for less critical pages to run only every 10 minutes or so. Using this strategy, you will be notified immediately if a critical page goes down or if the entire web site goes down, but you won't have an excessive number of monitors running all the time.

Status

The status reading shows the most recent result for the monitor. This status value is displayed in the URL Group table within SiteScope. It is also recorded in the SiteScope log files, e-mail alert messages, and can be transmitted as a pager alert. The possible status values are:

The status is logged as either good, warning, or error. A warning status or error status is returned if the current value of the monitor is a condition that you have defined as other than OK.

Completing the URL Monitor Form

To display the URL Monitor Form, either click the Edit link for an existing URL Monitor in a monitor table, or click the add a Monitor link on a group's detail page and choose the "Add URL Monitor" link.

Complete the items on the URL Monitor form as follows. When the required items are complete, click the "Add Monitor" button.

URL
Enter the URL that you want to monitor (for example, http://demo.thiscompany.com).

Note:for HTTPS monitoring (secure HTTP):

If the URL starts with HTTPS, then a secure connection will be made using SSL (for example, https://www.thiscompany.com). SiteScope uses Java SSL libraries for HTTPS monitoring. For SiteScope on NT, SiteScope uses the SSL library from Internet Explorer if you select the NT Challenge Response option. To monitor using SSL and NT Challenge Response, installing Internet Explorer 4.0 or later is recommended.

Match Content
Enter a string of text to check for in the returned page or frameset. If the text is not contained in the page, the monitor will display "no match on content". The search is case sensitive. Remember that HTML tags are part of a text document, so include the HTML tags if they are part of the text you are searching for (for example, "< B> Hello< /B> World"). This works for XML pages as well. You may also perform a Perl regular expression match by enclosing the string in forward slashes, with an "i" after the trailing slash indicating case-insensitive matching. (for example, "/href=Doc\d+\.html/" or "/href=doc\d+\.html/i"). If you want a particular piece of text to be saved and displayed as part of the status, use parentheses in a Perl regular expression. For example /Temperature: (\d+). This would return the temperature as it appears on the page and this could be used when setting an Error if or Warning if threshold.

Update every
Enter how frequently the monitor should check this URL. The drop-down list to the right of the text box lets you specify time increments of seconds, minutes, hours, or days. You must specify a time increment of at least 15 seconds.

Title (Optional)
Enter a name for this monitor. This name appears in the Name text box on the monitor table when you open the group's detail page. If you don't enter a name, a default name will be created.

Advanced Options

The advanced options give you the ability to customize error and warning thresholds, or define other optional settings. These includes:

Disable
Check this box to temporarily disable this monitor and any associated alerts. To enable the monitor again, clear the box.

Timeout
The number of seconds that the URL monitor should wait for a page to complete downloading before timing-out. Once this time period passes, the URL monitor will log an error and report an error status. If you have checked the Retrieve Frames or Retrieve Images option, SiteScope will wait for these items to be retrieved before considering the page to be fully downloaded.

HTTP Proxy
Optionally, a proxy server can be used to access the URL. Enter the domain name and port of an HTTP Proxy Server.

Retrieve Images
Check this box if you want the status and response time statistics to include the retrieval times for all of the embedded images in the page. Embedded images include those referenced by "IMG" and "INPUT TYPE=IMAGE" HTML tags. Images that appear more than once in a page are only retrieved once.

Retrieve Frames
Check this box if you want SiteScope to retrieve the all frames references in a frameset and count their retrieval time in the total time to download this page. If Retrieve Images is also checked, SiteScope will attempt to retrieve all images in all frames.

Error If Match
Enter a string of text to check for in the returned page or frameset. If the text is contained in the page, the monitor will indicate an error condition. The search is case sensitive. Remember that HTML tags are part of a text document, so include the HTML tags if they are part of the text you are searching for (for example, "< B> Error < /B> Message"). You may also perform a Perl regular expression match by enclosing the string in forward slashes, with an "i" after the trailing slash indicating case-insensitive matching. (for example, "/href=Doc\d+\.html/" or "/href=doc\d+\.html/i").

Check for Content Changes
Unless this is set to "no content checking" (the default) SiteScope will record a checksum of the document the first time the monitor runs and then does a checksum comparison each subsequent time it runs. If the checksum changes, the monitor will have a status of "content changed error" and go into error. If you want to check for content changes, you will usually want to use "compare to saved contents".

The options for this setting are:

Authorization User Name
If the URL specified requires a name and password for access, enter the user name in this box.

Authorization Password
If the URL specified requires a name and password for access, enter the password in this box.

NT Challenge Response
Check this box if you want SiteScope to use Window's NT Challenge Response authorization when retrieving this Web page.

Proxy Server User Name
If the proxy server requires a name and password to access the URL, enter the name here. Technical note: your proxy server must support Proxy-Authenticate for these options to function.

Proxy Server Password
If the proxy server requires a name and password to access the URL, enter the password here. Technical note: your proxy server must support Proxy-Authenticate for these options to function.

POST Data
If the URL is for a POST request, enter the post variables, one per line as name=value pairs. This option is used to verify that a form is working correctly by performing the same request that occurs when a user submits a form. See also the Match Content item for a way to verify that the correct form response was received. If this item is blank, a GET request is performed.

Advanced Option The POST Data can be used to send cookie data. To send cookies with the request use the format "Set-cookie: cookieName=cookieValue".
To change the content type of a post, use the format "Content-Type: application/my-format". To hide values in the POST data, add a line like: 

_private=_name=mysecret _value=rosebud
_private=_name=mypassword _privateValue=sesame
and then use the following form in the POST Data 
s|username=$private-mysecret$|
s|password=$private-mypassword$|
and SiteScope will substitute the values from the master.config into the POST Data.

Error If Redirected
Check this box if you want SiteScope to notify you if a URL is redirected. Normally, SiteScope follows redirects.

Show Detailed Measurement
Check this box if you want SiteScope to record a detailed break down of the process times involved in retrieving the requested URL. This includes DNS lookup, connect time, HTTP server response time, etc.

Baseline Interval
Enter the number of monitor runs to be averaged for use as a Rolling Baseline. Rolling baselines are calculated for an interval equal to time to complete the number of monitor runs entered here. See more information.


Verify Error
Check this box if you want SiteScope to automatically run this monitor again if it detects an error. When an error is detected, the monitor will immediately be scheduled to run again once.

Note: In order to change the run frequency of this monitor when an error is detected, use the Update every (on errors) option below.

Note: The status returned by the Verify Error run of the monitor will replace the status of the originally scheduled run that detected an error. This may cause the loss of important performance data if the data from the verify run is different than the initial error status.

Warning: Use of this option across many monitor instances may result in significant monitoring delays in the case that multiple monitors are rescheduled to verify errors at the same time.

Update Every (on error)
This option allows you to set a new monitoring interval for monitors that have registered an error condition. For example, you may want SiteScope to monitor this item every 10 minutes normally, but as often as every 2 minutes if an error has been detected. Note that this increased scheduling will also affect the number of alerts generated by this monitor.

Schedule (Optional)
By default, SiteScope's monitors are enabled every day of the week. You may, however, schedule your monitors to run only on certain days or on a fixed schedule. Choose the Edit schedule link to create or edit a monitor schedule. For information about creating schedules, read these instructions.

Monitor Description (Optional)
Enter additional information about this monitor. The Monitor Description can include HTML tags such as the <BR> <HR>, and <B> tags to control display format and style. The description will appear on the Monitor Detail page.

Report Description (Optional)
Enter a description for this monitor that will make it easier to understand what this monitor does. The description will appear on Management Reports and on the info list for a monitor.

Depends Condition (Optional)
If you choose to make the running of this monitor dependent on the status of another monitor, choose the status condition that the other monitor or monitor group should have in order for the current monitor to run normally. The current monitor will be run normally as long as the monitor or group on which it depends reports the condition selected in this option.

List Order (Optional)
By default, new monitors are listed last on the Monitor Detail page. You may use this drop-down list to choose a different placement for this monitor.

Error if
By default, SiteScope generates an error if the returned HTTP status is anything other than 200 ("OK"), which indicates a successful retrieval. You can choose to have SiteScope report an error status based on any of the following measurements:

Choose a comparison operator from the drop-down list, and enter a value for the comparison in the text box.

The URL Monitor follows HTTP redirect codes (301 and 302) to retrieve the actual page before returning the status of the URL retrieval. SiteScope will show a redirect error only if the redirects are more than 10 levels deep - this prevents infinite redirects from being followed, or if the Error On Redirect check box is selected.

Warning if
By default, SiteScope generates a warning if Check Images or Check Frames is chosen and there was a problem retrieving one of the images or frames. You may choose to have the monitor report a warning status based on any of the measurement options listed under Error if. Choose a comparison operator, and enter a value for the comparison to generate a warning.

Good if
By default, SiteScope reports an OK status if the URL returns an HTTP status of 200 ("OK"). You may also choose to have SiteScope base an OK status on any of the measurement options listed under Error if. Enter the value that should generate an OK status.




Copyright © 2003 Mercury Interactive Corporation.
All rights reserved.