Regression Testing with URL-Hammer

You can record a set of user interactions with the site and use this record (a script) for regression testing, performance benchmarking or to simulate multi-user loads.

Making the Recording
The Recorded Script
Recorded Statistics
Replaying the Script
Regression Testing

Making the recording

Start Dynamo
Open the recording servlet in the admin UI, which is usually here:
http://localhost:8830/nucleus/atg/dynamo/servlet/pipeline/RecordingServlet
(There are no links in the Admin UI to this URL)
Visiting this page starts the component if its was not started by default
Look at the recordFile property. This is where the log script will be saved, relative to dynamo home. The default file location is C:\ATG\Dynamo5.5\home\logs\record.log
Check the output filename and ensure that the log file does not already exist.
Set the recording property to true here
Optionally set the tracingMemory and keepingStatistics to true as well
Open a new browser window and perform the actions you wish to record.
Go back to the open Admin UI window
Set the recording property to false here
Review the generated log file C:\ATG\Dynamo5.5\home\logs\record.log
Save the script file to a new location of your choice!

The Recorded Script

This is a script which surfs PioneerCycling, not that it includes all the image requests and that the URI's are relative to the root, not specific to any given server.

The same scripts can be used to test various installations of the same site. However typically the production server does not serve static images, these are handled by the web server, so you may wish to remove these from the script to more accurately simulate user load.

You'll also want to review the delays in the script to make sure that they are reasonable and fit your needs (testing time vs. reality of user model).

The structure of each line is:

URI
delay in milliseconds
lines of posted data (-1 is the default for none)
session_ id

/Dynamo/solutions/PioneerCycling/ 0 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/splash-top.gif 40 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/splash-picture.jpg 0 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/common/Style.css 110 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/store_home.jhtml 4396 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/promotions/images/generic-marketing.gif 10 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/catalog/category_bundle.jhtml?id=cat180001&navAction=jump&navCount=12 4747 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/nav/nav.1.1.gif 80 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/nav/nav.2.1.gif 10 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/green-cart.gif 0 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/nav/nav.3.1.gif 10 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/images/cat7.jpg 10 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/catalog/product_bundle.jhtml?id=prod170002&navCount=13&navAction=push 1863 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/MEDIA/ProductCatalog/m170007_bundle2.gif 220 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/checkout/cart.jhtml 4607 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/Dynamo/solutions/PioneerCycling/en/store_home.jhtml?navAction=pop&id=cat10002&navCount=14 2263 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/MEDIA/ProductCatalog/m190093_shorts_mnylonblue_sm.gif 30 -1 G0SKIFBQOPWWMCQKIJKSFEQ
/MEDIA/ProductCatalog/m190032_hat_knit_logo_sm.gif 10 -1 G0SKIFBQOPWWMCQKIJKSFEQ

Recorded Statistics

Turning on the optional values tracingMemory and keepingStatistics allows you to develop baseline performance figures which can be used to gauge the impact of changes and updates to the site.

You should save the values from your baseline script for future comparison.

Replaying the Script

To replay the script assuming you want to run the script at C:\ATG\Dynamo5.5\home\logs\record.log usually though you'll have copied the file from elsewhere.

Open a new clean cmd window do something like the following, depending on your configuration

C:
set DYNAMO_HOME=C:\ATG\Dynamo5.5\home
cd %DYNAMO_HOME%
bin\dynamoEnv
java atg.core.net.URLHammer C:\ATG\Dynamo5.5\home\logs\record.log 1 1 -script -server localhost:8840

For multi-user simulation increase the first numeric argument to the number of threads you want say 10 and set the second numeric argument to the number of repetitions. The following gives us 10 concurrent users, repeating 5 times for a total of 50 users over time.

java atg.core.net.URLHammer C:\ATG\Dynamo5.5\home\logs\record.log 10 5 -script -server localhost:8840

Regression Testing

For basic regression testing all you need to do is replay the script and check for errors. That was described in the previous section.

What else can we do? As a part of the build checking procedure, I'd recommend

Checking page generation times
Checking page memory usage

Even if it is a new page, knowledge of these numbers will help you assess whether things are 'right' or not, for example in the sample above we see that pages take between 10 and 400 ms. If our new page takes more than 500 ms then that might be reason to investigate further.

To perform a memory and latency regression test:

Generate baseline numbers (do this once)
- Set the tracingMemory and keepingStatistics to true
- Create baseline numbers as described in the Making the Recording section
- Set the recording property to false here
- Save the contents of the Recording servlet page: http://localhost:8830/nucleus/atg/dynamo/servlet/pipeline/RecordingServlet
Run the regression test
- Ensure that the recording property is set to false here
- Set the tracingMemory and keepingStatistics to true
- Replay the script as described
- Set the tracingMemory and keepingStatistics to false
Analyze the results
- Compare the saved contents of the Recording servlet with the new figures: http://localhost:8830/nucleus/atg/dynamo/servlet/pipeline/RecordingServlet

Technical Training Advertise your Training Programs for Free! Los Angeles Web Design Shopping Cart Software Form a Corporation