Load Testing Tools

As websites, web applications, APIs and services become more and more critical, it's import to design and build applications so that they scale well for a large number of requests. To actually test and verify the behavior of services under load, testers and developers can perform load and stress tests. For web applications a load test would usually involve defining, writing and running test scenarios.

A test scenario could be a script that simulates the behavior of a typical user across multiple pages of a website or web application. Once you defined your test scenarios, you can execute it by simulating many concurrent virtual users and monitor how your application behaves. Likewise for APIs and other services, the idea is to generate requests that are as realistic and similar to the possible load of your production environment, so you can optimize and scale your application and services based on accurate and realistic data.

Load Testing Tools8 Tools

There are various tools that you can use to build, conduct and analyze load and stress tests for web applications and other services. Most tools require that you build test scenarios, usually scenarios a typical user would perform on a website, and then simulate many virtual users simultaneously.

Open Source

JMeter

JMeter is an open source Java-based load and performance testing tool. Besides testing web applications and services, you can also load test other services such as LDAP, mail servers or databases.

Open Source

Tsung

Tsung is an open source multi-protocol distributed load testing tool. The purpose of Tsung is to simulate users in order to test the scalability and performance of services, such as HTTP servers and other services.

Open Source

ApacheBench

This is a tool for benchmarking your HTTP server. It is designed to give you an impression of how your current Apache (or other web server) installation performs. This especially shows you how many requests per second your Apache installation is capable of serving and can be useful to benchmark web applications and APIs as well.

Open Source

Locust

Locust is a scalable and distributed load testing tool for web applications, websites and web-based services. Locust comes with a web-based user interface and allows you to write test scenarios in Python.

Commercial

WAPT

WAPT is a load and stress testing tool that allows you to build, generate and monitor load tests via a graphical user interface. WAPT comes in two editions and doesn't limit the number of virtual users you can use like most other commercial tools do.

Commercial

Webserver Stress Tool

The Webserver Stress Tool is an advanced load testing tool that comes with a graphical user interface to run and analyze your load tests. This tool can generate a number of different tests to verify your web applications and services.

Commercial

LoadUI

LoadUI is a cross-platform load and stress testing tool. LoadUI is available as both open source and commercial editions. Its UI allows you to create, configure and redistribute your load tests interactively and in real-time.

Commercial

Loadster

Loadster it a load and stress testing tool specifically designed for high-performance web applications. It comes in two parts: a tool to build and design your test scenarios, and an engine that you can use to actually run and distribute your load tests.

Load Testing Cloud Services

In addition to the above mentioned load testing tools, the following services make it easy to start large-scale load tests without having to build your own infrastructure. This is especially useful if you only have to conduct such tests from time to time, e.g. at specific points in your development efforts.

Service	Type	Description	Pricing
BlazeMeter	Web & APIs	JMeter-compatible load testing service	Starting @ $199/month
LoadStorm	Web & APIs	SaaS cloud testing service	Starting @ $40/month
Load Impact	Web & APIs	Websuite load testing service	Starting @ $60
Loader.io	Web & APIs	Free cloud testing service by Sendgrind	Free

Using Apache Bench for Simple Load Testing

If you have access to a Mac or Linux server, chances are you may already have a really simple http load generating tool installed called Apache Bench, or ab. If you are on windows and have Apache installed, you may also have ab.exe in yourapache/bin folder.

Suppose we want to see how fast Yahoo can handle 100 requests, with a maximum of 10 requests running concurrently:

ab -n 100 -c 10 http://www.yahoo.com/

It will then generate output as follows:

Concurrency Level:      10
Time taken for tests:   1.889 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      1003100 bytes
HTML transferred:       949000 bytes
Requests per second:    52.94 [#/sec] (mean)
Time per request:       188.883 [ms] (mean)
Time per request:       18.888 [ms] (mean, across all concurrent requests)
Transfer rate:          518.62 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       57   59   1.7     59      64
Processing:   117  126   7.5    124     162
Waiting:       57   62   7.0     60      98
Total:        175  186   8.0    184     224

Percentage of the requests served within a certain time (ms)
  50%    184
  66%    186
  75%    187
  80%    188
  90%    192
  95%    203
  98%    216
  99%    224
 100%    224 (longest request)

As you can see this is very useful information, it returned requests at a rate of 52.94 requests per second, the fastest request was 175ms, the slowest 224ms

So the next time you are tempted to whip out cfloop and GetTickCount to do some benchmarking on a piece of code, give ab a try, it's easy to use, and will yield much more realistic results.

Because ab supports concurrency, this has two big advantages over cfloop. The main one is that it allows you to test how your code runs concurrently, this can help you identify any possible race conditions, or locking issues. Concurrent requests are also a more natural simulation of load than loops.

Suppose you wanted to test multiple url's concurrently as well? You can do this by creating a shell script, with multipleab calls. At the end of each line place an & this makes the command run in the background, and lets the next command start execution. You will also want to redirect the output to a file for each url using > filename For example:

#!/bin/sh

ab -n 100 -c 10 http://127.0.0.1:8300/test.cfm > test1.txt &
ab -n 100 -c 10 http://127.0.0.1:8300/scribble.cfm > test2.txt &

The usage info from the ab version installed on my Mac (v2.3) is listed below. As you can see there are many useful options for outputting results, and sending additional data in the request.

Usage: ab [options] [http[s]://]hostname[:port]/path
Options are:
    -n requests     Number of requests to perform
    -c concurrency  Number of multiple requests to make
    -t timelimit    Seconds to max. wait for responses
    -b windowsize   Size of TCP send/receive buffer, in bytes
    -p postfile     File containing data to POST. Remember also to set -T
    -T content-type Content-type header for POSTing, eg.
        'application/x-www-form-urlencoded'
        Default is 'text/plain'
    -v verbosity    How much troubleshooting info to print
    -w              Print out results in HTML tables
    -i              Use HEAD instead of GET
    -x attributes   String to insert as table attributes
    -y attributes   String to insert as tr attributes
    -z attributes   String to insert as td or th attributes
    -C attribute    Add cookie, eg. 'Apache=1234. (repeatable)
    -H attribute    Add Arbitrary header line, eg. 'Accept-Encoding: gzip'
        Inserted after all normal header lines. (repeatable)
    -A attribute    Add Basic WWW Authentication, the attributes
        are a colon separated username and password.
    -P attribute    Add Basic Proxy Authentication, the attributes
        are a colon separated username and password.
    -X proxy:port   Proxyserver and port number to use
    -V              Print version number and exit
    -k              Use HTTP KeepAlive feature
    -d              Do not show percentiles served table.
    -S              Do not show confidence estimators and warnings.
    -g filename     Output collected data to gnuplot format file.
    -e filename     Output CSV file with percentages served
    -r              Don't exit on socket receive errors.
    -h              Display usage information (this message)
    -Z ciphersuite  Specify SSL/TLS cipher suite (See openssl ciphers)
    -f protocol     Specify SSL/TLS protocol (SSL2, SSL3, TLS1, or ALL)

Tester

http://www.c-sharpcorner.com/UploadFile/51e7af/basics-of-manual-testing/

http://www.guru99.com/web-application-testing.html

http://www.embeddedstar.com/weblog/2009/08/23/pragmatic-ui-tests/

http://reddytesting.blogspot.com/2011/05/reusable-test-cases-for-web-based.html