Introduction

It’s difficult to rank which type of testing is the most important in software development. Many types work in conjunction and allow teams to build and ship usable software. Regardless of order, before releasing an application out into the world it should be load tested first.

Furthermore, load testing is so vital that it is done in many industries on different types of products. The automobile industry wants to test car batteries to monitor minimum voltage under ‘load’. In the manufacturing sector, cranes are (literally) load-tested carrying a ‘load’ at different heights and under different conditions.

In this article, we discuss what load testing means in software development. Additionally, we’ll provide examples, tools, context, and how load testing fits into the software testing environment.

Load Testing

Load testing generally refers to the practice of modeling the expected usage of a software program by simulating multiple users accessing the program concurrently.

Wescott, Bob (2013). The Every Computer Performance Book, Chapter 6: Load Testing. (Wikipedia, accessed 12/21/2020)

Given the above definition, we can assume that load testing is not performed at the beginning of the software development process. It happens as you start putting pieces of your application into production (or a staging environment).

Unit testing, and functional testing, ensure that your application works. Load testing determines how well the application works under certain conditions. In a load test (like in a scientific experiment) we want to hold as many variables constant as possible. Then, we want to vary the load (i.e users, temperature, weight) and see how it performs.

Load Testing vs. Stress Testing

My uncle worked for the airplane manufacturer Boeing. They routinely test the plane’s physical capabilities. He described one of the tests that they performed on a new model plane.

The wings of the airplane were strapped to a device that would slowly pull the wings of the airplane upwards, bending the wings to create a U-shape. Some engineers hypothesized that the wings may touch before they broke. As they performed the test, the wings went further-and-further up, looking like they were going to touch, before eventually breaking. This was a stress test.

The goal was to break the wings. After they broke, they recorded under what conditions and where the wings broke. However, the chances that the plane would ever experience those kinds of conditions is virtually zero.

Load tests seek to discover how the system performs under normal conditions. Not to discover where things start to break.

Load Testing vs. Performance Testing

Next, let’s examine a common question. How is performance testing different than load testing? Here, we have an invalid comparison. Load and stress testing fall under the performance testing category. A list of performance test types includes:

• Load testing
• Stress testing
• Soak testing
• Spike testing
• Breakpoint testing
• Configuration testing
• Isolation and internet testing

What Are the Types of Testing?

Unit

Unit testing is dividing application source code into units and writing tests for each unit. These tests are often independent of the other units in the code. Furthermore, programmers often execute unit tests during development.

Functional

Functional tests combine and test different software modules together. Unit tests ensure software performs in an isolated environment. Functional testing integrates the different modules to see if they can achieve higher level requirements.

This type of testing uses ‘black-box testing’. The software’s implementation is unknown to the testers. Therefore, quality assurance (QA) agents are sometimes the people that perform functional tests.

The process is straightforward from the perspective of the functional tester. The component (function) receives inputs and then they compare the outputs to requirements.

Regression

Regression testing benchmarks functional tests. This ensures new features (changes) do not have a negative impact on current features or application functionality.

You can do regression testing by re-running the suite of functional tests.

Load

Unit, functional, and regression tests run consistently. Load testing, usually, is done sporadically or at important software milestones (v1 release, v2 release).

Automatic testing tools help with all forms of testing. However, load testing can require more resources.

Best Tools for Load Testing

Many tools exist to help with automated testing. Some are scoped to help with specific types. Below is a list of some of the best and most popular tools are load testing.

Apache JMeter

JMeter is a software application created by the Apache Software Foundation. You may have heard about Apache through its popular web server project. Apache JMeter is open source and written in Java. Therefore, you can use it with any Java-compatible operating system.

Thanks to the Apache Software Foundation, JMeter’s documentation is impressive. There are plenty of guides, tutorials, and information to get someone started.

Additionally, you can view the FAQ page for more information. Apache JMeter is a great choice if your team has expertise. Furthermore, because it’s open-source, it is free to use.

LoadNinja

LoadNinja is a cloud-based SmartBear creation for load-testing.

This product allows you to record and playback scripts in the browser. Many paid testing tools offer solutions that give users a better user interface. With LoadNinja you can view:

• response times
• throughput rates
• resource utilization

LoadNinja is a good choice for enterprise users that are looking to integrate their application into SmartBear’s software products.

LoadRunner

LoadRunner is another product produced by a software giant in Micro Focus. The companies’ website has a helpful resource page that details a guide, white-paper, brochure, and case studies that help get a better picture of the software.

For example, one of the studies covers a company that handles data from 200,000+ racing events in 22 companies.

Some of the application environments that LoadRunner supports testing for are:

• web/mobile
• MQTT
• HTML5
• WebSockets
• RDP
• Database
• Remote Terminal Emulators
• Java
• .NET

Eggplant

Another popular load testing tool is Eggplant. The company touts a unique ability to simulate virtual users of an application UI at scale. Eggplant is trusted by some significant organizations. They work to create realistic user scenarios in industries like:

• Aerospace & defense
• Financial services
• Healthcare
• Automotive

Load testing can require complex scenarios. Furthermore, many of those scenarios may be difficult to create with out-of-the-box products. Comparatively, I think load testing requires more research for choosing a product than unit, functional, etc.

Load Testing Example

Finally, let’s discuss an example of how load testing could be done for a web application. In this example, let’s assume your application:

• Can have ~8,000 users at any given time
• Generates financial wellness reports
• Allows users to download the reports

This example is small. Many applications have different modules, dashboards, and functionality. Therefore, many scenarios could be built that accurately simulate the user experience. However, we want to keep it simple.

Planning the Test

Now that we have calculated the users and an essential process, we can plan the test. Furthermore, it’s important that we are able to map user actions to parts of our application.

1. 1. User logs in (form)
   2. The user adds financial information (form)
   3. The report generates (button)
   4. The report is downloaded (button)

Next, we need to select how many users we want performing these activities at a time. Let’s stratify the users at six thousand, eight thousand, and ten thousand. This gives us a low, medium, and high estimate.

Setting Up the Test

This part of the application relies heavily on the software that you’re using. Additionally, it depends on how well you can simulate user activity.

You may want to build stoppage or wait times in these actions. Or, have the users enter different information. Another likely scenario would be to have x amount of users log-in but not all fill out or download the report. There are different ways to measure user experience. You can:

• View the visual browser results or playback
• Judge experience on server response time

It’s common to set an acceptable response time that the user is required to withstand. For example, logging in should not take longer than 4-seconds. Another possibility is that the financial report generation should take between 15-20 seconds. The time measurements can be adjusted to fit user load. It’s important to know if there are any set response timeouts in your application. If the timeouts are set below what users are experiencing, then there is a problem.

Analyzing Test Results

Reviewing test results is like any other form of software testing. We defined the steps of the test and the parameters of success. Therefore, we can easily judge success.

Some testing tools may provide areas where bottlenecks occur or steps that cause an increased amount of time. Additionally, charts and graphs can show you how your user load affected different parts of the test.

Conclusion

In the end, load testing could save your organization from costly performance errors. I hope this article has helped to explain what load testing is, how it compares to other types of testing, what the tools are, and how an application could be load tested.