Co-authors: Manoj Kumar, Rahul Kumar.

In the realm of software testing, performance and reliability are paramount. A crucial aspect of ensuring these qualities is timeout testing. Timeout testing evaluates a system’s behaviour when a process or operation exceeds a pre-set time limit. This helps developers and testers identify potential performance bottlenecks, deadlocks, or time-related failures before they impact end-users.

This document explores the necessity and execution of timeout testing in software testing, and how it can be applied to REST APIs and UI testing. We will also delve into HAProxy, a popular tool for Load Balance and reverse proxy, demonstrating how it can route API requests to a mock server to simulate real-world network delays.

What is Timeout Testing?

Timeout testing involves deliberately setting time limits (timeouts) for various operations, such as API requests, inter-system communications etc. The goal is to test the system’s reaction when an operation exceeds the defined time constraint, ensuring that proper error handling and failover mechanisms are in place.

During timeout testing, the tester verifies the following:

• System response to timeouts: How does the system behave when an operation exceeds the time limit? Does it throw an error, gracefully fail, or crash?
• Error handling: Are users notified of the timeout with clear error messages?
• System performance: Is the system still responsive under delay conditions? Does it remain stable during high loads or network failures?
• Recovery mechanisms: Can the system automatically retry the failed operation, or should users take further action?

Why is Timeout Testing Necessary?

1. Ensures System Stability

Timeout testing ensures that your system doesn’t hang, freeze, or crash due to long-running operations. When a timeout occurs, a well-designed system should provide error messages or alternate actions, such as retries, to maintain overall stability.

2. Improves User Experience

Long waits for a response can frustrate users. Timeout testing ensures that requests are handled efficiently, preventing users from being left hanging, which is critical for maintaining a positive user experience.

3. Ensures Scalability

As systems scale, network delays or heavy traffic can lead to longer response times. Timeout testing ensures that the system can still handle increased traffic without becoming unresponsive.

4. Compliance and SLAs

Many systems must meet performance guarantees defined in Service Level Agreements (SLAs). Timeout testing helps ensure that your application complies with these SLAs, minimizing the risk of penalties or user dissatisfaction.

Timeout Testing using Proxy Server

In our day-to-day scenarios, our requests are routed to the main server when we are performing feature or regression testing. 

But whenever we want to test the timeout scenarios, we re-route our requests to the mock server by changing the configuration to client service.

In this approach, reconfiguring the setup to route requests between the mock server and the main server is necessary. However, this process is manual, time-consuming, and often impacts other ongoing QA testing activities.

Solution:

To address this challenge, we proposed introducing a proxy layer between the client and server. This proxy layer can dynamically route requests to either the mock server or the main server based on predefined conditions, eliminating the need for manual configuration changes or additional effort.

Timeout Testing for REST APIs with HAProxy

Timeout testing for REST APIs is essential to ensure that the system handles slow or unresponsive services gracefully.

HAProxy is a powerful tool for load balancing, reverse proxying, and managing timeout behaviour. It can route API requests to mock servers, enabling the simulation of various network conditions and testing how your system handles timeouts.

Using HAProxy for Timeout Testing

HAProxy can be configured to simulate timeout scenarios, route traffic to a mock server, and ensure that the system reacts appropriately when the backend is slow or unresponsive. Below is an example of how you can set up HAProxy to route requests to a mock server and simulate timeouts for testing.

Example: Timeout Testing with HAProxy and a Mock Server

Let’s say we have a simple REST API that fetches user data. To simulate different timeouts, we can route API requests through HAProxy to a mock server that introduces artificial delays.

Step 1: Set Up HAProxy Configuration for Timeout Simulation (haproxy.cfg)

frontend http_front
  bind *:80

  # Timeout for client-side connections (10 seconds)
  timeout client 10s

  # Timeout for establishing connection to the backend (3 seconds)   
  timeout connect 3s

  # Timeout for waiting for the server response (5 seconds)
  timeout server 5s

  # Timeout for idle keep-alive connections (2 seconds)
  timeout http-keep-alive 2s

backend mock_server
  server mock1 127.0.0.1:8080 check
  server mock2 127.0.0.1:8081 check

backend main_server
  server main1 127.0.0.1:8082 check

In this configuration:

• Client Timeout is set to 10 seconds. If a client takes more than 10 seconds to establish a connection, the request is rejected.
• Connect Timeout is set to 3 seconds. If HAProxy cannot establish a connection to the backend in this time frame, it will return an error.
• Server Timeout is set to 5 seconds. If the backend server does not respond within 5 seconds, HAProxy will return a timeout error.
• Keep-Alive Timeout is set to 2 seconds to control how long idle connections can be kept open.

Step 2: Mock Server for Testing Timeouts

To simulate slow responses, you can use a simple mock server, such as one created with wiremock and Java. Below is an example of a basic Java code snippet that introduces a delay:

WireMockServer wireMockServer = new WireMockServer(8080);
wireMockServer.start();

// Configure the stub for "/api/users/{user_id}" with 7 sec delay 
wireMockServer.stubFor(get(urlEqualTo("/api/users/123"))
  .willReturn(aResponse()
  .withHeader("Content-Type", "application/json")
  .withBody("{\"user_id\": 123, \"name\": \"John Doe\"}")
  .withFixedDelay(7000) // Simulate 7 seconds delay
));

Condition Based Routing

We aim to integrate timeout testing into our regression, sanity, and feature testing processes and automation suites. To achieve this, we have leveraged the conditional-based routing capabilities of HAProxy.

In the HAProxy config file, we can define conditions to route certain requests to mock service while routing other requests to the main service. HAproxy supports ACL to define various conditions to route requests to a particular service.

Following is a sample config file that defines acl conditions and routing to mock service if conditions are met.

frontend http_front 
  bind *:80 

  # Timeout for client-side connections (10 seconds) 
  timeout client 10s 

  # Timeout for establishing connection to the backend (3 seconds)
  timeout connect 3s 

  # Timeout for waiting for the server response (5 seconds)
  timeout server 5s 

  # Timeout for idle keep-alive connections (2 seconds) 
  timeout http-keep-alive 2s

  acl is_get method GET
  acl timeout_user path_end /api/users/123

  use_backend mock_server if is_get timeout_user
  default_backend main_server

backend mock_server
  server mock1 127.0.0.1:8080 check 
  server mock2 127.0.0.1:8081 check

backend main_server
  server main1 127.0.0.1:8082 check

In this example:

• The name assigned to the first acl is is_get and to the second acl as timeout_user
• The method argument returns the method of request and sets the variable is_get as true if the request method is GET.
• The path_end argument checks if the url path of request ends with /api/users/123 and sets the variable timeout_user to true else false.
• The use_backend argument routes the request to the mentioned server if conditions are true.
  Here, the request will be routed to mock_server if both acls are true i.e. request method is GET and the path ends with /api/users/123 .
• The default_backend argument routes the request to the mentioned server if no conditions are met.
  Here, requests will be routed to the main server if any of the above-mentioned acl conditions are not met.

The above example represents a basic conditional routing to mock service if the request method is GET and the request path ends with /api/users/123 . If these conditions are not met then requests will be redirected to the main service.

Timeout Testing for UI

In modern applications, ensuring that the user interface (UI) remains responsive is critical. UI timeout testing focuses on evaluating how the front-end application handles delays in interaction, loading, or responding to requests. This includes checking how the UI reacts when a backend service takes too long to respond or when there is poor network performance.

Here are some examples of how to conduct timeout testing for UI applications:

Example: Simulating Timeout for Slow API Responses in a UI

Imagine you have a web application that fetches data from an API when a user clicks a button. If the backend service is slow or down, the UI should inform the user rather than hang indefinitely.

Scenario:

• Simulate a slow API response (or no response at all) and test how the UI reacts.
• Set a timeout of 5 seconds for the API request from the UI.

Expected Behavior:

• If the backend service doesn’t respond within 5 seconds, the UI should display a clear error message (e.g., “The request timed out. Please try again later”).
• The UI should remain responsive, and the user should be able to take other actions.

Example with Selenium for UI Testing:

In a Selenium-based test, you can set timeouts for waiting for elements or API responses. For example, if you’re testing a button that makes an API request:

try {
  // Open the web application
  driver.get("http://your-web-app.com");

  // Simulate clicking a button that makes an API request
  WebElementbutton = driver.findElement(By.id("load-data-button"));
  button.click();

  // Wait for the error message to appear, with a timeout of 5 seconds
  WebDriverWaitwait = new WebDriverWait(driver, 5);

  try {
    wait.until(ExpectedConditions
        .presenceOfElementLocated(By.id("errMsg")));

    System.out.println("Timeout occurred: error message displayed");
  } catch (TimeoutException e) {
    System.out.println("API res within time limit, UI is responsive");
  }
} catch (Exception e) {
  System.out.println("API res within time limit, UI is responsive");
} finally {
  // Quit the driver to close the browser
  driver.quit();
}

How HAProxy Can Help Simulate Timeout Scenarios

To introduce controlled delays for timeout testing, you can use tools like HAProxy. By setting up conditional routing, HAProxy can simulate slow or unresponsive backend services, making it easier to observe and validate how the UI reacts. This approach removes the need for manual configuration changes, streamlining the testing process.

Other Solutions

New Header on lower environment

Special HTTP headers could be introduced to simulate specific service states. For example, the header X-TEST_HTTP_FAIL_MODE=503 could force a service to return an HTTP 503 error without processing the request.

Add new Profiling in services that can point to mock

We can configure a new profile in the system that can point to mock service for every application. But this would need extra efforts in development and would need additional cost as well

Change in DNS Service at run time:

Change in DNS services of system where application is running and can be routed to mock service at run time

Conclusion

Timeout testing is a critical aspect of software testing that ensures your application can handle delays, network issues, and slow responses gracefully. For REST APIs, this means testing how the system responds when requests exceed time limits, simulating various latency and failure scenarios, and ensuring that proper error messages and retry mechanisms are in place.

By using tools like HAProxy for network-level timeout testing and routing requests to mock servers, you can simulate realistic timeout scenarios to ensure that your system remains reliable, scalable, and user-friendly under all conditions. HAProxy’s performance, scalability, and fine-grained timeout management make it an ideal tool for timeout testing, particularly when compared to other solutions.

For UI timeout testing, it’s essential to ensure that the user interface responds properly to slow backend operations or unresponsive services, providing users with clear feedback in case of errors.

By proactively addressing timeout scenarios, you help ensure that your software is both reliable and responsive, regardless of the network, server, or API issues it may encounter in the real world.

References

HAProxy Docs: https://www.haproxy.com/documentation/haproxy-configuration-manual/latest/

Wiremock Docs: https://wiremock.org/docs/