The quality of any application is determined by the robustness and scalability of the system. It's mandatory to simulate the actual environment and test the application for preparedness. Web Services-savvy applications need a different methodology for testing in a real-world scenario. The UI-less nature of Web Services presents a significant challenge in testing such applications. The whole persona of consumer stubs with different payloads dictates the planning of Web Services load-testing schemes. This paper talks about the different aspects of load testing and areas of contention that need special attention. This will be helpful in not only building a better application but also compiling a robust, high-quality enterprise architecture.
Web Services are the natural delivery mechanism to achieve SOA. While having the potential to free enterprises from the endless cycle of vendor-specific hardware/software upgrades by ensuring interoperability, they bring in integration complexities and the overhead of maintaining compatibility with the underlying EIS applications/systems. This brings in an absolutely different perspective to testing Web Services.
Web Services applications generally use a lot of data transformation, wraparounds (wrappers), translation, and abstraction to bring about the promised interoperability and portability. Their dependence on bandwidth-heavy protocols like SOAP doesn't ensure many performance benefits when compared to legacy applications (which tend to be very tightly coupled). Parameters like response time, throughput, and CPU utilization for transactions determine the viability of a real-world business application. Extensive testing of Web Services based on these parameters brings to the fore the most common performance constraints associated with them. The test results not only indicate whether the associated benchmarks are attained, but also if the service can scale to meet demands imposed by concurrent access from multiple users, simulated or otherwise.
Web Service endpoints generally also have very high visibility. They have to service multiple clients over the network simultaneously, maintaining robustness and availability at the same time. In such a situation, performance becomes even more crucial. Thus, the significance of proper performance testing for Web Services can't be overemphasized.
A Web Service, like any other application, can be subject to a wide range of test conditions and testing strategies. Some of them being functional testing, regression testing, performance testing, stress testing, and load testing. This paper will focus only on the load testing of Web Services. The expected behavior of a Web Service will be evaluated against various performance criteria when concurrent access by multiple clients is simulated. It becomes crucial to ensure that apart from optimizing design and implementation, Web Services have to be tested for throughput, efficiency, and response simulating real-world conditions as closely as possible. This is where load testing plays a major role. A properly designed load-testing strategy can simulate real-world load and performance scenarios with minimal hassle and cost. User loads and network conditions of varying nature can be effortlessly created and replicated. Testing can be undertaken till the output charts show a performance range considered acceptable for an application of its nature. Load-testing results can hence be taken as a strong indicator of application performance in actual business environments.
To ensure optimal testing of Web Services, the test cases have been designed keeping the following parameters in mind:
- Size of payload: This tests the amount or size of the incoming requests. This parameter is vital in determining the threshold of data beyond which the service behaves in unexpected ways.
- Concurrency: The test cases have to simulate simultaneous access of the service by multiple clients to replicate real-world conditions.
- Latency: It can be defined as the time from issuing a request to the service from the client and the receipt of the first response. This parameter encapsulates the performance of the service, bandwidth in the network, and other communication overheads. It's important that latency be minimized as far as possible, at least up to a tolerable point.
- System utilization: The net CPU and memory resources consumed by the service under varying loads in normal enterprise environments should be captured by the load-testing scheme. This helps in identifying potential bottlenecks and points out areas of improvement.
These parameters shall be discussed in detail later:
Load Testing with Reference to Web Services
Load testing of Web Services is significantly different from testing of other applications since their performance is not just attributed to how robust the underlying architecture is but also to the network overheads, underlying processing involved, and the performance of the Web server that hosts the service. The behavior of the SOAP engine also invariably adds to the architecture of service provider systems. Certain major areas of contention when evaluating the Web Service performance that will be discussed here are:
Load testing of Web Services is significantly different from testing of other applications since their performance is not just attributed to how robust the underlying architecture is but also to the network overheads, underlying processing involved, and the performance of the Web server that hosts the service. The behavior of the SOAP engine also invariably adds to the architecture of service provider systems. Certain major areas of contention when evaluating the Web Service performance that will be discussed here are:
- The impact of an incremental XML payload size wrapped inside a SOAP message
- The impact of a chosen style/use during the design of a Web Service
- The serialization/de-serialization involved in processing the SOAP message
- The underlying parsing model and validation schemes chosen to process the XML payload
CIO, CTO & Developer Resources
The results of the load testing such as response time graphs will further depict the vitality of the load testing of Web Services to ascertain their conformity prior to actual deployment to enable their wide-scale adoption without compromising their performance and scalability characteristics, enabling the enforcement of stringent operational, behavioral, and non-functional requirements that are inherent in the successful realization of any business process.
Load Testing Metrics and Parameters
The results obtained by load testing Web Services can potentially be reflected in terms of the following parameters.
The results obtained by load testing Web Services can potentially be reflected in terms of the following parameters.
- Response time: It's the most important parameter to reflect the quality of a Web Service. Response time is the total time it takes after the client sends a request till it gets a response. This includes the time the message remains in transit on the network, which can't be measured exclusively by any load-testing tool. So we're restricted to testing Web Services deployed on a local machine. The result will be a graph measuring the average response time against the number of virtual users.
- Number of transactions passed/failed: This parameter simply shows the total number of transactions passed or failed.
- Throughput: It's measured in bytes and represents the amount of data that the virtual users receive from the server at any given second. We can compare this graph to the response-time graph to see how the throughput affects transaction performance.
- Load size: The number of concurrent virtual users trying to access the Web Service at any particular instance in an interval of time.
- CPU utilization: The amount of CPU time used by the Web Service while processing the request.
- Memory utilization: The amount of memory used by the Web Service while processing the request.
- Wait Time (Average Latency): The time it takes from when a request is sent until the first byte is received.
Performance Bottlenecks & Areas of Contention
Web Services are simply components deployed on a server. Most of the Web Services today are exposed out of existing components such as Enterprise Java Beans. Hence, in theory, we should be able to use the existing testing mechanisms and performance-enhancing strategies. But as already discussed load testing Web Services is quite different. The performance of Web Services is influenced by a lot of factors like bottlenecks in the network, processing at intermediate nodes if any, pre-processing of the SOAP message at the SOAP engine before it's dispatched to the service, etc. To identify the areas of contention, we'll look first at the architecture of the SOAP message processing on the service side. (see Figure 1)
Web Services are simply components deployed on a server. Most of the Web Services today are exposed out of existing components such as Enterprise Java Beans. Hence, in theory, we should be able to use the existing testing mechanisms and performance-enhancing strategies. But as already discussed load testing Web Services is quite different. The performance of Web Services is influenced by a lot of factors like bottlenecks in the network, processing at intermediate nodes if any, pre-processing of the SOAP message at the SOAP engine before it's dispatched to the service, etc. To identify the areas of contention, we'll look first at the architecture of the SOAP message processing on the service side. (see Figure 1)
A client application creates a SOAP message containing the XML payload, which can be either a SOAP-RPC-encoded request or a document-style message. The client sends this message along with the service endpoint URL to the SOAP client runtime, which in turn sends it over the network. Once the SOAP message is delivered to the SOAP runtime at the service, it passes through handlers (if any) that handle the processing of any additional tags for WS-Security, WS-Addressing, etc. Then the SOAP runtime converts the XML message into programming language-specific objects if required by the application. The Web Service processes the request message and formulates a response. The SOAP runtime on the service side takes care of creating a SOAP message and dispatching it back to the client.
So, apart from the actual processing of the Web Service, there's some additional processing involved before and after the Web Service builds a response. Let's identify the bottlenecks involved in invoking a Web Service:
- XML processing and overheads: Since XML data is verbose and contains lot of metadata information, processing of XML is a major performance bottleneck. Processing XML involves parsing, validating the data against schema, and marshalling/unmarshalling. It's quite memory- and CPU-intensive and the response time takes a hit if the proper strategies aren't followed.
- Parsing of XML: The larger the SOAP message, the longer it takes to parse it. Parsing SOAP messages is a major contributor to performance issues with Web Services. A memory-efficient parser like StAX (a pull parser) should be used in place of a memory-intensive parser like DOM.
- Serialization/de-serialization: When the SOAP engine on the service side receives a SOAP request, it de-serializes the XML data according to the encoding format mentioned, i.e., extracts the payload out of it, and creates objects that are used by the Web Service. After the Web Service executes the business logic, the SOAP engine takes care of serializing the response back to XML and sends the data to the client. For huge XML documents, the serialization/de-serialization takes a performance hit if a proper mechanism isn't selected.
- Select a proper style for your Web Service: The two most pre-dominantly used styles of Web Services are document/literal and RPC/encoded. The SOAP message of the RPC-encoded style of Web Service contains the type-encoding information, which is an overhead on the SOAP engine and degrades the throughput performance whereas the document/literal SOAP message contains no such type-encoding information. The XML payload can be easily validated against the schema included in the WSDL. Also, the data binding specific to a SOAP engine can be switched off in the case of a document/literal-style Web Service. This enables one to use a custom binding framework like XMLBeans, castor, JAXB, etc. This is especially useful when the application uses a large number of complex custom data types.
- Processing by handlers: The SOAP engine first dispatches the SOAP message to the handlers. The handlers may be responsible for performing additional processing like authentication, encryption/decryption of the XML payload, parsing the SOAP message for any information like WS-Security, WS-Addressing, etc.