Data driven web service testing with Apache Jmeter

You will not get enough coverage in your web service testing, if you repeatedly send the same request to the webservice under testing. You must read data from a data source and parameterize each request. If you are dealing with SOAP messages, you should parameterize SOAP message payload.
This post guides you how to do data driven web service testing using Apache Jmeter.

Pre-requisites:
Install Apache Jmeter 2.3.4 or later

Step 1

We are going to invoke a publicly available web service, Temerature Conversion service . The WSDL of this web service can be accessible through http://webservices.daehosting.com/services/TemperatureConversions.wso?WSDL
You can invoke the CelciusToFahrenheit operation of this webservice using SOAPUI and capture the request message. It will be similar to the below.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
<soapenv:Header/>
<soapenv:Body>
<tem:CelciusToFahrenheit>
<tem:nCelcius>37</tem:nCelcius>
</tem:CelciusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>
Step 2

Lets create a new JMeter test plan and add a thread group. Then, add SOAP/XML-RPC sampler into the thread group and copy and paste the above SOAP request into the SOAP/XML-RPC data section of the sampler.
Enter the URL of the service as http://webservices.daehosting.com/services/TemperatureConversions.wso (You can capture this from the location attribute of the address element in the WSDL)
Now add a listener to view the result.
Save your test plan. Your test plan will be similar to the below.



Run the test and check the results. You will get the Fahrenheit value of the provided Celsius figure.

Step 3

Now, you can increase the thread count and extend this test into a performance test. However, in that case you will be sending the same request again and again. It will not be a good simulation of a real-world scenario. You should be able to alter the payload (in our example, Celsius value) of the SOAP request with each thread.
In order to do so, you should read Celsius data from a data source. In Jmeter, you can easily read data from a csv file.
Lets create a csv file, temperature.csv and save it in the location where you saved the above JMeter test plan.
Enter a set of values row-by-row in the temperature.csv
eg:-
10
20
30
40

Step 4

Next, we will add CSV Data Set Config element which will read data from the csv file.
Right click on Thread Group and select Add --> Config Element ---> CSV Data Set Config
Now you can configure your CSV data source as follows.

Filename: <Give the full path of temperature.csv>
Variable Names: celcius

Keep the other values intact.

Step 5

Now access the SOAP/XML-RPC Data section of the request and replace the hard-coded celcius figure with the variable name we have configured in CSV Data Set Config element (i.e:- ${celcius})
After parameterizing, your SOAP request will be as follows.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
<soapenv:Header/>
<soapenv:Body>
<tem:CelciusToFahrenheit>
<tem:nCelcius>${celcius}</tem:nCelcius>
</tem:CelciusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>

Increase the thread count corresponding to the row count in your csv file and run the test. You will notice that the Celcius figure will be varied in each request by reading data from CSV data source.

In this way, you can easily do data driven web service testing using Jmeter.

Test Reviews - An efficient way to validate test coverage

When the complexity of testing increases, deriving test scenarios becomes extremely hard. Individual testers are not always capable of thinking all aspects of the products and finding out test cases. In agile processes which demands frequent releases, this becomes much harder since individual testers do not get sufficient time to learn and explore products.

At WSO2, code reviews have been an integral part of engineering process. Code review groups get together and go through different code segments in weekly basis.

Similar to code reviews, WSO2 QA team conducts weekly test reviews. The primary objective of reviewing tests is to finding out gaps in a particular product's testing methodology.

Usually there are 2 main actors take part in a test review meeting.

- Tester(s) who test a particular feature

- Developer (s) who implemented the feature

Tester goes through the existing test scenarios and developer provides his or her feedback on the tests. Specially when a particular use case is not clear enough, tester can clarify it during reviews very effective manner.

It is the testers responsibility to try out the new test scenarios which have been captured during the review in next release cycles.

Test review will not take more than 1 hour and testers should be prepared properly to get the maximum out of this short time and capture more and more test scenarios.

Similar to the manual test scenarios, all automated tests can also be reviewed during test reviews.

In agile processes where you do not have detailed system specs to derive tests, test reviews are one of the most useful mechanisms to validate your test coverage.

How to enable child-first class loading in WSO2 Application Server or Axis2

By default, WSO2 Application Server (or Axis2) uses parent-first class loading mechanism.

If you deploy an AAR service, which can load classes from the following locations.

• CARBON_HOME/lib (CARBON_HOME is the location where you installed WSO2 Application Server)
• CARBON_HOME/repository/deployment/server/axis2services/lib
• AAR service/lib (lib directory under your service archive)

If your service implementation class has a package import for a class, which is available in both CARBON_HOME/repository/deployment/server/axis2services/lib and the lib directory under service archive, the class placed under CARBON_HOME/repository/deployment/server/axis2services/lib will get loaded.

Which means, the parent class always get loaded first.

The class loading sequence is CARBON_HOME/lib ---> CARBON_HOME/repository/deployment/server/axis2services/lib -----> AAR service/lib

Sometimes, we want to load the class from service archive lib first without loading the class which is available either one of above parent locations. In other words, child-first class loading will be required for some instances.

child-first class loading can be enabled simply by adding the following parameter in to services.xml in your service archive.

<parameter name="EnableChildFirstClassLoading">true</parameter>

If you set this parameter in axis2.xml of your WSO2 Application Server (or Axis2) instance, all services will use child-first class loading mechanism.

WS-Addressing with SOAPUI

WS-Addressing is a standard way of including message routing data within SOAP headers without relying on transport specific routing properties. If you are talking to a web service which expects WS-Addressing information in request SOAP messages, you could either follow a programmatic approach to write a client and insert WS-A headers manually or use a commercial or free web service client tool. SOAPUI is the simplest tool which can be used to insert WS-Addressing headers in to request SOAP messages.
In this blog post, we will;

• Talk to a WS-Addressing enabled service which is hosted in WSO2 Application Server using SOAPUI

• Assert response SOAP message to validate WS-A headers

Pre-requisites:

1. Download and install the free version of SOAPUI 3.0.1 or later
2. Download and install WSO2 Application Server 4.0

Step 1

We are going to invoke the default HelloService which is shipped with WSO2 Application Server. Go to WSO2_APPSERVER/home/bin and start the server by running wso2server.sh{bat}
Access management console with https://localhost:9443/carbon and log in using the default administrator credentials (username=admin, password=admin)
Navigate to Manage --> Services --> List in the left menu and select "HelloService"
You will be directed to the dashboard of HelloService. Click on Modules link.



You will notice that "addressing-3.1.0" is listed under "Globel Level". By default WS-Addressing is enabled for all the services hosted in WSO2 Application Server. Therefore, you do not have to configure anything at the server side to enable WS-Addressing.

Step 2

Now, we will create a new SOAPUI project referring to http://localhost:9763/services/HelloService?wsdl as follows. Make sure to select "Create Test Suite" check box as well.



After creating the project, select greet request under HelloServiceSoap11Binding TestSuite



If you click on run just by providing input string, you will notice that WS-A headers will not be included in the request SOAP message which can be observed if you toggle "Raw" option.

Step 3

Lets add WS-A headers in to the request SOAP message. Select greet request and click on WS-A option and select the following properties.

Enable WS-A addressing

Add default wsa:Action

Add default wsa:To

Randomly generated message ID




Keep the rest of the default values and click on run. You will notice the following WS-A headers in the request message if you look at the raw message in SOAPUI.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greet</wsa:Action>
<wsa:MessageID>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:MessageID>
<wsa:To>http://localhost:9763/services/HelloService.HelloServiceHttpSoap11Endpoint/</wsa:To>
</soapenv:Header>

Also, the response headers will be similar to the following.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greetResponse</wsa:Action>
<wsa:RelatesTo>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:RelatesTo>
</soapenv:Header>

Simple.. isn't it? we just invoked a service with WS-Addressing headers using SOAPUI.

Here, we manually verified that the response message includes the correct WS-A headers. We looked at the response message and verified each header. If you maintain an automated test suite to verify your web service interactions, the above procedure cannot not be considered as over until you add an assertion so that the test automatically reports the correctness or failures of response/request messages.

Step 4

Now, we will add an assertion to validate response headers. In this example, we are going to add an XPath match assertion to check whether the wsa:Action header of the response equals to urn:greetResponse

Click on Assertions option and select "Add an assertion to this item" icon. Then select, XPAth Match as the assertion.



Next, you should specify the XPath expression and expected result as follows. First, click on Declare button which will automatically define the namespaces. Specify //wsa:Action as the xpath. Also, specify urn:greetResponse as the expected result.



Run the test again. You will notice that the test will be marked in green denoting that the test is successful.

We are honored - Hats off to WSO2 QA team!!

When I joined WSO2 back in 2006 October, I always had a doubt whether I could survive in a world full of various technologies. Before joining WSO2, I used to work under well defined processes, managed by various policies, tests were driven by comprehensive requirements specs, Spent time with the QA teams writing detailed test cases etc.. But I was challenged to test Apache Axis2 at WSO2 as my first duty without having any of those materials. Apache Axis2 is a web service engine, middleware used by developers. So, ideally, I had to look at it through middleware user's in other words web service developer's perspective. It was a great challenge for a person who was new to web services and it was indeed a different experience for a QA engineer.

At the same time, Evanthika, who joined WSO2 one month before me, and myself started testing the first WSO2 middleware product, WSO2 WSAS. First, we started to write comprehensive test cases as we do in traditional QA processes. Later on, we understood that our methodology did not go along with WSO2's vision. We thought to adopt to a completely different agile testing mechanism and started to act fast without being an overhead for a fast-moving organization. We learned everyday, talked to developers and clarified doubts, lived with the product and uncovered a lot of critical bugs. We tried every attempt to report the real bugs and improve quality of the products with each releases.

In early 2007, Evanthika and myself were allocated to test Apache Synapse and WSO2 ESB which demanded us to explore different methods for testing because it was a completely different experience for us. Eventually, we defined our own testing methodology which nicely moved along with WSO2's release processes.

After 4 years, our team grew up and became highly effective with a lot of hard work done by our great team, Krishantha, Yumani, Chamara, Ishani, Pavithra, Nirodha, Thilini, Chamara Anuradha. Now I'm confident that we are in a position to accept testing of any product with very short ramp-up period. Our lightweight process helps everyone to focus on uncovering bugs fast as well as move forward with various different technologies used by WSO2. It is not an easy task to test complex middleware products as I explained in a previous blog post. We have built the foundation for the new testers to join our team and move along with the world of complex middleware testing.

With all these efforts during past few years, we have been awarded as the outstanding team in 2010 at the WSO2 awards ceremony which was held yesterday at Cinnamon lakeside, Colombo. As Paul Fremantle mentioned during the ceremony, QA teams are kept aside and do not appreciate their work much in most organizations. But WSO2 believes the values of each team and always respect the hard work performed by our QA team.

I agree that we are not yet perfect and there are situations some bugs slipped through our test cycles. But we always try our best to maintain an acceptable quality with all releases. It is evident by the usage of our products in large organizations such as eBay, Deutche Bank etc..

Are you doing software testing or quality assurance?

While going through some of the software QA related stuff, I found a very nice presentation done by Michael Bolton. There, he challenged the traditional way of defining software quality assurance teams. While agreeing with his insight on QA, I thought to highlight the contents included in one slide of his presentation.

If you are involved in software quality assurance (or if you think that you are a software QA person) then the following will be able to help you to get yourself categorized into the correct role that you are playing in your organization.

Do you

• design the product?
• hire programmers?
• decide which bugs to fix?
• allocate staff?
• set the schedule?
• fix problems in code?
• decide on raises?
• allocate training budgets?
• produce manuals?
• choose the development model?
• fire some programmers?
• control the budget?
• set the company's strategic direction?

If the answer for all of the above questions is , YES, then you assure the quality of the products produced by your organization. So you are doing software QA!

I have never found a QA/testing team who performs all of the above activities in their job. But most of us test the products. We explore the products in order to reveal the issues and provide feedback to the stakeholders of the product under development. Therefore our role is Software testing NOT quality assurance.

James Bach defines software testing as questioning the product in order to evaluate it.

Cem Kaner says software testing is a technical, empirical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek.

So, there is no assurance!

Are you testing or checking?

Just found a nice post from Micheal Bolton who expressed his opinions about the difference between Testing and Checking.

Extracted from the post;

Checking Is Confirmation

Checking is something that we do with the motivation of confirming existing beliefs. Checking is a process ofconfirmation, verification, and validation. When we already believe something to be true, we verify our belief bychecking. We check when we’ve made a change to the code and we want to make sure that everything that worked before still works. When we have an assumption that’s important, we check to make sure the assumption holds. Excellent programmers do a lot of checking as they write and modify their code, creating automated routines that they run frequently to check to make sure that the code hasn’t broken. Checking is focused on making sure that the program doesn’t fail.

Testing Is Exploration and Learning

Testing is something that we do with the motivation of finding new information. Testing is a process of exploration, discovery, investigation, and learning. When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing. We’re testing when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before. As James Bach and I say in our Rapid Software Testing classes, testing is focused on “learning sufficiently everything that matters about how the program works and about how it might not work.”