Test Reviews - An efficient way to validate test coverage

When the complexity of testing increases, deriving test scenarios becomes extremely hard. Individual testers are not always capable of thinking all aspects of the products and finding out test cases. In agile processes which demands frequent releases, this becomes much harder since individual testers do not get sufficient time to learn and explore products.

At WSO2, code reviews have been an integral part of engineering process. Code review groups get together and go through different code segments in weekly basis.

Similar to code reviews, WSO2 QA team conducts weekly test reviews. The primary objective of reviewing tests is to finding out gaps in a particular product's testing methodology.

Usually there are 2 main actors take part in a test review meeting.

- Tester(s) who test a particular feature

- Developer (s) who implemented the feature

Tester goes through the existing test scenarios and developer provides his or her feedback on the tests. Specially when a particular use case is not clear enough, tester can clarify it during reviews very effective manner.

It is the testers responsibility to try out the new test scenarios which have been captured during the review in next release cycles.

Test review will not take more than 1 hour and testers should be prepared properly to get the maximum out of this short time and capture more and more test scenarios.

Similar to the manual test scenarios, all automated tests can also be reviewed during test reviews.

In agile processes where you do not have detailed system specs to derive tests, test reviews are one of the most useful mechanisms to validate your test coverage.

How to enable child-first class loading in WSO2 Application Server or Axis2

By default, WSO2 Application Server (or Axis2) uses parent-first class loading mechanism.

If you deploy an AAR service, which can load classes from the following locations.

• CARBON_HOME/lib (CARBON_HOME is the location where you installed WSO2 Application Server)
• CARBON_HOME/repository/deployment/server/axis2services/lib
• AAR service/lib (lib directory under your service archive)

If your service implementation class has a package import for a class, which is available in both CARBON_HOME/repository/deployment/server/axis2services/lib and the lib directory under service archive, the class placed under CARBON_HOME/repository/deployment/server/axis2services/lib will get loaded.

Which means, the parent class always get loaded first.

The class loading sequence is CARBON_HOME/lib ---> CARBON_HOME/repository/deployment/server/axis2services/lib -----> AAR service/lib

Sometimes, we want to load the class from service archive lib first without loading the class which is available either one of above parent locations. In other words, child-first class loading will be required for some instances.

child-first class loading can be enabled simply by adding the following parameter in to services.xml in your service archive.

<parameter name="EnableChildFirstClassLoading">true</parameter>

If you set this parameter in axis2.xml of your WSO2 Application Server (or Axis2) instance, all services will use child-first class loading mechanism.

WS-Addressing with SOAPUI

WS-Addressing is a standard way of including message routing data within SOAP headers without relying on transport specific routing properties. If you are talking to a web service which expects WS-Addressing information in request SOAP messages, you could either follow a programmatic approach to write a client and insert WS-A headers manually or use a commercial or free web service client tool. SOAPUI is the simplest tool which can be used to insert WS-Addressing headers in to request SOAP messages.
In this blog post, we will;

• Talk to a WS-Addressing enabled service which is hosted in WSO2 Application Server using SOAPUI

• Assert response SOAP message to validate WS-A headers

Pre-requisites:

1. Download and install the free version of SOAPUI 3.0.1 or later
2. Download and install WSO2 Application Server 4.0

Step 1

We are going to invoke the default HelloService which is shipped with WSO2 Application Server. Go to WSO2_APPSERVER/home/bin and start the server by running wso2server.sh{bat}
Access management console with https://localhost:9443/carbon and log in using the default administrator credentials (username=admin, password=admin)
Navigate to Manage --> Services --> List in the left menu and select "HelloService"
You will be directed to the dashboard of HelloService. Click on Modules link.



You will notice that "addressing-3.1.0" is listed under "Globel Level". By default WS-Addressing is enabled for all the services hosted in WSO2 Application Server. Therefore, you do not have to configure anything at the server side to enable WS-Addressing.

Step 2

Now, we will create a new SOAPUI project referring to http://localhost:9763/services/HelloService?wsdl as follows. Make sure to select "Create Test Suite" check box as well.



After creating the project, select greet request under HelloServiceSoap11Binding TestSuite



If you click on run just by providing input string, you will notice that WS-A headers will not be included in the request SOAP message which can be observed if you toggle "Raw" option.

Step 3

Lets add WS-A headers in to the request SOAP message. Select greet request and click on WS-A option and select the following properties.

Enable WS-A addressing

Add default wsa:Action

Add default wsa:To

Randomly generated message ID




Keep the rest of the default values and click on run. You will notice the following WS-A headers in the request message if you look at the raw message in SOAPUI.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greet</wsa:Action>
<wsa:MessageID>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:MessageID>
<wsa:To>http://localhost:9763/services/HelloService.HelloServiceHttpSoap11Endpoint/</wsa:To>
</soapenv:Header>

Also, the response headers will be similar to the following.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greetResponse</wsa:Action>
<wsa:RelatesTo>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:RelatesTo>
</soapenv:Header>

Simple.. isn't it? we just invoked a service with WS-Addressing headers using SOAPUI.

Here, we manually verified that the response message includes the correct WS-A headers. We looked at the response message and verified each header. If you maintain an automated test suite to verify your web service interactions, the above procedure cannot not be considered as over until you add an assertion so that the test automatically reports the correctness or failures of response/request messages.

Step 4

Now, we will add an assertion to validate response headers. In this example, we are going to add an XPath match assertion to check whether the wsa:Action header of the response equals to urn:greetResponse

Click on Assertions option and select "Add an assertion to this item" icon. Then select, XPAth Match as the assertion.



Next, you should specify the XPath expression and expected result as follows. First, click on Declare button which will automatically define the namespaces. Specify //wsa:Action as the xpath. Also, specify urn:greetResponse as the expected result.



Run the test again. You will notice that the test will be marked in green denoting that the test is successful.

We are honored - Hats off to WSO2 QA team!!

When I joined WSO2 back in 2006 October, I always had a doubt whether I could survive in a world full of various technologies. Before joining WSO2, I used to work under well defined processes, managed by various policies, tests were driven by comprehensive requirements specs, Spent time with the QA teams writing detailed test cases etc.. But I was challenged to test Apache Axis2 at WSO2 as my first duty without having any of those materials. Apache Axis2 is a web service engine, middleware used by developers. So, ideally, I had to look at it through middleware user's in other words web service developer's perspective. It was a great challenge for a person who was new to web services and it was indeed a different experience for a QA engineer.

At the same time, Evanthika, who joined WSO2 one month before me, and myself started testing the first WSO2 middleware product, WSO2 WSAS. First, we started to write comprehensive test cases as we do in traditional QA processes. Later on, we understood that our methodology did not go along with WSO2's vision. We thought to adopt to a completely different agile testing mechanism and started to act fast without being an overhead for a fast-moving organization. We learned everyday, talked to developers and clarified doubts, lived with the product and uncovered a lot of critical bugs. We tried every attempt to report the real bugs and improve quality of the products with each releases.

In early 2007, Evanthika and myself were allocated to test Apache Synapse and WSO2 ESB which demanded us to explore different methods for testing because it was a completely different experience for us. Eventually, we defined our own testing methodology which nicely moved along with WSO2's release processes.

After 4 years, our team grew up and became highly effective with a lot of hard work done by our great team, Krishantha, Yumani, Chamara, Ishani, Pavithra, Nirodha, Thilini, Chamara Anuradha. Now I'm confident that we are in a position to accept testing of any product with very short ramp-up period. Our lightweight process helps everyone to focus on uncovering bugs fast as well as move forward with various different technologies used by WSO2. It is not an easy task to test complex middleware products as I explained in a previous blog post. We have built the foundation for the new testers to join our team and move along with the world of complex middleware testing.

With all these efforts during past few years, we have been awarded as the outstanding team in 2010 at the WSO2 awards ceremony which was held yesterday at Cinnamon lakeside, Colombo. As Paul Fremantle mentioned during the ceremony, QA teams are kept aside and do not appreciate their work much in most organizations. But WSO2 believes the values of each team and always respect the hard work performed by our QA team.

I agree that we are not yet perfect and there are situations some bugs slipped through our test cycles. But we always try our best to maintain an acceptable quality with all releases. It is evident by the usage of our products in large organizations such as eBay, Deutche Bank etc..

Are you doing software testing or quality assurance?

While going through some of the software QA related stuff, I found a very nice presentation done by Michael Bolton. There, he challenged the traditional way of defining software quality assurance teams. While agreeing with his insight on QA, I thought to highlight the contents included in one slide of his presentation.

If you are involved in software quality assurance (or if you think that you are a software QA person) then the following will be able to help you to get yourself categorized into the correct role that you are playing in your organization.

Do you

• design the product?
• hire programmers?
• decide which bugs to fix?
• allocate staff?
• set the schedule?
• fix problems in code?
• decide on raises?
• allocate training budgets?
• produce manuals?
• choose the development model?
• fire some programmers?
• control the budget?
• set the company's strategic direction?

If the answer for all of the above questions is , YES, then you assure the quality of the products produced by your organization. So you are doing software QA!

I have never found a QA/testing team who performs all of the above activities in their job. But most of us test the products. We explore the products in order to reveal the issues and provide feedback to the stakeholders of the product under development. Therefore our role is Software testing NOT quality assurance.

James Bach defines software testing as questioning the product in order to evaluate it.

Cem Kaner says software testing is a technical, empirical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek.

So, there is no assurance!

Are you testing or checking?

Just found a nice post from Micheal Bolton who expressed his opinions about the difference between Testing and Checking.

Extracted from the post;

Checking Is Confirmation

Checking is something that we do with the motivation of confirming existing beliefs. Checking is a process ofconfirmation, verification, and validation. When we already believe something to be true, we verify our belief bychecking. We check when we’ve made a change to the code and we want to make sure that everything that worked before still works. When we have an assumption that’s important, we check to make sure the assumption holds. Excellent programmers do a lot of checking as they write and modify their code, creating automated routines that they run frequently to check to make sure that the code hasn’t broken. Checking is focused on making sure that the program doesn’t fail.

Testing Is Exploration and Learning

Testing is something that we do with the motivation of finding new information. Testing is a process of exploration, discovery, investigation, and learning. When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing. We’re testing when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before. As James Bach and I say in our Rapid Software Testing classes, testing is focused on “learning sufficiently everything that matters about how the program works and about how it might not work.”

WSO2 Application Server - Run web applications and web services on the same server

Today WSO2 released its newest product (rather a renamed version of WSO2 web services application server - WSAS, with a lot of improvements), WSO2 Application Server - 4.0.0

WSO2 Application server is an unified platform to host web services and web applications together. This completely open-source product is based on world's most popular web application server, Apache Tomcat and No:1 web service engine, Apache Axis2.

WSO2 Application Server is part of WSO2 Carbon platform hence it brings the power of highly flexible component model hence your web applications can utilize the features provided by carbon platform. For example, web applications can be authorized using WSO2 identity and user management components. As the other WSO2 products, WSO2 application server comes with a rich graphical management console which allows you to manage your web applications very easy and effective manner.

Lets have a glance at WSO2 Application Server......

Step 1

Download WSO2 Application Server-4.0.0 binary distribution from wso2 oxygen tank

Extract the downloaded binary. We will refer to the downloaded directory as CARBON_HOME

Step 2

Go to CARBON_HOME/bin directory and run wso2server.sh {bat}

Once the server is started, access management console using https://localhost:9443/carbon

Log in to management console using the admin credentials (username= admin, password=admin)

Step 3

At the main menu, you will find, Manage --> Web Application option. You can upload new web apps or view the existing web applications in there.
Click on Manage --> Web Applications --> Add
You will be directed to web app uploading screen where you can upload any war file.



Browse for a web application archive (war) in your file system and click on upload. You will be prompted with the confirmation message if the deployment is successful.

Step 4

Once the web app deployment is done, it will be listed in Running Web Applications page as shown below.



If you click on web application context in the web app list, you will be directed to web app dashboard as follows. You can carry out multiple operations on the deployed web app within this dashboard. For example, you can reload web app, expire all sessions etc..



Step 5

As I described at the beginning, you can deploy various kind of web services together with web applications using WSO2 Application Server. You will find Manage --> Services menu option at the left navigation pane in management console, there you will see Axis2 Servces, Jar Services, Spring Services and JAX-WS service deployment options as shown below.



Download WSO2 Application Server-4.0.0 today and try this out your self.