We are honored - Hats off to WSO2 QA team!!

When I joined WSO2 back in 2006 October, I always had a doubt whether I could survive in a world full of various technologies. Before joining WSO2, I used to work under well defined processes, managed by various policies, tests were driven by comprehensive requirements specs, Spent time with the QA teams writing detailed test cases etc.. But I was challenged to test Apache Axis2 at WSO2 as my first duty without having any of those materials. Apache Axis2 is a web service engine, middleware used by developers. So, ideally, I had to look at it through middleware user's in other words web service developer's perspective. It was a great challenge for a person who was new to web services and it was indeed a different experience for a QA engineer.

At the same time, Evanthika, who joined WSO2 one month before me, and myself started testing the first WSO2 middleware product, WSO2 WSAS. First, we started to write comprehensive test cases as we do in traditional QA processes. Later on, we understood that our methodology did not go along with WSO2's vision. We thought to adopt to a completely different agile testing mechanism and started to act fast without being an overhead for a fast-moving organization. We learned everyday, talked to developers and clarified doubts, lived with the product and uncovered a lot of critical bugs. We tried every attempt to report the real bugs and improve quality of the products with each releases.

In early 2007, Evanthika and myself were allocated to test Apache Synapse and WSO2 ESB which demanded us to explore different methods for testing because it was a completely different experience for us. Eventually, we defined our own testing methodology which nicely moved along with WSO2's release processes.

After 4 years, our team grew up and became highly effective with a lot of hard work done by our great team, Krishantha, Yumani, Chamara, Ishani, Pavithra, Nirodha, Thilini, Chamara Anuradha. Now I'm confident that we are in a position to accept testing of any product with very short ramp-up period. Our lightweight process helps everyone to focus on uncovering bugs fast as well as move forward with various different technologies used by WSO2. It is not an easy task to test complex middleware products as I explained in a previous blog post. We have built the foundation for the new testers to join our team and move along with the world of complex middleware testing.

With all these efforts during past few years, we have been awarded as the outstanding team in 2010 at the WSO2 awards ceremony which was held yesterday at Cinnamon lakeside, Colombo. As Paul Fremantle mentioned during the ceremony, QA teams are kept aside and do not appreciate their work much in most organizations. But WSO2 believes the values of each team and always respect the hard work performed by our QA team.

I agree that we are not yet perfect and there are situations some bugs slipped through our test cycles. But we always try our best to maintain an acceptable quality with all releases. It is evident by the usage of our products in large organizations such as eBay, Deutche Bank etc..

Are you doing software testing or quality assurance?

While going through some of the software QA related stuff, I found a very nice presentation done by Michael Bolton. There, he challenged the traditional way of defining software quality assurance teams. While agreeing with his insight on QA, I thought to highlight the contents included in one slide of his presentation.

If you are involved in software quality assurance (or if you think that you are a software QA person) then the following will be able to help you to get yourself categorized into the correct role that you are playing in your organization.

Do you

• design the product?
• hire programmers?
• decide which bugs to fix?
• allocate staff?
• set the schedule?
• fix problems in code?
• decide on raises?
• allocate training budgets?
• produce manuals?
• choose the development model?
• fire some programmers?
• control the budget?
• set the company's strategic direction?

If the answer for all of the above questions is , YES, then you assure the quality of the products produced by your organization. So you are doing software QA!

I have never found a QA/testing team who performs all of the above activities in their job. But most of us test the products. We explore the products in order to reveal the issues and provide feedback to the stakeholders of the product under development. Therefore our role is Software testing NOT quality assurance.

James Bach defines software testing as questioning the product in order to evaluate it.

Cem Kaner says software testing is a technical, empirical investigation of a product, done on behalf of stakeholders, with the intention of revealing quality-related information of the kind that they seek.

So, there is no assurance!

Are you testing or checking?

Just found a nice post from Micheal Bolton who expressed his opinions about the difference between Testing and Checking.

Extracted from the post;

Checking Is Confirmation

Checking is something that we do with the motivation of confirming existing beliefs. Checking is a process ofconfirmation, verification, and validation. When we already believe something to be true, we verify our belief bychecking. We check when we’ve made a change to the code and we want to make sure that everything that worked before still works. When we have an assumption that’s important, we check to make sure the assumption holds. Excellent programmers do a lot of checking as they write and modify their code, creating automated routines that they run frequently to check to make sure that the code hasn’t broken. Checking is focused on making sure that the program doesn’t fail.

Testing Is Exploration and Learning

Testing is something that we do with the motivation of finding new information. Testing is a process of exploration, discovery, investigation, and learning. When we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated, we’re testing. We’re testing when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before. As James Bach and I say in our Rapid Software Testing classes, testing is focused on “learning sufficiently everything that matters about how the program works and about how it might not work.”

WSO2 Application Server - Run web applications and web services on the same server

Today WSO2 released its newest product (rather a renamed version of WSO2 web services application server - WSAS, with a lot of improvements), WSO2 Application Server - 4.0.0

WSO2 Application server is an unified platform to host web services and web applications together. This completely open-source product is based on world's most popular web application server, Apache Tomcat and No:1 web service engine, Apache Axis2.

WSO2 Application Server is part of WSO2 Carbon platform hence it brings the power of highly flexible component model hence your web applications can utilize the features provided by carbon platform. For example, web applications can be authorized using WSO2 identity and user management components. As the other WSO2 products, WSO2 application server comes with a rich graphical management console which allows you to manage your web applications very easy and effective manner.

Lets have a glance at WSO2 Application Server......

Step 1

Download WSO2 Application Server-4.0.0 binary distribution from wso2 oxygen tank

Extract the downloaded binary. We will refer to the downloaded directory as CARBON_HOME

Step 2

Go to CARBON_HOME/bin directory and run wso2server.sh {bat}

Once the server is started, access management console using https://localhost:9443/carbon

Log in to management console using the admin credentials (username= admin, password=admin)

Step 3

At the main menu, you will find, Manage --> Web Application option. You can upload new web apps or view the existing web applications in there.
Click on Manage --> Web Applications --> Add
You will be directed to web app uploading screen where you can upload any war file.



Browse for a web application archive (war) in your file system and click on upload. You will be prompted with the confirmation message if the deployment is successful.

Step 4

Once the web app deployment is done, it will be listed in Running Web Applications page as shown below.



If you click on web application context in the web app list, you will be directed to web app dashboard as follows. You can carry out multiple operations on the deployed web app within this dashboard. For example, you can reload web app, expire all sessions etc..



Step 5

As I described at the beginning, you can deploy various kind of web services together with web applications using WSO2 Application Server. You will find Manage --> Services menu option at the left navigation pane in management console, there you will see Axis2 Servces, Jar Services, Spring Services and JAX-WS service deployment options as shown below.



Download WSO2 Application Server-4.0.0 today and try this out your self.

QA testing in agile world - Attitude matters most!

Few short tips about a viable approach for testing in agile world. May be valid for most organizations.

• Understand your development team first. Study each engineer and adjust your way of acting (communication, management etc..) according to the team you are working on

• Do not act as a quality police. Everyone is responsible for quality. It is your task to guide the others NOT finding faults and complain
  

• Be as flexible as you can in all situations

• Be quick on everything - learn, design, implement, configure, deploy quickly and test fast
  

• Be innovative

• Identify highly risked features first and start hunting bugs

Finally;

• CREATE A VALUE as a tester for your organization and BUILD the CONFIDENCE among the team!
  

Invoking secure web services using SOAPUI - part1

SOAPUI is a very useful free tool which can be used in SOA testing. Service invocation using SOAPUI is straight forward and you can find a lot of references by surfing web. However, there are limited information about invoking services with various QOS (Quality of Service) features such as WS-Security, WS-Addressing etc.

This post takes you through the simplest QOS scenario, a web service is secured with UserName token policy and how SOAPUI is used to invoke such a service.

I will use the default HelloService hosted in WSO2 WSAS in this demonstration but the service invocation approach is similar with any service provider.

Pre-requisites:

1. Download and install SOAPUI-3.6.1 or later (free version)

2. Download and install WSO2 WSAS-3.*

Step 1

First, we need to secure a web service hosted in WSAS. We will configure HelloService so that only the users in admin group will be allowed to invoke the service. In order to do that, start WSO2 WSAS and log in to management console. Then, select HelloService from Deployed Services page.

You will be directed to the service dashboard of HelloService as follows.

Click on Security link and Select "Enable Security" option. Then select, UserNameToken security scenario.

In the next screen, select admin user group and click on finish to apply the security policy for HelloService.

Step 2
We have configured server side security policy. If you look at the WSDL of HelloService (http://localhost:9763/services/HelloService?wsdl), you will notice that UserNameToken security policy is added to the service.
Now, we need to configure SOAPUI project to talk to HelloService with the required user credentials.

First, start SOAPUI and create a new project. Specify https://localhost:9443/services/HelloService?wsdl as the initial WSDL. Keep the other default settings.
You will see the following project structure.



Replace ? with an input value in the request editor and run the test. You will encounter a SOAPFault, Missing wsse:Security header in request, since we did not send security headers with the request.
Lets configure client side security now. In the Request Properties pane of SOAPUI project, you will find the following properties.

Username
Password
WSS-Password Type
WSS TimeToLive

Specify the following values for the above properties.

Username = admin
Password = admin
WSS-Password Type = PasswordText
WSS TimeToLive = 2000

Run the test. Have a look at the raw request view. You will see security header is added to the request.
Thats all! We will look at more advanced scenarios such as signing and encrypting messages with SOAPUI in future posts.

Java bench - Simple and lightweight service load testing tool

At WSO2, we use various types of tools and programmatic approaches to load, stress and performance test SOA middleware products. Out of them, one particular tool is part of almost all engineers workspaces. It is Java bench (or java-ab).

Java bench is the java clone of the popular ApacheBench load generator customized for better HTTP1.1 support.

Java bench is a very simple and easy to use utility. One of the greatest advantages of using it for performance testing is, its lesser overhead on load generation machines. Most of the load generators consist of rich UIs and reporting utilities hence the load generator it self consumes considerable amount of CPU and memory when you run performance testing in single server. It is always not practical to run load generators in separate machines. Sometimes we need to run load/performance tests against a server running on a local machine in order to provide quick feedback about the product under test or reproduce perf/memory issues. In these situations, Java-bench is the ideal solution.

Lets see how we can run a basic performance test using java bench.

Pre-requisites:

Linux or windows OS

jdk1.5 or higher

Step 1

Download Java bench from here and extract the downloaded zip file.

Step 2

Go to the root of the extracted directory and run 'java -jar benchmark.jar' to see the available options.

usage: HttpBenchmark [options] [http://]hostname[:port]/path

-v Set verbosity level - 4 and above prints response

content, 3 and above prints information on headers, 2 and above prints

response codes (404, 200, etc.), 1 and above prints warnings and info.

-H

Add arbitrary header line, eg. 'Accept-Encoding:

gzip' inserted after all normal header lines. (repeatable)

-n Number of requests to perform for the benchmarking

session. The default is to just perform a single request which usually

leads to non-representative benchmarking results.

-T Content-type header to use for POST data.

-c Concurrency while performing the benchmarking

session. The default is to just use a single thread/client.

-h Display usage information.

-i Do HEAD requests instead of GET.

-k Enable the HTTP KeepAlive feature, i.e., perform

multiple requests within one HTTP session. Default is no KeepAlive

-p File containing data to POST.

Step 3

We will run a load test against one of the publicly available web services. I use a simple web service hosted in xmethods which generates a list of prime numbers that are less than a specified max value.

First you need to invoke this service using SOAPUI or some other client and capture the SOAP request message. Then save it as a xml in your file system.

Sample request:

<?xml version="1.0" encoding="UTF-8"?>

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://microsoft.com/webservices/">

<soapenv:Header/>

<soapenv:Body>

<web:GetPrimeNumbers>

<web:max>56 </web:max>

</web:GetPrimeNumbers>

</soapenv:Body>

</soapenv:Envelope>

Step 4

Now, we can use java bench to send series of requests to the above web service as follows.

java -jar benchmark.jar -p SoapRequest.xml -n 100 -c 10 -k -H "SOAPAction: http://microsoft.com/webservices/GetPrimeNumbers" -T "text/xml; charset=UTF-8" http://www50.brinkster.com/vbfacileinpt/np.asmx

As you can see, we specify the soap request xml with -p parameter. -c specifies the no.of concurrent threads/clients used in the test. -n is used to specify the no.of requests transmitted per a keep-alive connection. In our example, each client carries 100 requests therefore 1000 total requests will be sent to the service. -k enables HTTP keep-alive so that multiple requests share the same HTTP session. -H is used to specify the HTTP headers. In our example, SOAPAction HTTP header is appended to the message. You can find the relevant SOAPAction from the WSDL of the service. -T is used to specify the Content-Type header of the request. The last parameter is the endpoint of target service.

After the test is completed, you will see the result as shown below.

Server Software: Microsoft-IIS/6.0

Server Hostname: www50.brinkster.com

Server Port: 80

Document Path: http://www50.brinkster.com/vbfacileinpt/np.asmx

Document Length: 429 bytes

Concurrency Level: 10

Time taken for tests: 59.378401 seconds

Complete requests: 1000

Failed requests: 0

Write errors: 0

Total transferred: 756000 bytes

Requests per second: 16.84 [#/sec] (mean)

Time per request: 593.784 [ms] (mean)

Time per request: 59.378 [ms] (mean, across all concurrent requests)

Transfer rate: 7.22 [Kbytes/sec] received

5.51 kb/s sent

12.73 kb/s total

Simple. Isn't it? We will look in to more interesting tests with java-bench in future posts.