WSO2 Carbon-3.2.0 - The latest with many new features and a release to remember

On 12th of June 2011, the latest version of WSO2 SOA middleware platform, WSO2 Carbon-3.2.0 has been released. Since 2009 January, WSO2 SOA middleware platform has been grown and matured and now it consists of 12 different products.

WSO2 Application Server

WSO2 Enterprise Service Bus

WSO2 Mashup Server

WSO2 Identity Server

WSO2 Governance Registry

WSO2 Data Services Server

WSO2 Business Process Server

WSO2 Business Activity Monitor

WSO2 Gadget Server

WSO2 Business Rules Server

WSO2 Complex Event Processing Server

WSO2 Message Broker

In this latest version, two new members joined into WSO2 Carbon product family. Those are WSO2 Complex Event Processing Server and WSO2 Message Broker.

In this post, I'm not going to explain the new features and enhancements included in each of these products. You can find them out in the respective product pages in wso2.com.

But, I would like to share how we evolve in terms of features, process and release methodology during the last couple of years.

In late 2008, we started the implementation of WSO2 Carbon platform. Before that, we had four isolated products (WSO2 ESB, WSO2 WSAS, WSO2 IS and WSO2 Registry) but those had limitations to work as a platform. The major objective of WSO2 Carbon platform was to build a suite of products which can be integrated easily and run together with minimum configuration overhead. At the end of the first WSO2 Carbon platform release (version 1.5.0), we were able to achieve that objective to some extent. The first release was a VERY different experience for us since we used to do releases with one product at a time so that the test team had sufficient time and resources to try out various scenarios. Because of that, we had to move in to a totally different testing paradigm.

In early 2009, we had to do a patch release of WSO2 Carbon platform (version 1.5.1) because we uncovered some critical issues after the first Carbon release. With this, we badly felt the need of adjusting our testing methodology to align with WSO2 platform vision. We spent hours in each build to test management console UIs manually and our first objective was to cut down that time and effort. In order to do that, we started developing a selenium based test automation framework. In 2009, we did 3 WSO2 Carbon platform releases (version 2.0.0, 2.0.1, 2.0.2) and we were able to make use of selenium tests heavily in all those release cycles.

In 2010, we went through a major UI refactoring process across the whole WSO2 Carbon platform. Unfortunately, we could not keep maintaining selenium tests with the pace of UI changes. Therefore, we wanted to have a test framework with minimum maintenance overhead. We wanted to call methods of various features bypassing UI elements so that the frequent UI changes do not break the tests. We started admin services based test automation framework, which used to call the operations of administration services associated with each of the features programmatically (Java/Junit).

2010 was an extremely busy year for most of us since we did a lot of frequent releases, but there were very limited automated tests. At the end of 2010, WSO2 Carbon became a fully componentized, modular SOA middleware platform which consisted of multi-tenancy and a lot of usability enhancements. As the testing team, we kept on observing the growth of WSO2 Carbon platform more than anyone else since we deal with almost all aspects of each and every product. As I said at the beginning, one of the objectives of WSO2 product platform was to provide users with ease of integration. By end of 2010, we realized how far we achieved that. Installing a new feature on any of the WSO2 Carbon based product became a simple click'n'click process. Most features worked out-of-the-box.

Now, after 2.5 years of the first Carbon release, our product platform has been improved in various aspects with 12 different products. Our development and testing methodology has been changing with each release to accommodate the pace of growth and objectives in WSO2 SOA platform. In the latest 3.2.0 release, our focus shifted towards more platform level testing, which required us to learn and familiar with each product. With the platform aspect, we cannot do individual product releases. We do all 12 products at once, which demands huge work load on both development and testing teams. At WSO2, we do not trust or rely on the number of testers in the team but their individual capabilities. We realize the need of having 80% of test coverage through some kind of automation in order to sustain with the pace. For all these years, we do not adopt or follow any specific process blindly because that works for some other organization so that we do. Instead, we are following a context-driven testing approach. We learn and enhance our processes with our own mistakes and experiences. While WSO2 is revolutionizing the world of enterprise middleware, testing and development methodology of middleware will also be redefined.

WSO2 SOA middleware Deployment Tips - 1 - validationQuery to avoid broken DB connections

I thought to put together some best practices, guidelines on deploying WSO2 SOA middleware platform. This post will remind you a well-known best practice used in most of the product deployments.

When you are maintaining DB connections, it is always recommended to use a validationQuery to check the health of the TCP connection of the connections stay in DB connection pool.

Because the connection opening is an expensive and time consuming operation, after a connection is created, it will be kept open for a specific time in the pool. When re-using these connections from the pool, there can be situations that the TCP connection to the DB is interrupted and the connection consumer gets errors such as communication link failures etc..

In order to avoid that, a validationQuery, an SQL statement specific to DBMS type, can be used which runs before using the connection.



In WSO2 middleware platform, you usually use central governance/configuration registry to store SOA metadata and various governance tasks. Governance registry supports multiple DBs as underlying data stores. You establish the connection to DB in CARBON_HOME/repository/conf/registry.xml as follows.

<dbConfig name="wso2registry">

<url>jdbc:mysql://localhost:3306/config_db</url>

<userName>>regadmin</userName>

<password>regadmin</password>

<driverName>com.mysql.jdbc.Driver</driverName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<minIdle>5</minIdle>

</dbConfig>

This is one of the places where you may experience DB connection issues as I explained above. Therefore, it is always a best practice to use a validationQuery.

If your DBMSs is MySQL or MSSQL, then use the following.

<validationQuery>SELECT 1</validationQuery>

In Oracle;

<validationQuery>SELECT 1 FROM DUAL</validationQuery>

In Postgres,

<validationQuery>SELECT version(); </validationQuery>

Similarly, when you make DB connections in WSO2 Data Services Server, make sure to use validationQuery in data source definition section as follows.

<property name="org.wso2.ws.dataservice.validationquery">SELECT 1</property>

Monitoring WSO2 ESB using JConsole

My latest tutorial on how to monitor WSO2 ESB using JConsole is now published on WSO2 Oxygen Tank

How to send JSON messages to web services deployed on WSO2 Application Server

JSON (JavaScript Object Notation) is an open and text-based data exchange format, that provides a standardized data exchange format better suited for Ajax style web applications. You can find more information about JSON from www.json.org

WSO2 SOA middleware platform supports sending and receiving JSON messages.

This post takes you through the steps to deploy a simple web service in WSO2 Application Server and write a client using Axis2 ServiceClient API to invoke the service by sending JSON message.

Pre-requisites:

Download and install WSO2 Application Server 4.0.0 or later

Step 1

We are using a simple web service, which echo's user input as follows.

public class EchoService {

 public OMElement echo(OMElement element) {
           return element;
    }
}

You can download the service archive (EchoService.aar) from here and deploy on WSO2 Application Server.

Step 2

Have a look at CARBON_HOME/repository/conf/axis2.xml (CARBON_HOME is the root directory of WSO2 Application Server). You will notice that the JSON specific message builders and formatters are enabled by default.

<!--JSON Message Formatters-->

<messageFormatter contentType="application/json"

class="org.apache.axis2.json.JSONMessageFormatter"/>

<messageFormatter contentType="application/json/badgerfish"

class="org.apache.axis2.json.JSONBadgerfishMessageFormatter"/>

<messageFormatter contentType="text/javascript"

class="org.apache.axis2.json.JSONMessageFormatter"/>

<!--JSON Message Builders-->

<messageBuilder contentType="application/json"

class="org.apache.axis2.json.JSONOMBuilder"/>

<messageBuilder contentType="application/json/badgerfish"

class="org.apache.axis2.json.JSONBadgerfishOMBuilder"/>

<messageBuilder contentType="text/javascript"

class="org.apache.axis2.json.JSONOMBuilder"/>

WSO2 Application Server accepts any JSON message with the content type application/json, application/json/badgerfish or text/javascript. In this example, we will use a message with the content-type, application/json

Step 3

Now, If you invoke the service using the Tryit utility associated with EchoService and trace the message, you will notice the payload of the SOAP message as follows.

<p:echo xmlns:p="http://service.carbon.wso2.org">

<echo xmlns="http://service.carbon.wso2.org" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">charitha</echo>

</p:echo>

Basically, the message payload will be similar to the following.

<echo>

<value>charitha</value>

</echo>

Step 4

Now, we are going to send the above payload as a JSON message. The JSON format of the above message payload will be as follows.

{"echo":{"value":"charitha"}}

Add the following class to your java project. Compile it using the libraries included in CARBON_HOME/repository/component/plugins directory. You may also need to add CARBON_HOME/lib/endorsed into your class path.

package org.wso2.carbon.service;

import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.methods.PostMethod; import org.apache.commons.httpclient.methods.RequestEntity; import org.apache.commons.httpclient.methods.StringRequestEntity;

public class JSONClient {

    public static void main(String[] args) throws Exception {
          String strURL = "http://localhost:8281/services/JSONProxy/";                  PostMethod post = new PostMethod(strURL);                RequestEntity entity = new StringRequestEntity("{\"echo\":{\"value\":\"charitha\"}}","application/json", "UTF-8");         post.setRequestEntity(entity);         HttpClient httpclient = new HttpClient();         try {             int result = httpclient.executeMethod(post);             System.out.println("Response status code: " + result);             System.out.println("Response body: ");             System.out.println(post.getResponseBodyAsString());         } finally {             post.releaseConnection();        }    } 
}

You could use the same axis2.xml located at CARBON_HOME/repository/conf as the client axis2.xml when creating ConfigurationContext

If you forward the message to tcpmon and trace it, you will see the request as follows.

POST /services/EchoService HTTP/1.1

Content-Type: application/json; charset=UTF-8

User-Agent: WSO2 WSAS-3.2.0

Host: 127.0.0.1:9764

Transfer-Encoding: chunked

1d

{"echo":{"value":"charitha"}}

0

Data driven web service testing with Apache Jmeter

You will not get enough coverage in your web service testing, if you repeatedly send the same request to the webservice under testing. You must read data from a data source and parameterize each request. If you are dealing with SOAP messages, you should parameterize SOAP message payload.
This post guides you how to do data driven web service testing using Apache Jmeter.

Pre-requisites:
Install Apache Jmeter 2.3.4 or later

Step 1

We are going to invoke a publicly available web service, Temerature Conversion service . The WSDL of this web service can be accessible through http://webservices.daehosting.com/services/TemperatureConversions.wso?WSDL
You can invoke the CelciusToFahrenheit operation of this webservice using SOAPUI and capture the request message. It will be similar to the below.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
<soapenv:Header/>
<soapenv:Body>
<tem:CelciusToFahrenheit>
<tem:nCelcius>37</tem:nCelcius>
</tem:CelciusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>
Step 2

Lets create a new JMeter test plan and add a thread group. Then, add SOAP/XML-RPC sampler into the thread group and copy and paste the above SOAP request into the SOAP/XML-RPC data section of the sampler.
Enter the URL of the service as http://webservices.daehosting.com/services/TemperatureConversions.wso (You can capture this from the location attribute of the address element in the WSDL)
Now add a listener to view the result.
Save your test plan. Your test plan will be similar to the below.



Run the test and check the results. You will get the Fahrenheit value of the provided Celsius figure.

Step 3

Now, you can increase the thread count and extend this test into a performance test. However, in that case you will be sending the same request again and again. It will not be a good simulation of a real-world scenario. You should be able to alter the payload (in our example, Celsius value) of the SOAP request with each thread.
In order to do so, you should read Celsius data from a data source. In Jmeter, you can easily read data from a csv file.
Lets create a csv file, temperature.csv and save it in the location where you saved the above JMeter test plan.
Enter a set of values row-by-row in the temperature.csv
eg:-
10
20
30
40

Step 4

Next, we will add CSV Data Set Config element which will read data from the csv file.
Right click on Thread Group and select Add --> Config Element ---> CSV Data Set Config
Now you can configure your CSV data source as follows.

Filename: <Give the full path of temperature.csv>
Variable Names: celcius

Keep the other values intact.

Step 5

Now access the SOAP/XML-RPC Data section of the request and replace the hard-coded celcius figure with the variable name we have configured in CSV Data Set Config element (i.e:- ${celcius})
After parameterizing, your SOAP request will be as follows.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
<soapenv:Header/>
<soapenv:Body>
<tem:CelciusToFahrenheit>
<tem:nCelcius>${celcius}</tem:nCelcius>
</tem:CelciusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>

Increase the thread count corresponding to the row count in your csv file and run the test. You will notice that the Celcius figure will be varied in each request by reading data from CSV data source.

In this way, you can easily do data driven web service testing using Jmeter.

Test Reviews - An efficient way to validate test coverage

When the complexity of testing increases, deriving test scenarios becomes extremely hard. Individual testers are not always capable of thinking all aspects of the products and finding out test cases. In agile processes which demands frequent releases, this becomes much harder since individual testers do not get sufficient time to learn and explore products.

At WSO2, code reviews have been an integral part of engineering process. Code review groups get together and go through different code segments in weekly basis.

Similar to code reviews, WSO2 QA team conducts weekly test reviews. The primary objective of reviewing tests is to finding out gaps in a particular product's testing methodology.

Usually there are 2 main actors take part in a test review meeting.

- Tester(s) who test a particular feature

- Developer (s) who implemented the feature

Tester goes through the existing test scenarios and developer provides his or her feedback on the tests. Specially when a particular use case is not clear enough, tester can clarify it during reviews very effective manner.

It is the testers responsibility to try out the new test scenarios which have been captured during the review in next release cycles.

Test review will not take more than 1 hour and testers should be prepared properly to get the maximum out of this short time and capture more and more test scenarios.

Similar to the manual test scenarios, all automated tests can also be reviewed during test reviews.

In agile processes where you do not have detailed system specs to derive tests, test reviews are one of the most useful mechanisms to validate your test coverage.

How to enable child-first class loading in WSO2 Application Server or Axis2

By default, WSO2 Application Server (or Axis2) uses parent-first class loading mechanism.

If you deploy an AAR service, which can load classes from the following locations.

• CARBON_HOME/lib (CARBON_HOME is the location where you installed WSO2 Application Server)
• CARBON_HOME/repository/deployment/server/axis2services/lib
• AAR service/lib (lib directory under your service archive)

If your service implementation class has a package import for a class, which is available in both CARBON_HOME/repository/deployment/server/axis2services/lib and the lib directory under service archive, the class placed under CARBON_HOME/repository/deployment/server/axis2services/lib will get loaded.

Which means, the parent class always get loaded first.

The class loading sequence is CARBON_HOME/lib ---> CARBON_HOME/repository/deployment/server/axis2services/lib -----> AAR service/lib

Sometimes, we want to load the class from service archive lib first without loading the class which is available either one of above parent locations. In other words, child-first class loading will be required for some instances.

child-first class loading can be enabled simply by adding the following parameter in to services.xml in your service archive.

<parameter name="EnableChildFirstClassLoading">true</parameter>

If you set this parameter in axis2.xml of your WSO2 Application Server (or Axis2) instance, all services will use child-first class loading mechanism.