WSO2 SOA middleware Deployment Tips - 1 - validationQuery to avoid broken DB connections

I thought to put together some best practices, guidelines on deploying WSO2 SOA middleware platform. This post will remind you a well-known best practice used in most of the product deployments.

When you are maintaining DB connections, it is always recommended to use a validationQuery to check the health of the TCP connection of the connections stay in DB connection pool.

Because the connection opening is an expensive and time consuming operation, after a connection is created, it will be kept open for a specific time in the pool. When re-using these connections from the pool, there can be situations that the TCP connection to the DB is interrupted and the connection consumer gets errors such as communication link failures etc..

In order to avoid that, a validationQuery, an SQL statement specific to DBMS type, can be used which runs before using the connection.



In WSO2 middleware platform, you usually use central governance/configuration registry to store SOA metadata and various governance tasks. Governance registry supports multiple DBs as underlying data stores. You establish the connection to DB in CARBON_HOME/repository/conf/registry.xml as follows.

<dbConfig name="wso2registry">

<url>jdbc:mysql://localhost:3306/config_db</url>

<userName>>regadmin</userName>

<password>regadmin</password>

<driverName>com.mysql.jdbc.Driver</driverName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<minIdle>5</minIdle>

</dbConfig>

This is one of the places where you may experience DB connection issues as I explained above. Therefore, it is always a best practice to use a validationQuery.

If your DBMSs is MySQL or MSSQL, then use the following.

<validationQuery>SELECT 1</validationQuery>

In Oracle;

<validationQuery>SELECT 1 FROM DUAL</validationQuery>

In Postgres,

<validationQuery>SELECT version(); </validationQuery>

Similarly, when you make DB connections in WSO2 Data Services Server, make sure to use validationQuery in data source definition section as follows.

<property name="org.wso2.ws.dataservice.validationquery">SELECT 1</property>

Monitoring WSO2 ESB using JConsole

My latest tutorial on how to monitor WSO2 ESB using JConsole is now published on WSO2 Oxygen Tank

How to send JSON messages to web services deployed on WSO2 Application Server

JSON (JavaScript Object Notation) is an open and text-based data exchange format, that provides a standardized data exchange format better suited for Ajax style web applications. You can find more information about JSON from www.json.org

WSO2 SOA middleware platform supports sending and receiving JSON messages.

This post takes you through the steps to deploy a simple web service in WSO2 Application Server and write a client using Axis2 ServiceClient API to invoke the service by sending JSON message.

Pre-requisites:

Download and install WSO2 Application Server 4.0.0 or later

Step 1

We are using a simple web service, which echo's user input as follows.

public class EchoService {

 public OMElement echo(OMElement element) {
           return element;
    }
}

You can download the service archive (EchoService.aar) from here and deploy on WSO2 Application Server.

Step 2

Have a look at CARBON_HOME/repository/conf/axis2.xml (CARBON_HOME is the root directory of WSO2 Application Server). You will notice that the JSON specific message builders and formatters are enabled by default.

<!--JSON Message Formatters-->

<messageFormatter contentType="application/json"

class="org.apache.axis2.json.JSONMessageFormatter"/>

<messageFormatter contentType="application/json/badgerfish"

class="org.apache.axis2.json.JSONBadgerfishMessageFormatter"/>

<messageFormatter contentType="text/javascript"

class="org.apache.axis2.json.JSONMessageFormatter"/>

<!--JSON Message Builders-->

<messageBuilder contentType="application/json"

class="org.apache.axis2.json.JSONOMBuilder"/>

<messageBuilder contentType="application/json/badgerfish"

class="org.apache.axis2.json.JSONBadgerfishOMBuilder"/>

<messageBuilder contentType="text/javascript"

class="org.apache.axis2.json.JSONOMBuilder"/>

WSO2 Application Server accepts any JSON message with the content type application/json, application/json/badgerfish or text/javascript. In this example, we will use a message with the content-type, application/json

Step 3

Now, If you invoke the service using the Tryit utility associated with EchoService and trace the message, you will notice the payload of the SOAP message as follows.

<p:echo xmlns:p="http://service.carbon.wso2.org">

<echo xmlns="http://service.carbon.wso2.org" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">charitha</echo>

</p:echo>

Basically, the message payload will be similar to the following.

<echo>

<value>charitha</value>

</echo>

Step 4

Now, we are going to send the above payload as a JSON message. The JSON format of the above message payload will be as follows.

{"echo":{"value":"charitha"}}

Add the following class to your java project. Compile it using the libraries included in CARBON_HOME/repository/component/plugins directory. You may also need to add CARBON_HOME/lib/endorsed into your class path.

package org.wso2.carbon.service;

import org.apache.commons.httpclient.HttpClient; import org.apache.commons.httpclient.methods.PostMethod; import org.apache.commons.httpclient.methods.RequestEntity; import org.apache.commons.httpclient.methods.StringRequestEntity;

public class JSONClient {

    public static void main(String[] args) throws Exception {
          String strURL = "http://localhost:8281/services/JSONProxy/";                  PostMethod post = new PostMethod(strURL);                RequestEntity entity = new StringRequestEntity("{\"echo\":{\"value\":\"charitha\"}}","application/json", "UTF-8");         post.setRequestEntity(entity);         HttpClient httpclient = new HttpClient();         try {             int result = httpclient.executeMethod(post);             System.out.println("Response status code: " + result);             System.out.println("Response body: ");             System.out.println(post.getResponseBodyAsString());         } finally {             post.releaseConnection();        }    } 
}

You could use the same axis2.xml located at CARBON_HOME/repository/conf as the client axis2.xml when creating ConfigurationContext

If you forward the message to tcpmon and trace it, you will see the request as follows.

POST /services/EchoService HTTP/1.1

Content-Type: application/json; charset=UTF-8

User-Agent: WSO2 WSAS-3.2.0

Host: 127.0.0.1:9764

Transfer-Encoding: chunked

1d

{"echo":{"value":"charitha"}}

0

Data driven web service testing with Apache Jmeter

You will not get enough coverage in your web service testing, if you repeatedly send the same request to the webservice under testing. You must read data from a data source and parameterize each request. If you are dealing with SOAP messages, you should parameterize SOAP message payload.
This post guides you how to do data driven web service testing using Apache Jmeter.

Pre-requisites:
Install Apache Jmeter 2.3.4 or later

Step 1

We are going to invoke a publicly available web service, Temerature Conversion service . The WSDL of this web service can be accessible through http://webservices.daehosting.com/services/TemperatureConversions.wso?WSDL
You can invoke the CelciusToFahrenheit operation of this webservice using SOAPUI and capture the request message. It will be similar to the below.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
<soapenv:Header/>
<soapenv:Body>
<tem:CelciusToFahrenheit>
<tem:nCelcius>37</tem:nCelcius>
</tem:CelciusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>
Step 2

Lets create a new JMeter test plan and add a thread group. Then, add SOAP/XML-RPC sampler into the thread group and copy and paste the above SOAP request into the SOAP/XML-RPC data section of the sampler.
Enter the URL of the service as http://webservices.daehosting.com/services/TemperatureConversions.wso (You can capture this from the location attribute of the address element in the WSDL)
Now add a listener to view the result.
Save your test plan. Your test plan will be similar to the below.



Run the test and check the results. You will get the Fahrenheit value of the provided Celsius figure.

Step 3

Now, you can increase the thread count and extend this test into a performance test. However, in that case you will be sending the same request again and again. It will not be a good simulation of a real-world scenario. You should be able to alter the payload (in our example, Celsius value) of the SOAP request with each thread.
In order to do so, you should read Celsius data from a data source. In Jmeter, you can easily read data from a csv file.
Lets create a csv file, temperature.csv and save it in the location where you saved the above JMeter test plan.
Enter a set of values row-by-row in the temperature.csv
eg:-
10
20
30
40

Step 4

Next, we will add CSV Data Set Config element which will read data from the csv file.
Right click on Thread Group and select Add --> Config Element ---> CSV Data Set Config
Now you can configure your CSV data source as follows.

Filename: <Give the full path of temperature.csv>
Variable Names: celcius

Keep the other values intact.

Step 5

Now access the SOAP/XML-RPC Data section of the request and replace the hard-coded celcius figure with the variable name we have configured in CSV Data Set Config element (i.e:- ${celcius})
After parameterizing, your SOAP request will be as follows.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:tem="http://webservices.daehosting.com/temperature">
<soapenv:Header/>
<soapenv:Body>
<tem:CelciusToFahrenheit>
<tem:nCelcius>${celcius}</tem:nCelcius>
</tem:CelciusToFahrenheit>
</soapenv:Body>
</soapenv:Envelope>

Increase the thread count corresponding to the row count in your csv file and run the test. You will notice that the Celcius figure will be varied in each request by reading data from CSV data source.

In this way, you can easily do data driven web service testing using Jmeter.

Test Reviews - An efficient way to validate test coverage

When the complexity of testing increases, deriving test scenarios becomes extremely hard. Individual testers are not always capable of thinking all aspects of the products and finding out test cases. In agile processes which demands frequent releases, this becomes much harder since individual testers do not get sufficient time to learn and explore products.

At WSO2, code reviews have been an integral part of engineering process. Code review groups get together and go through different code segments in weekly basis.

Similar to code reviews, WSO2 QA team conducts weekly test reviews. The primary objective of reviewing tests is to finding out gaps in a particular product's testing methodology.

Usually there are 2 main actors take part in a test review meeting.

- Tester(s) who test a particular feature

- Developer (s) who implemented the feature

Tester goes through the existing test scenarios and developer provides his or her feedback on the tests. Specially when a particular use case is not clear enough, tester can clarify it during reviews very effective manner.

It is the testers responsibility to try out the new test scenarios which have been captured during the review in next release cycles.

Test review will not take more than 1 hour and testers should be prepared properly to get the maximum out of this short time and capture more and more test scenarios.

Similar to the manual test scenarios, all automated tests can also be reviewed during test reviews.

In agile processes where you do not have detailed system specs to derive tests, test reviews are one of the most useful mechanisms to validate your test coverage.

How to enable child-first class loading in WSO2 Application Server or Axis2

By default, WSO2 Application Server (or Axis2) uses parent-first class loading mechanism.

If you deploy an AAR service, which can load classes from the following locations.

• CARBON_HOME/lib (CARBON_HOME is the location where you installed WSO2 Application Server)
• CARBON_HOME/repository/deployment/server/axis2services/lib
• AAR service/lib (lib directory under your service archive)

If your service implementation class has a package import for a class, which is available in both CARBON_HOME/repository/deployment/server/axis2services/lib and the lib directory under service archive, the class placed under CARBON_HOME/repository/deployment/server/axis2services/lib will get loaded.

Which means, the parent class always get loaded first.

The class loading sequence is CARBON_HOME/lib ---> CARBON_HOME/repository/deployment/server/axis2services/lib -----> AAR service/lib

Sometimes, we want to load the class from service archive lib first without loading the class which is available either one of above parent locations. In other words, child-first class loading will be required for some instances.

child-first class loading can be enabled simply by adding the following parameter in to services.xml in your service archive.

<parameter name="EnableChildFirstClassLoading">true</parameter>

If you set this parameter in axis2.xml of your WSO2 Application Server (or Axis2) instance, all services will use child-first class loading mechanism.

WS-Addressing with SOAPUI

WS-Addressing is a standard way of including message routing data within SOAP headers without relying on transport specific routing properties. If you are talking to a web service which expects WS-Addressing information in request SOAP messages, you could either follow a programmatic approach to write a client and insert WS-A headers manually or use a commercial or free web service client tool. SOAPUI is the simplest tool which can be used to insert WS-Addressing headers in to request SOAP messages.
In this blog post, we will;

• Talk to a WS-Addressing enabled service which is hosted in WSO2 Application Server using SOAPUI

• Assert response SOAP message to validate WS-A headers

Pre-requisites:

1. Download and install the free version of SOAPUI 3.0.1 or later
2. Download and install WSO2 Application Server 4.0

Step 1

We are going to invoke the default HelloService which is shipped with WSO2 Application Server. Go to WSO2_APPSERVER/home/bin and start the server by running wso2server.sh{bat}
Access management console with https://localhost:9443/carbon and log in using the default administrator credentials (username=admin, password=admin)
Navigate to Manage --> Services --> List in the left menu and select "HelloService"
You will be directed to the dashboard of HelloService. Click on Modules link.



You will notice that "addressing-3.1.0" is listed under "Globel Level". By default WS-Addressing is enabled for all the services hosted in WSO2 Application Server. Therefore, you do not have to configure anything at the server side to enable WS-Addressing.

Step 2

Now, we will create a new SOAPUI project referring to http://localhost:9763/services/HelloService?wsdl as follows. Make sure to select "Create Test Suite" check box as well.



After creating the project, select greet request under HelloServiceSoap11Binding TestSuite



If you click on run just by providing input string, you will notice that WS-A headers will not be included in the request SOAP message which can be observed if you toggle "Raw" option.

Step 3

Lets add WS-A headers in to the request SOAP message. Select greet request and click on WS-A option and select the following properties.

Enable WS-A addressing

Add default wsa:Action

Add default wsa:To

Randomly generated message ID




Keep the rest of the default values and click on run. You will notice the following WS-A headers in the request message if you look at the raw message in SOAPUI.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greet</wsa:Action>
<wsa:MessageID>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:MessageID>
<wsa:To>http://localhost:9763/services/HelloService.HelloServiceHttpSoap11Endpoint/</wsa:To>
</soapenv:Header>

Also, the response headers will be similar to the following.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greetResponse</wsa:Action>
<wsa:RelatesTo>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:RelatesTo>
</soapenv:Header>

Simple.. isn't it? we just invoked a service with WS-Addressing headers using SOAPUI.

Here, we manually verified that the response message includes the correct WS-A headers. We looked at the response message and verified each header. If you maintain an automated test suite to verify your web service interactions, the above procedure cannot not be considered as over until you add an assertion so that the test automatically reports the correctness or failures of response/request messages.

Step 4

Now, we will add an assertion to validate response headers. In this example, we are going to add an XPath match assertion to check whether the wsa:Action header of the response equals to urn:greetResponse

Click on Assertions option and select "Add an assertion to this item" icon. Then select, XPAth Match as the assertion.



Next, you should specify the XPath expression and expected result as follows. First, click on Declare button which will automatically define the namespaces. Specify //wsa:Action as the xpath. Also, specify urn:greetResponse as the expected result.



Run the test again. You will notice that the test will be marked in green denoting that the test is successful.