Friday, June 20, 2008

WS-Reliable messaging with WSO2 WSAS -1

Web services are an essential component of distributed computing. Reliable communication between services and service consumers should be taken in to account when designing SOA based systems.
WS-Reliability is a SOAP-based specification that fulfills reliable messaging requirements critical to web service applications.
The objective of this post is not to discuss the features and architectural details of WS-Reliable Messaging (WS-RM). You can find the WS-RM specification from here and a lot of useful materials are available on the web. I'm going to demonstrate the simplest RM scenario, one-way single channel invocation using WSO2 mercury, which is an implementation of WS-RM using Apache Axis2.
WSO2 mercury is shipped as a module with WSO2 web services application server version 2.3 (WSO2 WSAS-2.3) as well as a separate piece of product. I'm going to describe the scenario using WSO2 WSAS, however you may also follow the given steps with Axis2-1.4 .

We are going to send a set of ping messages (one-way messages) to a web service in reliable manner. In other words, we send a set of requests to a service and block the channel at the middle of message transmission. Then we restart the transport channel. If WS-RM is not enabled, the message transmission will be lost when the transport channel goes down. You have to re-send messages. However, if our service and client are configured to use Wso2 mercury (WS-RM implementation), it will take care of guaranteed delivery of the rest of the messages when the transport channel is back again.

Lets see how reliable messaging can be used with WSO2 WSAS and Mercury.

Pre-requisites:
Install WSO2 WSAS-2.3

Step 1 - Service configuration

Create a simple web service implementation class as follows.

package org.wso2.wsas.service;
public class MercuryService{

//Oneway messaging
public void Ping(String s){
System.out.println("*****Received the ping request*****");
}
}

Compile and save the class.

Next, create a service descriptor (services.xml) as given below and save it in a META-INF directory.

<service name="MercuryService">
<Description>
Mercury test service
</Description>
<messageReceivers>
<messageReceiver mep="http://www.w3.org/2006/01/wsdl/in-only"
class="org.apache.axis2.rpc.receivers.RPCInOnlyMessageReceiver"/>
<messageReceiver mep="http://www.w3.org/2006/01/wsdl/in-out"
class="org.apache.axis2.rpc.receivers.RPCMessageReceiver"/>
</messageReceivers>
<module ref="Mercury" />
<parameter name="ServiceClass" locked="false">
org.wso2.wsas.service.MercuryService
</parameter>
</service>

Note that <module> element is used to engage the Mercury module, which provides reliable messaging at the server side.

Now, create a deployable Axis2 service archive (*.aar) using the above service implementation class and META-INF/services.xml file. (Please have a look at Axis2 user guide if you are not familiar with service archive structure)

Copy the created service archive (Lets say it is "MercuryService.aar") to WSAS_HOME/repository/services directory. It will be deployed on WSAS. Start WSO2 WSAS using wso2wsas.bat{sh} and login in to admin console (https://localhost:9443)

Go to the services and service group management page and you should see the "MercuryService" there. Click on the service name. You will be directed to MercuryService's service management page. Click on "Manage Module Engagements" link. You will notice that the mercury module is listed under "Modules engaged at service level".



That's all for configuring RM at the web service level. Let's take a look at the client side configuration.

Step 2 - Generate stubs

First, we need to generate client side stubs through WSO2 WSAS management console. In the MercuryService's service management page, select "Generate Client". "Stub Generation" page will be displayed. Click on "Generate" with the code gen options as specified in the following image.

Save the generated jar file in your file system.

Step 3 - Write Client

Now write the client to invoke service in reliable manner. Make sure to add jars in WSAS_HOME/lib and the generated client stub jar in the previous step to your class path in order to compile the client.

package org.wso2.wsas.client;

public class OnewayAnnonSOAP11Client{

public static void main(String args[])throws AxisFault{

ConfigurationContext cc = ConfigurationContextFactory.createConfigurationContextFromFileSystem
("C:\\RM\\client-repo",C:\\RM\\client-repo\\conf\\axis2.xml");

MercuryServiceStub stub = new MercuryServiceStub(cc, "http://localhost:9080/services/MercuryService");
MercuryServiceStub.Ping request = new MercuryServiceStub.Ping();
request.setS("ping");

stub._getServiceClient().engageModule("Mercury");

for (int i = 0; i < 100; i++) {
try {
stub.Ping(request);
} catch (RemoteException e) {
e.printStackTrace();
} try
{ Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
// Setting the last message stub._getServiceClient().getOptions().setProperty("MercuryLastMessage", Constants.VALUE_TRUE);
try { stub.Ping(request);
} catch (RemoteException e) {
e.printStackTrace(); }
try { Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace(); } } }

In order to get the reliable messaging support at the client, we need to engage Mercury module with "stub._getServiceClient().engageModule("Mercury")". We need to have ConfigurationContext to define the client repository where the Mercury.mar and axis2 configuration file (axis2.xml) is placed. Create a directory, client-repo and copy WSAS_HOME/repository/modules/mercury.mar to client-repo/modules directory. Also, create a sub-directory conf under client-repo and copy an axis2.xml file ( I have used axis2.xml copied from Axis2-1.4) in to that folder. You can call ping method of your webservice inside a loop so that multiple requests will be submitted. stub.Ping(request); In reliable messaging, a Sequence is created using a CreateSequence interaction, and terminated when finished with a TerminateSequence interaction. Therefore, a CreateSequence message is generated by client as the first step. If the CreateSequenceResponse message is received, then the actual application message delivery is started. Each message in a sequence has a message number, which starts at one and increments by one for each message. These message numbers are used to acknowledge the messages. In order to complete the sequence, the last message should be stated explicitly with stub._getServiceClient().getOptions().setProperty("MercuryLastMessage", Constants.VALUE_TRUE); Lets try to understand this scenario by sending messages via tcpmon. Start tcpmon and configure listen port as 9080 and target port as 9762.


Step 4 - Run client and monitor with TCPmon

Now run the client and check the messages delivered through tcpmon. You will notice that the first message is "CreateSequence". You will see that <wsrm:MessageNumber>1</wsrm:MessageNumber> soap header element in the next message where the actual ping request payload is transmitted.

Finally, check whether the messages are delivered reliably when your transport medium is interrupted. While the request soap messages are being sent, stop the incoming transport channel through tcpmon. You may do so by just stopping the port 9080. Wait a few seconds. Restart port 9080. You should see that the message transmission is resumed.

Thats all for now! will post more on WSO2 WSAS/Axis2 and RM soon.

Thursday, June 12, 2008

Isolated test estimations do very little in agile world

The traditional test estimation techniques do not seem to work well in agile systems. In general, most test estimate models are based on development estimates and often you don't get sufficient time for complete test coverage. There will always be a battle between devs. and testers in scheduling and effort estimation.

Think about a simple scenario. Manager of project A asks a developer how many days he needs to implement a particular module. Suppose developer requests 3 weeks. Then
the manager asks how long will it take to do testing of that particular module. Test lead may request 2 weeks based on his estimates.
The immediate answer will be "No, do it in 1 week, you can get help from a few more testers working on project B"

Suppose the test estimate has done using valid estimation mechanism and enough time is allocated for executing all test cases.
Will the testing be completed within 2 weeks (or 1 week with more testers) as scheduled?
According to my experience, It depends.
If your programmers write unit tests and do code reviews, the test schedule will not be deviated much from the original estimates.
However, if the above disciplines are not followed, the estimations will no longer valid. There will be hell a lot of bugs when QA team start testing the code. Blockers may prevent exercising major components and therefore product Schedule slippages are ineviatable.

I think there shouldn't be two different estimates for development and testing. Both should come together in a common project estimate to achieve on time delivery.
The estimates to complete a module should be prepared, which includes both dev and testing effort.
In the above example, developer and tester can work together to come up with a combined estimation which captures code quality and testing implications. Both parties should monitor the progress and adjust the estimates accordingly. If unit tests and code reviews are not happened as scheduled, project leads should agree to allocate more time for testing.

Keep in mind. I'm expressing my views according to the Agile environments. Not the outsourced test models which requires to have separate QA/test estimates to be signed off by clients before start testing cycles.

Tuesday, June 10, 2008

WSO2 WSAS -2.3 is out!

WSO2 Web services application server (WSAS) - 2.3 was released yesterday. It was an extremely tough release. We were given the June 10th deadline less than 2 weeks back (on 27th of May). There were major changes happened in core Axis2. We were supposed to ship with Axis2-1.4 which had not been gone through a rigorous QA/test cycle to verify compatibility with WSO2 WSAS. It was one of the most challenging test/QA cycles I have ever gone through. It is obvious that less than 2 weeks are not enough to do a full scale functional/regression test, no option, we had to mange somehow.

Milestone builds were planned to out for QA testing on every friday though it did not happen. There were a lot of regression issues and most of them were uncovered during the initial QA tests. Fixes were getting delayed and some blockers avoided exercising some major features until last moment. Worked around the clock to complete functional as well as regression testing.
During the last few days 2-3 release candidates were out in a day. :) We had to make sure the regressions are minimum in between RCs. Interoperability with other SOA frameworks such as .NET was one of the major requirements expected from WSAS-2.3 release. Interop test suites were already in place when we started test cycles. However, most of java test suites were out dated, no build scripts or failed in execution. Therefore we had to update some of them/write from scratch.
Automated ant based functional test-framework helped to do quick smoke tests. However, as I expected and believed, good/critical defects were always uncovered during manual tests.

WSAS-2.3 release can be considered as a true TEAM effort. Everyone worked hard to get the things done within the given extremely tough deadline. I'm confident that WSAS-2.3 will also be demonstrated the better product quality as in previous releases though everything done within short time.

Finally our effort and hard work paid off as the interop demo of WSAS was very successful at the Microsofts' TECH ED 2008.

One thing I should mention here is, if the deadline is fixed (if it is not negotiable) and the schedule is very aggressive with limited time allocated for testing, you have to forget some process bullshit and work hard to find more and more BUGS. Yes, Risks are apparent, however you have to set the expectations with your leads making sure everyone understands the constraints/issues clearly.

WSO2 WSAS-2.3 release can be downloaded from http://wso2.org/projects/wsas/java

From the WSO2 WSAS 2.3 - Release Note - 10th June 2008
======================================================

WSO2 WSAS is an enterprise ready Web services engine powered by Apache
Axis2 which offers a complete middleware solution. It is a lightweight,
high performing platform for Service Oriented Architectures, enabling
business logic and applications.
Bringing together a number of Apache Web services projects, WSO2 WSAS
provides a secure, transactional and reliable runtime for deploying and
managing Web services.

Key Features
------------
* Data services support - Expose you enterprise data as a services in a
jiffy
* WSAS IDE - Eclipse IDE integration
* Clustering support for High Availability & High Scalability
* Full support for WS-Security, WS-Trust, WS-Policy and WS-Secure
Conversation and XKMS
* EJB service provider support - Expose your EJBs as services
* Axis1 backward compatibility - Deploy Axis1 services on WSAS & Engage
advanced WS-* protocols in front of legacy services
* JMX & Web interface based monitoring and management
* WS-* & REST support
* GUI, command line & IDE based tools for Web service development


New Features In This Release
----------------------------
* Improved interoperability
* Improved Data Services support
* Various bug fixes to Apache Axis2, Apache Rampart & WSAS
* WSO2 Mercury Integration - A new WS-RM Implementation

Data Services - Bringing Enterprise Data to Web
-----------------------------------------------
* Service enable data locked in relational databases, CSV & Excel files
in no time
* Zero code. Simple descriptor file describes the data to service mapping
* Controlled access to your data
* Customizable XML output
* Benefit from REST & WS-* support
* Built-in Connection pooling support
* Supports exposing Stored procedures & functions
* Built-in caching
* Throttling - to ensure your database is never overloaded.
* Easy configuration via graphical console
* Test your services via Try-it tool

Training
--------

WSO2 Inc. offers a variety of professional Training Programs, including
training on general Web services as well as WSO2 WSAS, Apache Axis2,
Data Services
and a number of other products.

For additional support information please refer to
http://wso2.com/training/course-catalog/


Support
-------

WSO2 Inc. offers a variety of development and production support
programs, ranging from Web-based support up through normal business
hours, to premium 24x7 phone support.

For additional support information please refer to http://wso2.com/support/

For more information on WSO2 WSAS, visit the WSO2 Oxygen Tank
(http://wso2.org)

How to do various things with WSAS - WSAS HOWTO
Series(http://wso2.org/library/2707)

For further information see the full release note
http://wso2.org/project/wsas/java/2.3/docs/release_notes.html

Monday, June 2, 2008

Paul Fremantle (WSO2 co-founder/CTO) is among world's best CTOs

Today Inforworld has published the list of top leaders for technology and management innovation - The best CTOs of 2008

It is with a great pleasure to inform you that Paul Fremantle is being recognized as one of 2008 InfoWorld Top 25 CTOs!

Congratulations Paul!!!!

Saturday, May 31, 2008

Exposing an Axis1 service through WSO2 WSAS

Do you want to make use of the powerful QOS features of Axis2 with your legacy Axis1 applications? Yes it is possible with WSO2 Web services application server. It allows easy deployment of any Apache Axis1-based Web service and engage advanced WS-* services, such as WS-RM and WS-Policy in front of legacy Axis1 services.
Lets see how we can use this cool feature.

Step 1

Write the service impl class.

package org.wso2.wsas.service;

public class Calculator{

public int addition(int x, int y){
return x+y;
}

public int multiplication(int a, int b){
return a*b;
}

public int subtraction(int c, int d){
return c-d;
}

public double division(double m, double n){
return m/n;
}
}


Compile and make a jar with the above class. Suppose it is Axis1resources.jar

Step 2

Write the deployment descriptor (*.wsdd) for your service impl class and save it as calculator.wsdd

<deployment xmlns="http://xml.apache.org/axis/wsdd/"
xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
<service name="Axis1calculator" style="wrapped">
<parameter name="className" value="org.wso2.wsas.service.Calculator"/>
<parameter name="allowedMethods" value="*"/>
</service>
</deployment>

Step 3

Start WSO2 WSAS (If you haven't downloaded it yet, get it from http://wso2.org/projects/wsas/java

Access WSO2 WSAS management console using https://localhost:9443 (Default admin user credentials are admin/admin)

Go to Services and service group management page and click on "Upload Axis1 service" link. You will be directed to the following page



Browse for the above calculator.wsdd and Axis1resources.jar and click on Upload. Axis1 service will be deployed on WSAS and you will get a confirmation message.

Step 4

If the deployment is successful, you will see the Axis1 service is shown in the services and service group management page.
Click on "Axis1calculator" service.
In the "Service Management" page, select "Tryit"
Invoke service using some inputs and see the output.

Sunday, May 25, 2008

HTTP Compression

I wasn't much aware of HTTP compression though I heard the term often. Thanks to a nice article written by Martin Brown, I was able to get a basic understanding as well as how to get Apache and IIS servers configured to use HTTP compression.
HTTP compression is an mechanism to utilize the available bandwidth effectively. HTTP protocol data is compressed before it is sent from the server. No specific configuration is required at the client side and server configuration is also relatively simple.
If you are involved in web performance testing, It would be worthwhile to get some knowledge on HTTP compression.

Saturday, May 10, 2008

Weblogic application classloading

We wanted to use our own stax parser implementation in a web application deployed on Weblogic 9.2. The user guides and BEA forums suggested two approaches.

1. Using weblogic.xml web application deployment descriptor with <prefer-web-inf-classes> element. Setting this to true allows a Web application to use our own version of a third-party class

2. Using filtering classloader
Here we are allowed to explicitly specify packages which are required to be loaded from the application. These packages can be listed in weblogic-application.xml

I tried both approaches separately in order to use wstx-asl-3.2.1.jar shipped with our application instead of loading weblogic stax parser.

Unfortunately, both class loading mechanisms failed to pick up our stax parser :(

We even tried to hack weblogic.jar by editing property files which tightly map the parser interface to the impl*. We extracted weblogic.jar and edited the stax impl class entries (i.e:- com.ctc.wstx.sax.WstxSAXParserFactory) of the following 4 files included in META-INF/services directory.
javax.xml.parsers.SAXParserFactory
javax.xml.stream.XMLEventFactory
javax.xml.stream.XMLInputFactory
javax.xml.stream.XMLOutputFactory

However, weblogic reported version conflicting issue when restarting server with the edited weblogic.jar.
I did not encounter these class loading problems with JBoss or tomcat. I'm uncertain about why it is such a pain in configuring a commercial product like Weblogic to use application specific packages.