WSO2 Deployment Synchronizer - Sharing deployment artifacts across a product cluster

This post is about a new feature in WSO2 Carbon product platform. I will use the latest versions of WSO2 Governance Registry(WSO2-G-reg-4.1.0) and WSO2 Enterprise Service Bus (ESB-4.0.2) for the demonstration.

Before going through the configuration steps, lets look at the problem which is going to be addressed using Deployment Synchronizer.

Suppose we have a WSO2 ESB product cluster with a single READ-WRITE node and a several READ-ONLY nodes which shares a common configuration registry. When we deploy a proxy service from READ-WRITE node, in order for other nodes to be synced with that new service deployment, all READ-ONLY nodes have to be restarted. This is a painful concern in a large product cluster.

WSO2 Deployment Synchronizer feature has been implemented to resolve that concern. With that, once you deploy a service (or any deployable artifact such as ESB sequence, scheduled task etc..) from one node in a product cluster, the other nodes automatically get the changes and sync up with the master (or READ-WRITE) node.

The Deployment Synchronizer makes use of two different approaches for deployment artifact synchronization.

1. SVN based synchronizer

2. Registry based synchronizer

Lets look at each of these in detail.

Pre-Requisites:

Download WSO2 ESB-4.0.2 and WSO2 Governance Registry-4.1.0 binary distributions from wso2 oxygen tank

Product Clustering Setup

Lets proceed through setting up a two node WSO2 Carbon product cluster as shown above.

Step 1:

Extract the downloaded WSO2ESB-4.0.2.zip and make two copies of it as wso2esb-rw and wso2esb-ro
wso2esb-rw directory is used as the master node of the cluster and wso2esb-ro node will be used as the slave node.

Step 2:

Extract the downloaded WSO2greg-4.1.0.zip into a new directory. This will be used as the central governance and configuration registry in our ESB cluster.

Step 3:
The above WSO2 G-reg instance will run on a mysql DB instead of the default H2 database. Therefore, lets create a mysql DB first.
Open a mysql prompt in your server. Type the following commands to create a database and assign user privileges.

mysql>create database reg_db;
mysql>use reg_db;
mysql>grant all on reg_db.* TO regadmin@localhost identified by "regadmin";

Edit the CARBON_HOME/repository/conf/registry.xml of the WSO2 G-reg server as follows.

<currentDBConfig>mysql-reg</currentDBConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>

<dbConfig name="mysql-reg">
<url>jdbc:mysql://localhost:3306/reg_db</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>5</maxActive>
<maxWait>60000</maxWait>
<minIdle>50</minIdle>
<validationQuery>SELECT 1</validationQuery></dbConfig>

Now, copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2Greg server and start the server with -Dsetup switch.

sh wso2server.sh -Dsetup

This will start WSO2-Greg server on mysql.

Step 4

Now, we have started central governance and configuration registry instance of our WSO2 ESB product cluster. Now, lets proceed with configuring WSO2 ESB nodes.
Lets configure read-write node first.

We are going to run 3 carbon servers in the same machine. Therefore, we need to change the port index in CARBON_HOME/repository/conf/carbon.xml so that each of the WSO2 ESB nodes will run on their own ports without conflicting with each other.
In carbon.xml, change the following element in order to run the ESB read-write node in HTTP port 9764 and HTTPS port 9444.


<Offset>1</Offset>

We are going to store the configuration data of ESB nodes in the cluster in /_system/esbnodes space of the above registry. Also, the governance data will be stored in /_system/governance directory. Therefore, lets add the following registry mounts through CARBON_HOME/repository/conf/registry.xml.

<dbConfig name="mysql-reg">
<url>jdbc:mysql://localhost:3306/reg_db</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>5</maxActive>
<maxWait>60000</maxWait>
<minIdle>50</minIdle>
<validationQuery>SELECT 1</validationQuery></dbConfig>

<remoteInstance url="https://localhost:9443/registry">
<id>conf-gov-registry</id>
<dbConfig>mysql-reg</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>

<!-- Governance data will be stored in /_system/governance collection of central registry instance --> 
<mount overwrite="true" path="/_system/governance">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

<!-- Configuration data will be stored in /_system/esbnodes collection of central registry instance -->  
<mount overwrite="true" path="/_system/config">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>

Copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2 ESB read-write node and start the server.

sh wso2server.sh

Step 5

We have configured and started the READ-WRITE node of ESB cluster. Now, we can configure the READ-ONLY node. The configuration is almost same as READ-WRITE node except the highlighted elements given below.

First, change the port offset to 2 so that the ports will not be conflicted with the other server ports.

<Offset>2</Offset>

Change the default NIO HTTP and HTTPS ports in CARBON_HOME/repository/conf/axis2.xml as follows.

<transportReceiver class="org.apache.synapse.transport.nhttp.HttpCoreNIOListener" name="http">
<parameter locked="false" name="port">8281</parameter>

<transportReceiver class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener" name="https">
<parameter locked="false" name="port">8244</parameter>

Add the following registry mounts through CARBON_HOME/repository/conf/registry.xml of ESB READ-ONLY node.

<dbConfig name="mysql-reg">
<url>jdbc:mysql://localhost:3306/reg_db</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>5</maxActive>
<maxWait>60000</maxWait>
<minIdle>50</minIdle>
<validationQuery>SELECT 1</validationQuery></dbConfig>

<remoteInstance url="https://localhost:9443/registry">
<id>conf-gov-registry</id>
<dbConfig>mysql-reg</dbConfig>
<readOnly>true</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>

<!-- Governance data will be stored in /_system/governance collection of central registry instance --> 
<mount overwrite="true" path="/_system/governance">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

<!-- Configuration data will be stored in /_system/esbnodes collection of central registry instance -->  
<mount overwrite="true" path="/_system/config">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>

Copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2 ESB read-only node and start the server.

sh wso2server.sh

Now we are done with the clustering setup but we have not done any configurations related to the deployment synchronizer yet.
We will look in to the registry based deployment synchronization first.

Step 6 - Registry based deployment synchronizer

In this mode, once you deploy an artifact from the READ-WRITE node, the artifacts will be stored in the relevant collection in the configuration registry. The other nodes of the cluster will use the registry checkin-checkout client and checkout the deployment artifacts to their file system. Then with the hot deployment functionality, the artifacts will get deployed in the cluster nodes.

Lets look at how we can use the registry based deployment synchronizer.

• Log in to management console of ESB READ-WRITE node (https://localhost:9444/carbon)
• Navigate to Configure --> Deployment Synchronizer UI

• Select Auto Commit option and click on Enable
• Log in to management console of ESB READ-ONLY node (https://localhost:5/carbon)
• Access Configure --> Deployment Synchronizer UI
• Select Auto Checkout option and click on Enable
  

• Create a proxy service (eg:-proxy1) in READ-WRITE node
• After 60 seconds (the default synchronization period), log in to the management console of the READ-ONLY node of the cluster.
• You will notice that the proxy1 is listed in the services list of READ-ONLY node
  

Step 7 - SVN based deployment synchronizer

Deployment synchronization can be achieved using a SVN repository as well. Lets look at SVN based deployment synchronizer.

In this mode, instead of a registry, we use a subversion repository as the deployment artifact store. As we check-in files to SVN, the synchronizer use a SVN client API and commit and update deployment artifacts periodically by using a SVN location.

• First, revert the registry based deployment synchronizer settings which we did above. (Disable deployment synchronizer in Configure --> Deployment Synchronizer UI)

• Have a proper SVN location. You can use an existing SVN location or create a new one.
• Add the following configuration in CARBON_HOME/repository/conf/carbon.xml of both ESB nodes. Make sure to specify correct SVN credentials and location URL according to your environment.
  

<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>true</AutoCommit>
<AutoCheckout>true</AutoCheckout>
<RepositoryType>svn</RepositoryType>
<SvnUrl>https://svn.wso2.com/wso2/custom/projects/projects/qa/deployment-synchronizer/esb</SvnUrl>
<SvnUser>qasvn</SvnUser>
<SvnPassword>test</SvnPassword>
<SvnUrlAppendTenantId>false</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

• Restart both ESB nodes.
  
• Deploy a proxy service from either one of the nodes
• Changes will get reflected into other nodes after 60 seconds (default synchronization period)
  

We looked at two different ways of sharing deployment artifacts among cluster nodes. If you come across any issues when configuring either one of the above approaches, please drop me a mail.

How to pass system properties when WSO2 Carbon server is running in daemon mode

When you start wso2 carbon server using a startup script such as wso2server.sh, you can simply pass system properties such as -DosgiConsole, just by passing system property in command line.

e.g:- sh wso2server.sh -DosgiConsole

However, if you start the server as a System process (daemon), how should you send those parameters?

Lets look at how we can start OsgiConsole, if we start server in daemon mode.

1. Open CARBON_HOME/repository/conf/wrapper.conf

2. Add the following parameter under Java additional properties section
wrapper.java.additional.11=-DosgiConsole=1234

3. Start the server as "sh wso2server.sh -start"

4. Open a new shell and "telnet localhost 1234"

ApacheCon NA 2011 - a week full of Apache community gathering

ApacheCon is the official event of Apache Software Foundation which brings together the global Apache community in a week full of trainings, live demos, hands-on sessions and various other meet ups.

The event will be commenced tomorrow, 7th of November 2011 in the Westin Bayshore Vancouver Canada.

I'm witnessing a lot of technology enthusiasts from diverse parts of the world are checking-in at Westin Bayshore hotel at the moment predicting a successful function.

With the tightly-coupled relationship of WSO2 and various Apache projects, WSO2 is playing a key role in ApacheCon 2011 as well. WSO2 is presenting 3 talks and is an exhibitor Sponsor too.

Afkham Azeez speaks about building scalable multi-tenant application server on cloud using Tomcat, Axis2 and Synapse on Wednesday, November 9th.

He is also talking about an architecture for enabling multi-tenancy for Apache Axis2 on Thursday, November 10th.

Prabath Siriwardhana will also do a half-day training on web services security on Monday, November 7th.

Come and meet us at WSO2 booth at ApacheCon. You will find how WSO2 built world's first free and open source PaaS and renowned SOA platform using various Apache project components and how we adhered to the Apache process in project releases.

Why is testing taking so long?

When you are involved in software testing, you may have heard the following 3 common questions at the end, beginning or middle of any testing cycle.

1. Why did not you find that bug during testing cycle?

2. Why is testing taking so long?

3. Why did not you automate the damn thing?

I have answers for all three questions but Michael Boloton, the testing genius in our era, explained the answer for question 2 with some great examples. I would recommend this to be a must read for anyone work in software engineering.

[1]http://www.developsense.com/blog/2009/11/why-is-testing-taking-so-long-part-1/

[2]http://www.developsense.com/blog/2009/11/what-does-testing-take-so-long-part-2/

Study the facts given by Michael. You will realize the truth of test estimation and traditional way of managing test processes.

Was I able to do that?

For the last 2-3 months, the readers of my blog may have seen a small icon at the top of the pages giving you a hint that I'm presenting at WSO2Con 2011, the technology conference of the year. A few months back, Samisa, the engineering head of WSO2, forced me to submit a talk for this conference. Hesitating, I prepared an abstract of a talk about SOA and quality and sent my proposal. After a few weeks, I was informed that my proposal had been accepted and that I would speak at the third day of the conference. First, I was shocked. Why? I always thought negatively about presentations and public speaking. As my batch mates at university of Colombo knows, I myself and a lot of others had a bad experience w.r.t presentations during campus era :). Worst, I had not given a public speech for a time that I remember. In summary, it had been kind of a nightmare to me.

I did not have any other option, but do this speech at an level that I can watch it later without cursing me at least :) Therefore, I started preparation. The subject and topic is not a surprise for me. It is the subject that I live in and spent 8 years of my career, testing and QA. SOA is also not new to me, I have been working with SOA for 5 years out of 8 years. Therefore, I knew that I have a LOT to talk. But just talking non-sense was not required. SOA testing is a relatively new topic and there is not universally agreed methodology for testing service oriented solutions. I thought to put together my experience into a methodology and present.

I borrowed SOA design, concepts and principles book written by Thomas Erl from WSO2 library and read it completely. It was a great reference and I was able to clarify a lot of concerns that I had about some SOA concepts.

Then, I started to prepare slides. When the days of WSO2con were coming closer, the nervousness and stress going up straightly. One day I met with my close friend and the presentation and public speaking master blaster at WSO2, Prabath Siriwardhana. He nicely explained some of the tips that he followed during presentations and he mentioned that practicing a speak is the best way to do it nicely. So, I practiced. I practiced a lot everyday. I learned a lot. I explored everything related to SOA testing. I my self came into a position that I was able to answer very complicated questions related to SOA testing. The confidence started to build. BUT, nervousness and negative mind set still did not disappear.

As usual, I'm not a closed person. Hence, I openly talked with colleagues at WSO2 and my friends about my nervousness and possible failure cases. Fortunately, I live in a world where 90% of my friends, WSO2 colleagues are truely friends. They listened to me and helped me a lot. The senior wso2 folks such as Afkham Azeez, Selvarathnam Uthaiyashankar, Samisa, Prabath, reviewed my slide deck and helped me to correct some errors. Then, most of the folks listened to my boring rehearsal sessions at office premises :) Iranga, one of my closest friends, visited me all the way from his office and went through the slides and speech. His input was also invaluable because of his experience on QA.

Finally, on 15th of Thursday, September 2011, I delivered my first public speech at WSO2Con in front of a large gathering including a lot of folks from various companies all over the world.

Quality, key to successful SOA

View more presentations from Charitha Kankanamge

As the presenter, I'm not sure whether it is a success or failure, but everyone came into me and congratulated that it was a nice talk. I still cannot believe! I'm very lucky to see the tweets as below as the response to my speech!

Must attend talk in WSO2Con2011 - Engineering to take over the world by Samisa Abeysinghe

WSO2Con 2011, a week full of tutorials, tech talks, networking events will be commenced shortly at WatersEdge, Colombo, Sri Lanka. If you are attending the conference, I'm sure you will look for a session which gives you an overall picture about how WSO2 has become a leading technology provider and what are the secret ingredients of the process that has been followed.

There is no other individual than Samisa Abeysinghe, VP of Engineering at WSO2, who knows top to bottom of WSO2 engineering process as well as lead the engineering team for the last five years to become where we are now. If you are bored with listening to traditional process sermons which are usually heard in conferences, this will be a totally different session for you. We are at WSO2, while redefining the middleware space, reinventing the software development process too. We have our own way of designing, developing and testing products. Wanna know more about how we do that? then do not forget to attend the session by Samisa which will be held on 15th of September 2011 at 15.30 PM!

Must attend talk in WSO2Con2011 - Security in practice by Prabath Siriwardhana

Time flew in light speed and finally the days have arrived! WSO2Con 2011, a conference which you should not miss. Though you are a seasoned IT professional or a student keen in learning latest technologies, it is the event of the year.

Out of more than 25 sessions which are planned to be held, my attention goes to one particular session first. It is "Security in Practice" by Prabath Siriwardhana which will be held on 14th of September, 2011 at 3.30 PM. Why do I like it than any other session?

Prabath is a brilliant speaker. I like the way he presenting. He has his own style which attract anyone in the audience.

Prabath is the expert of the most complex and hard-to-understand aspect of Service Oriented Architecture, Security.

You can imagine the complexity of delivering a presentation in such a tough, complicated subject matter

But, Prabath has the ability to inject complex subject matters into your head in very nice manner. You should not miss this great session of Prabath if you are attending WSO2Con 2011!