Invoking WSO2 Carbon admin services with soapUI

The management aspects of WSO2 Carbon platform are primarily achieved through SOAP web services interface known as admin services. All Carbon products ship with a management console (front-end user interface) which communicates with these web services and provides users with various administration capabilities.

In some situations, we need to by-pass the management UI and call the backend web services directly. Specially, in test automation, it is important to minimize the risk of frequent UI changes hence focusing on admin service interactions can be considered as a viable solution. WSO2 test automation framework is built upon this approach which programatically calls the backend web services to manage deployment, configuration and various other tasks.

These backend web services are secured to prevent anonymous invocations. WSO2 Carbon server secures these services through multiple methodologies. For example;

• HTTP Basic Authentication over SSL

• WS-Security username token

• Session based authentication

You can use any SOAP client and communicates with the admin services by authenticating through above security protocols.

In this post, I will take  you through consuming an admin service using soapUI since soapUI is the most user-friendly service testing tool out there to test SOAP or RESTful web services.

We will use HTTP basic auth authentication mechanism out of the auth options described above. If you like to use a different approach such as carbon session based authentication, you may refer to Nandika's blog post.

Pre-requisite:
soapUI 4.5.1 or later
WSO2 Carbon 4.X version (You can use any member product of WSO2 Carbon family)

Step 1

By default, the WSDLs of admin web services are hidden from consumers. Therefore, we need to enable them first to import admin service WSDL into a soapUI project.

Open CARBON_HOME/repository/conf/carbon.xml and set HideAdminServiceWSDLs property to false.

<HideAdminServiceWSDLs>false</HideAdminServiceWSDLs> 

Start WSO2 Carbon server.

Step 2

 

There are large number of admin services which serve many administration and management functionalities such as ServiceAdmin, StatisticsAdmin, ProxyAdmin etc.. However, I will use a simplest admin service, UserAdmin for this example.

Open  https://localhost:9443/services/UserAdmin?wsdl in your browser and check whether it is accessible.

Step 3

Create a SOAP web service project in soapUI using the above WSDL

Step 4

We are going to invoke one of the web services operations exposed through UserAdmin web service. Select listAllUsers operation under UserAdminSoap11Binding interface and click on SOAP request.

Step 5

We are going to submit listAllUsers SOAP request using HTTP basic authentication headers. Therefore, click on Aut tab under the bottom of the request editor and specify the admin  user name and password of carbon server.

Enter the following values for the required parameters of the SOAP request.

<xsd:filter>*</xsd:filter>
 <xsd:limit>100</xsd:limit>

Step 6

Now, we can submit the request to UserAdmin web service.You will get a SOAP response with all users available in the user store of Carbon server.

How to use SecureVault when WSO2 Carbon servers are started as background processes

SecureVault can be used to encrypt the plain-text passwords specified in various configurations files in WSO2 Carbon products. You can find more information about how to secure plain-text passwords using securevault in this blog written by Asela.

When you use SecureVault to encrypt the passwords as explained there, you are supposed to specify the primary keystore password at the server startup. However, this is not possible when you start the server as a background process.

This post summarizes the complete procedure of securing plain text passwords using secure vault and additional configurations when you start the server as a background process.

Suppose, we need to encrypt the LDAP ConnectionPassword value in CARBON_HOME/repository/conf/user-mgt.xml

1. Locate cipher-text.properties which can be found at CARBON_HOME/repository/conf/security directory
2.  Keep a back up of the cipher-text.properties file
3.  Now, remove all key-value pairs which have there by default in cipher-text.properties file.(In this example, we just need to encrypt ConnectionPassword value)
4. Add the following line. Make sure to include your plain_text LDAP connection password in [plain_text_ldap_password]
   
   UserStoreManager.Property.ConnectionPassword=[plain_text_ldap_password]
5.  Locate ciphertool.sh script which can be found at CARBON_HOME/bin directory
6.  Run ciphertool.sh as follows
     ciphertool.sh -Dconfigure
   
   This will prompt "[Please Enter Primary KeyStore Password of Carbon Server : ]" message. Enter "wso2carbon" as the primary keystore password
7.  If the script execution completed successfully, you will see the following message.
   "Secret Configurations are written to the property file successfully"
8.  Now, go back and look at the cipher-text.properties file. The plain text LDAP password will be replaced by a cipher value.
9.  You will also look at CARBON_HOME/repository/conf/user-mgt.xml where we have specified the connection password for LDAP user.
   You will notice that it will be modified by the ciphertool script as follows.
   <Property name="ConnectionPassword" svns:secretAlias="UserStoreManager.Property.ConnectionPassword">password</Property>
10. Now, you can start the server.
    e.g:- sh wso2server.sh
11. This will prompt "[Enter KeyStore and Private Key Password :]" at the server startup because we need to decrypt the encrypted passwords to connect to LDAP.
    You can enter "wso2carbon" and the server will be started successfully.
    
    But you will not be able to provide this password value if you start WSO2 Carbon server as a background process.
    i.e:- ./wso2server.sh start
    In that case, you can follow a simple set of additional steps as explained below
12. Have a file named "password-tmp" in CARBON_HOME/ directory. Add "wso2carbon" (the primary keystore password) to this file and save
13. Now, start the server as a background process.
    ./wso2server.sh start
14. Keystore password will be picked up from password-tmp file. Once the server is started, this fill will automatically be deleted from the file system. Make sure to add this temporary file back whenever you start the sever as a background process.

          NOTE : If you make the name of the password file as "password-persist" instead of "password-tmp" then the fie will not be deleted after reading. Then you don't need to provide the password in subsequent startups.

        

WSO2 Deployment Synchronizer - Sharing deployment artifacts across a product cluster

This post is about a new feature in WSO2 Carbon product platform. I will use the latest versions of WSO2 Governance Registry(WSO2-G-reg-4.1.0) and WSO2 Enterprise Service Bus (ESB-4.0.2) for the demonstration.

Before going through the configuration steps, lets look at the problem which is going to be addressed using Deployment Synchronizer.

Suppose we have a WSO2 ESB product cluster with a single READ-WRITE node and a several READ-ONLY nodes which shares a common configuration registry. When we deploy a proxy service from READ-WRITE node, in order for other nodes to be synced with that new service deployment, all READ-ONLY nodes have to be restarted. This is a painful concern in a large product cluster.

WSO2 Deployment Synchronizer feature has been implemented to resolve that concern. With that, once you deploy a service (or any deployable artifact such as ESB sequence, scheduled task etc..) from one node in a product cluster, the other nodes automatically get the changes and sync up with the master (or READ-WRITE) node.

The Deployment Synchronizer makes use of two different approaches for deployment artifact synchronization.

1. SVN based synchronizer

2. Registry based synchronizer

Lets look at each of these in detail.

Pre-Requisites:

Download WSO2 ESB-4.0.2 and WSO2 Governance Registry-4.1.0 binary distributions from wso2 oxygen tank

Product Clustering Setup

Lets proceed through setting up a two node WSO2 Carbon product cluster as shown above.

Step 1:

Extract the downloaded WSO2ESB-4.0.2.zip and make two copies of it as wso2esb-rw and wso2esb-ro
wso2esb-rw directory is used as the master node of the cluster and wso2esb-ro node will be used as the slave node.

Step 2:

Extract the downloaded WSO2greg-4.1.0.zip into a new directory. This will be used as the central governance and configuration registry in our ESB cluster.

Step 3:
The above WSO2 G-reg instance will run on a mysql DB instead of the default H2 database. Therefore, lets create a mysql DB first.
Open a mysql prompt in your server. Type the following commands to create a database and assign user privileges.

mysql>create database reg_db;
mysql>use reg_db;
mysql>grant all on reg_db.* TO regadmin@localhost identified by "regadmin";

Edit the CARBON_HOME/repository/conf/registry.xml of the WSO2 G-reg server as follows.

<currentDBConfig>mysql-reg</currentDBConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>

<dbConfig name="mysql-reg">
<url>jdbc:mysql://localhost:3306/reg_db</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>5</maxActive>
<maxWait>60000</maxWait>
<minIdle>50</minIdle>
<validationQuery>SELECT 1</validationQuery></dbConfig>

Now, copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2Greg server and start the server with -Dsetup switch.

sh wso2server.sh -Dsetup

This will start WSO2-Greg server on mysql.

Step 4

Now, we have started central governance and configuration registry instance of our WSO2 ESB product cluster. Now, lets proceed with configuring WSO2 ESB nodes.
Lets configure read-write node first.

We are going to run 3 carbon servers in the same machine. Therefore, we need to change the port index in CARBON_HOME/repository/conf/carbon.xml so that each of the WSO2 ESB nodes will run on their own ports without conflicting with each other.
In carbon.xml, change the following element in order to run the ESB read-write node in HTTP port 9764 and HTTPS port 9444.


<Offset>1</Offset>

We are going to store the configuration data of ESB nodes in the cluster in /_system/esbnodes space of the above registry. Also, the governance data will be stored in /_system/governance directory. Therefore, lets add the following registry mounts through CARBON_HOME/repository/conf/registry.xml.

<dbConfig name="mysql-reg">
<url>jdbc:mysql://localhost:3306/reg_db</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>5</maxActive>
<maxWait>60000</maxWait>
<minIdle>50</minIdle>
<validationQuery>SELECT 1</validationQuery></dbConfig>

<remoteInstance url="https://localhost:9443/registry">
<id>conf-gov-registry</id>
<dbConfig>mysql-reg</dbConfig>
<readOnly>false</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>

<!-- Governance data will be stored in /_system/governance collection of central registry instance --> 
<mount overwrite="true" path="/_system/governance">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

<!-- Configuration data will be stored in /_system/esbnodes collection of central registry instance -->  
<mount overwrite="true" path="/_system/config">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>

Copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2 ESB read-write node and start the server.

sh wso2server.sh

Step 5

We have configured and started the READ-WRITE node of ESB cluster. Now, we can configure the READ-ONLY node. The configuration is almost same as READ-WRITE node except the highlighted elements given below.

First, change the port offset to 2 so that the ports will not be conflicted with the other server ports.

<Offset>2</Offset>

Change the default NIO HTTP and HTTPS ports in CARBON_HOME/repository/conf/axis2.xml as follows.

<transportReceiver class="org.apache.synapse.transport.nhttp.HttpCoreNIOListener" name="http">
<parameter locked="false" name="port">8281</parameter>

<transportReceiver class="org.apache.synapse.transport.nhttp.HttpCoreNIOSSLListener" name="https">
<parameter locked="false" name="port">8244</parameter>

Add the following registry mounts through CARBON_HOME/repository/conf/registry.xml of ESB READ-ONLY node.

<dbConfig name="mysql-reg">
<url>jdbc:mysql://localhost:3306/reg_db</url>
<userName>regadmin</userName>
<password>regadmin</password>
<driverName>com.mysql.jdbc.Driver</driverName>
<maxActive>5</maxActive>
<maxWait>60000</maxWait>
<minIdle>50</minIdle>
<validationQuery>SELECT 1</validationQuery></dbConfig>

<remoteInstance url="https://localhost:9443/registry">
<id>conf-gov-registry</id>
<dbConfig>mysql-reg</dbConfig>
<readOnly>true</readOnly>
<enableCache>true</enableCache>
<registryRoot>/</registryRoot>
</remoteInstance>

<!-- Governance data will be stored in /_system/governance collection of central registry instance --> 
<mount overwrite="true" path="/_system/governance">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/governance</targetPath>
</mount>

<!-- Configuration data will be stored in /_system/esbnodes collection of central registry instance -->  
<mount overwrite="true" path="/_system/config">
<instanceId>conf-gov-registry</instanceId>
<targetPath>/_system/esbnodes</targetPath>
</mount>

Copy mysql jdbc driver (mysql-connector-java-5.1.7-bin.jar or later) to CARBON_HOME/repository/component/lib directory of WSO2 ESB read-only node and start the server.

sh wso2server.sh

Now we are done with the clustering setup but we have not done any configurations related to the deployment synchronizer yet.
We will look in to the registry based deployment synchronization first.

Step 6 - Registry based deployment synchronizer

In this mode, once you deploy an artifact from the READ-WRITE node, the artifacts will be stored in the relevant collection in the configuration registry. The other nodes of the cluster will use the registry checkin-checkout client and checkout the deployment artifacts to their file system. Then with the hot deployment functionality, the artifacts will get deployed in the cluster nodes.

Lets look at how we can use the registry based deployment synchronizer.

• Log in to management console of ESB READ-WRITE node (https://localhost:9444/carbon)
• Navigate to Configure --> Deployment Synchronizer UI

• Select Auto Commit option and click on Enable
• Log in to management console of ESB READ-ONLY node (https://localhost:5/carbon)
• Access Configure --> Deployment Synchronizer UI
• Select Auto Checkout option and click on Enable
  

• Create a proxy service (eg:-proxy1) in READ-WRITE node
• After 60 seconds (the default synchronization period), log in to the management console of the READ-ONLY node of the cluster.
• You will notice that the proxy1 is listed in the services list of READ-ONLY node
  

Step 7 - SVN based deployment synchronizer

Deployment synchronization can be achieved using a SVN repository as well. Lets look at SVN based deployment synchronizer.

In this mode, instead of a registry, we use a subversion repository as the deployment artifact store. As we check-in files to SVN, the synchronizer use a SVN client API and commit and update deployment artifacts periodically by using a SVN location.

• First, revert the registry based deployment synchronizer settings which we did above. (Disable deployment synchronizer in Configure --> Deployment Synchronizer UI)

• Have a proper SVN location. You can use an existing SVN location or create a new one.
• Add the following configuration in CARBON_HOME/repository/conf/carbon.xml of both ESB nodes. Make sure to specify correct SVN credentials and location URL according to your environment.
  

<DeploymentSynchronizer>
<Enabled>true</Enabled>
<AutoCommit>true</AutoCommit>
<AutoCheckout>true</AutoCheckout>
<RepositoryType>svn</RepositoryType>
<SvnUrl>https://svn.wso2.com/wso2/custom/projects/projects/qa/deployment-synchronizer/esb</SvnUrl>
<SvnUser>qasvn</SvnUser>
<SvnPassword>test</SvnPassword>
<SvnUrlAppendTenantId>false</SvnUrlAppendTenantId>
</DeploymentSynchronizer>

• Restart both ESB nodes.
  
• Deploy a proxy service from either one of the nodes
• Changes will get reflected into other nodes after 60 seconds (default synchronization period)
  

We looked at two different ways of sharing deployment artifacts among cluster nodes. If you come across any issues when configuring either one of the above approaches, please drop me a mail.

How to pass system properties when WSO2 Carbon server is running in daemon mode

When you start wso2 carbon server using a startup script such as wso2server.sh, you can simply pass system properties such as -DosgiConsole, just by passing system property in command line.

e.g:- sh wso2server.sh -DosgiConsole

However, if you start the server as a System process (daemon), how should you send those parameters?

Lets look at how we can start OsgiConsole, if we start server in daemon mode.

1. Open CARBON_HOME/repository/conf/wrapper.conf

2. Add the following parameter under Java additional properties section
wrapper.java.additional.11=-DosgiConsole=1234

3. Start the server as "sh wso2server.sh -start"

4. Open a new shell and "telnet localhost 1234"

WSO2 Carbon-3.2.0 - The latest with many new features and a release to remember

On 12th of June 2011, the latest version of WSO2 SOA middleware platform, WSO2 Carbon-3.2.0 has been released. Since 2009 January, WSO2 SOA middleware platform has been grown and matured and now it consists of 12 different products.

WSO2 Application Server

WSO2 Enterprise Service Bus

WSO2 Mashup Server

WSO2 Identity Server

WSO2 Governance Registry

WSO2 Data Services Server

WSO2 Business Process Server

WSO2 Business Activity Monitor

WSO2 Gadget Server

WSO2 Business Rules Server

WSO2 Complex Event Processing Server

WSO2 Message Broker

In this latest version, two new members joined into WSO2 Carbon product family. Those are WSO2 Complex Event Processing Server and WSO2 Message Broker.

In this post, I'm not going to explain the new features and enhancements included in each of these products. You can find them out in the respective product pages in wso2.com.

But, I would like to share how we evolve in terms of features, process and release methodology during the last couple of years.

In late 2008, we started the implementation of WSO2 Carbon platform. Before that, we had four isolated products (WSO2 ESB, WSO2 WSAS, WSO2 IS and WSO2 Registry) but those had limitations to work as a platform. The major objective of WSO2 Carbon platform was to build a suite of products which can be integrated easily and run together with minimum configuration overhead. At the end of the first WSO2 Carbon platform release (version 1.5.0), we were able to achieve that objective to some extent. The first release was a VERY different experience for us since we used to do releases with one product at a time so that the test team had sufficient time and resources to try out various scenarios. Because of that, we had to move in to a totally different testing paradigm.

In early 2009, we had to do a patch release of WSO2 Carbon platform (version 1.5.1) because we uncovered some critical issues after the first Carbon release. With this, we badly felt the need of adjusting our testing methodology to align with WSO2 platform vision. We spent hours in each build to test management console UIs manually and our first objective was to cut down that time and effort. In order to do that, we started developing a selenium based test automation framework. In 2009, we did 3 WSO2 Carbon platform releases (version 2.0.0, 2.0.1, 2.0.2) and we were able to make use of selenium tests heavily in all those release cycles.

In 2010, we went through a major UI refactoring process across the whole WSO2 Carbon platform. Unfortunately, we could not keep maintaining selenium tests with the pace of UI changes. Therefore, we wanted to have a test framework with minimum maintenance overhead. We wanted to call methods of various features bypassing UI elements so that the frequent UI changes do not break the tests. We started admin services based test automation framework, which used to call the operations of administration services associated with each of the features programmatically (Java/Junit).

2010 was an extremely busy year for most of us since we did a lot of frequent releases, but there were very limited automated tests. At the end of 2010, WSO2 Carbon became a fully componentized, modular SOA middleware platform which consisted of multi-tenancy and a lot of usability enhancements. As the testing team, we kept on observing the growth of WSO2 Carbon platform more than anyone else since we deal with almost all aspects of each and every product. As I said at the beginning, one of the objectives of WSO2 product platform was to provide users with ease of integration. By end of 2010, we realized how far we achieved that. Installing a new feature on any of the WSO2 Carbon based product became a simple click'n'click process. Most features worked out-of-the-box.

Now, after 2.5 years of the first Carbon release, our product platform has been improved in various aspects with 12 different products. Our development and testing methodology has been changing with each release to accommodate the pace of growth and objectives in WSO2 SOA platform. In the latest 3.2.0 release, our focus shifted towards more platform level testing, which required us to learn and familiar with each product. With the platform aspect, we cannot do individual product releases. We do all 12 products at once, which demands huge work load on both development and testing teams. At WSO2, we do not trust or rely on the number of testers in the team but their individual capabilities. We realize the need of having 80% of test coverage through some kind of automation in order to sustain with the pace. For all these years, we do not adopt or follow any specific process blindly because that works for some other organization so that we do. Instead, we are following a context-driven testing approach. We learn and enhance our processes with our own mistakes and experiences. While WSO2 is revolutionizing the world of enterprise middleware, testing and development methodology of middleware will also be redefined.

WSO2 SOA middleware Deployment Tips - 1 - validationQuery to avoid broken DB connections

I thought to put together some best practices, guidelines on deploying WSO2 SOA middleware platform. This post will remind you a well-known best practice used in most of the product deployments.

When you are maintaining DB connections, it is always recommended to use a validationQuery to check the health of the TCP connection of the connections stay in DB connection pool.

Because the connection opening is an expensive and time consuming operation, after a connection is created, it will be kept open for a specific time in the pool. When re-using these connections from the pool, there can be situations that the TCP connection to the DB is interrupted and the connection consumer gets errors such as communication link failures etc..

In order to avoid that, a validationQuery, an SQL statement specific to DBMS type, can be used which runs before using the connection.



In WSO2 middleware platform, you usually use central governance/configuration registry to store SOA metadata and various governance tasks. Governance registry supports multiple DBs as underlying data stores. You establish the connection to DB in CARBON_HOME/repository/conf/registry.xml as follows.

<dbConfig name="wso2registry">

<url>jdbc:mysql://localhost:3306/config_db</url>

<userName>>regadmin</userName>

<password>regadmin</password>

<driverName>com.mysql.jdbc.Driver</driverName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<minIdle>5</minIdle>

</dbConfig>

This is one of the places where you may experience DB connection issues as I explained above. Therefore, it is always a best practice to use a validationQuery.

If your DBMSs is MySQL or MSSQL, then use the following.

<validationQuery>SELECT 1</validationQuery>

In Oracle;

<validationQuery>SELECT 1 FROM DUAL</validationQuery>

In Postgres,

<validationQuery>SELECT version(); </validationQuery>

Similarly, when you make DB connections in WSO2 Data Services Server, make sure to use validationQuery in data source definition section as follows.

<property name="org.wso2.ws.dataservice.validationquery">SELECT 1</property>

Input validation of data services

Input validation is a new feature included in the latest version of WSO2 Data Services Server (WSO2 DSS). With this, the further processing of a data service request message can be stopped at the service layer without reaching the backend data source based on a pre-defined input validation logic.

This is achieved either using a set of built in validators or custom validator implemented by the service author.

There are four different built-in validators.

1. Long range validator

This can be used to validate an integer input parameter value as follows.

<param name="id" paramType="SCALAR" sqlType="INTEGER" type="IN" ordinal="1">

<validateLongRange minimum="1" maximum="40" />

</param>

2. Double range validator

This is useful when a validity of a float value has to be checked.

<param name="distance" paramType="SCALAR" sqlType="DOUBLE" type="IN" ordinal="2">

<validateDoubleRange minimum="1.5" maximum="850.45" />

</param>

3. Length validator

This can be used to check whether the input parameter value conform to the speficied string length.

<param name="name" paramType="SCALAR" sqlType="STRING" type="IN" ordinal="3">

<validateLength minimum="2" maximum="20" />

</param>

4. Pattern validator

This validates the string value of the input parameter against a given regular expression.

<param name="indexno" paramType="SCALAR" sqlType="STRING" type="IN" ordinal="4">

<validatePattern pattern="(?:[a-z0-9]" />

</param>

Finally, you can write your own validator based on your requirements and use it in the data service query definition. In order to do that, you must implement org.wso2.carbon.dataservices.core.validation.Validator interface in your custom validator class as follows.

import org.wso2.carbon.dataservices.core.validation.Validator;

import org.wso2.carbon.dataservices.core.validation.ValidationContext;

import org.wso2.carbon.dataservices.core.validation.ValidationException;

import org.wso2.carbon.dataservices.core.engine.ParamValue;

public class MyCustomValidator implements Validator {

public void validate(ValidationContext validationContext, String s, ParamValue paramValue)

throws ValidationException {

if (!paramValue.getScalarValue().startsWith("2")) {

throw new ValidationException("Not starting with 2!!",s,paramValue);

}

}

}

Then, you can build a jar with the custom validator class and place it in CARBON_HOME/repository/components/lib directory. After restarting the server, you can use it inside query definition as follows.

<param name="id" paramType="SCALAR" sqlType="STRING" type="IN" ordinal="1">

<validateCustom class="org.test.MyCustomValidator" />

</param>

How to start multiple WSO2 Carbon server instances as windows services

We can start more than one WSO2 Carbon (WSO2 ESB, WSAS, G-reg, GS, Mashup, IS, BPS, BRS, BAM, DS) instance by updating the transport configuration given in mgt-transport.xml (or transports.xml in 2.X series) as explained in here.

Suppose, you need to start multiple carbon server instances as windows services instead of regular wso2server.sh executable.

Then, there is an additional setting which has to be configured.

• Find wrapper.conf file inside CARBON_HOME/repository/conf

• Locate "Wrapper Windows NT/2000/XP Service Properties" section

• Update the Name of the service and Display name of the service in each instance as follows

# Name of the service

wrapper.ntservice.name=WSO2Carbon

# Display name of the service

wrapper.ntservice.displayname=WSO2 Carbon

WSO2 Stratos - Introducing WSO2 middleware Platform as a Service (PaaS)

Yesterday we released our newest integrated cloud middleware platform, WSO2 Stratos. It is now online and ready for use!

For the past few years, WSO2 has been doing many releases based on revolutionary Carbon platform, which helped to consistently improve the platform with introducing latest state-of-art technologies. The initial releases of Carbon product platform intended to improve the features of the individual products. For example, a lot of core features were introduced to WSO2 ESB, WSAS, Governance Registry etc.. Then we focused on ease of integration and more component based model. We introduced Equinox based provisioning model to build your SOA platform by picking and choosing the components as and when you need.
Then, the core product platform has been enhanced to support multi-tenancy. With the multi-tenant supported architecture at hand, our brilliant development team was able to introduce the first ever comprehensive middleware PaaS. Why is this so important?
Now, WSO2 Carbon middleware platform is available on Cloud. Anybody can try out it online by accessing http://cloud.wso2.com
Not only that, the code is 100% opensource! As far as I know, this is the first ever opensource cloud PaaS offering.

Without digging in to more details, lets try WSO2 Stratos out.

Step 1

As most of you are familiar with Google Apps, without any guidance you will be able to start using WSO2 Stratos. However, I will start from the scratch. Access http://cloud.wso2.com. This will bring up WSO2 Stratos Manager, which is the is the point of entry for all WSO2 Cloud Services such as Application Server, Business Activity Monitor, Gadget Server, Governance, Identity and Mashup Server. Click on "Register" in order to create an account for you (your organization).



This will direct you to "Select a domain for your organization" page. Specify the domain for your organization and check its availability by clicking on "Check Availability". Select "Next" to sign up your organization.



Enter the required information in "sign up your organization" page. If your registration is successful, you will see a confirmation message. With this, you just created the administration account for your organization. You will receive an email to validate Email instructions specified by you. Click on the link given in the email to access the Stratos Manager login page.
Now, you can log in to the Stratos Manager, by giving the admin credentials you have just specified during the registration process.

Step 2

After log in to Stratos Manager, you will see the cloud services list offered by WSO2 Cloud service platform.



In addition to the cloud services, you can configure a new theme for your account or you can manage new user accounts (add new users, grant necessary permissions to them etc..) through the Stratos Manager.

Suppose, you need to use cloud application server out of the seven cloud services enabled by default. Click on "Cloud Application Server" link. This will bring up "WSO2 Stratos Application Server" login page as follows.



Log in to app server by providing the same admin credentials given above. (You may question why a separate login is required. We have not provided Single Sign On support between Stratos Manager and Cloud services in the alpha1 version. This will be available soon)
The Cloud application server Home page will be shown as follows after you log in to the application server.



If you are already familiar with WSO2 WSAS, this will not make you feel strange. You could either deploy services, secure them, monitor and do a lot of tasks. In addition to that, WSO2 Cloud Application Server, is now provides with web application deployment support. You could use this as your servlet container!

Similarly, you can try out the rest of the cloud services.

How to specify a custom scope for WS-Discovery target service

As I explained in a previous blog post, WS-Discovery support is included in the latest WSO2 carbon release (3.0.0).
WSO2 WSAS can be considered as a target service hosting provider. A default scope is assigned to all the target services discovered by a Discovery Proxy.
If a discovery client looks for a service based on the service type, or some scopes, the client sends probe message to the DiscoveryProxy. Then the proxy responds back to the client with the appropriate service metadata.
Suppose we need to specify a scope for the services deployed on WSAS (target services). How can we do that?

You can specify the service scope(s), by adding a parameter in the services.xml of the Axis2 Service archive (*.aar) as follows.

<parameter name="wsDiscoveryParams">
<Scopes>http://wso2.org/engineering</Scopes>
</parameter>

Then, you can probe the services either through WS-Discovery Control Panel in WSO2 ESB management console as shown here or using WS-Discovery Client API as follows.

DiscoveryClient client = new DiscoveryClient(cfgCtx, discoveryProxyURL);
TargetService[] services = client.probe(types, scopes, matchingCriteria);

WS-Discovery with WSO2 Carbon

WS-Discovery defines a protocol to locate web services on a network, more specifically in a SOA.

The newest release of WSO2 Carbon platform (version 3.0.0) provides a complete implementation of the WS-Discovery managed mode.

This post demonstrates an end-to-end workflow of WS-Discovery support in WSO2 Carbon. With WS-Discovery components for WSO2 Carbon, any Carbon server can act as a WS-Discovery client, a WS-Discovery proxy or a WS-Discovery target service. We will use three products from WSO2 Carbon product suite, namely WSO2 Governance Registry (G-reg), WSO2 ESB and WSO2 WSAS.

WSO2 G-reg acts as the central repository in which the discovered services and the related meta data are stored. WSO2 WSAS is used to host the target services. WSO2 ESB participates in the scenario as a discovery client which looks for services and endpoints.

Lets start with setting up the environment.

Pre-requistes:

Download WSO2 Governance Registry-3.5.0, WSO2 WSAS-3.2.0 and WSO2 ESB-3.0.0

Install and run each product. In this demonstration, I will run G-reg in http port 9763 and https port 9443. WSAS on http port 9764 and https port 9444. ESB on http port 9765 and https port 9445. In this way, I can run all three products in single machine.

Step 1

As I explained initially, WSO2 Governance Registry acts as the discovery proxy. If you access the https://localhost:9443/services/DiscoveryProxy?wsdl, you will find out the wsdl of the discovery proxy service. Make a note of the DiscoveryProxy service endpoint (https://localhost:9443/services/DiscoveryProxy)

Step 2

Now, we need to configure WSAS instance, so that it takes part in service discovery scenario. WSAS is used to host target services. Therefore, it sends a unicast Hello messages to a Discovery Proxy when the services join a network. So, we must configure the Discovey Proxy service endpoint in WSAS.

Open WSAS_HOME/repository/conf/axis2.xml and add the following parameter.

<parameter name="DiscoveryProxy">https://localhost:9443/services/DiscoveryProxy</parameter>

Save the file and restart WSAS. This will enable WSAS to send unicast hello messages when the services are deployed.

Now, access the management console of WSO2 Governance Registry instance and go to Metadata-->List-->Services page. You will see that 3 discovered services are listed there.

These are the three default services included in WSAS. When starting WSAS, those three services have sent the hello messages to the discovery proxy and register them in G-reg.

Image:- Discovered services in G-reg



Now, log in to WSAS management console and deploy a new service. If you refresh the above service list page, you will see a new service is discovered by G-reg and it is assigned a unique service name.



Step 3



The third member participates in our scenario is WSO2 ESB, which acts as a discovery client. WSO2 ESB provides you with a user interface to mange the client aspects of WS-Discovery. Using that, you can connect to remote WS-Discovery proxies and probe them for any services and service endpoints which have been already discovered.



Log in to ESB management console and select Configure > WS-Discovery from the left navigation menu to access WS-Discovery Control panel.







Image:- WS-Discovery Control Panel in ESB



In order to connect to the remote DiscoveryProxy (in G-reg) and make use of the discovered services, we should configure a discovery proxy in ESB. Click on "Add Discovery Proxy" link to add a new proxy. You will be directed to discovery proxy settings screen in which you can give a name for the proxy and the remote DiscoveryProxy URL (In our case https://localhost:9443/services/DiscoveryProxy). You will be redirected to the home page of WS-Discovey Control Panel once the proxy is created. The created proxy will be listed in the control panel home page. Click on "View" to find the target services and endpoints discovered by the proxy. You will be directed to a page as shown below.







Here, you will see that 4 service are discovered and shown with a UUID and associated endpoints of each discovered service.

Click on a discovered service UUID. You will be directed to a new page with more information about the service. Using this page, either you can create an ESB endpoint or directly create an ESB proxy service referring to the discovered service.



We have seen the basic workflow of WS-Discovey implementation of WSO2 Carbon platform. If you have any issues with the above steps, drop me a mail or contact WSO2 ESB forum at www.wso2.org

WSO2 QA Test Framework - Fundamentals

WSO2 QA Test framework has been developed to replace repetitive manual test procedures followed during the release cycles. We identified the tests which provide much ROI with automation and used them for phase1 of the test framework development project.
With the introduction of WSO2 Carbon product platform in late 2008, all java based products are implemented using the base carbon platform. All products (9 all together) are released at the same day which is a very different experience specially for a QA team. All products are supposed to go through a set of common tests and a set of product specific tests. All are supposed to work on multiple application servers, multiple JVMs, multiple browsers, multiple operating systems, multiple DBMSs etc.. hell a lot of test combinations! The product count exceeded the number of people in test team. More products are expected to be introduced in near future. Therefore, the only viable solution to manage the QA process is to automating as more tests as possible.
So, we started implementing our automated test framework in 2009 March.
As I discussed in some previous posts, we chose selenium for automation. We used Selenium Remote Control Java client driver to drive selenium tests with Junit. However our tests were not restricted to web based selenium scripts. We have written numerous tests which used Axis2 ServiceClient API and some other API methods to invoke web services, sending messages via secure channels, reliable messaging, message mediation etc.
We derived our project structure which adheres to maven as given below. You can get a SVN checkout of 2.0.3 branch of the test framework (which is the most stable version at the moment) from https://wso2.org/repos/wso2/branches/commons/qa/web-test-framework/2.0.3

common
bps
registry
wsas
esb
gs
mashup
is
ds

Each project is built using its pom.xml at the root of the project directory. We used selenium maven plugin and surefire plugin to start selenium RC server and launch Junit tests respectively.
commons project is used to maintain tests which are used in all products. For example, org.wso2.carbon.web.test.common.RegistryBrowser is common for all products since all products have a common registry browser. We used svn:externals to link the common classes to relevant projects.
You can find all common classes at https://wso2.org/repos/wso2/branches/commons/qa/web-test-framework/2.0.3/commons/src/test/java/org/wso2/carbon/web/test/common/

In addition to that, commons project is used to store the test artifacts which are shared among multiple products. For example JDBC connector jars and keystores are used in all products and those are stored in commons/lib directory.

The other most important resource included in commons project is framework.properties configuration file. It is used to configure the test framework according to the test environment. Under the "global properties" section of the framework.properties file, you could find the following properties.

host.name=172.16.37.1
http.port=9763
https.port=9443
carbon.home=/home/charitha/products/wsas/wso2wsas-3.1.3
context.root=/wsas
browser.version=firefox
admin.username=admin
admin.password=admin
module.version=2.03

You must change these properties as per your test environment settings. http.port and https.port are the embedded tomcat servlet transport ports used in WSO2 Carbon products. (full explanation of these properties will be included in a future post)

Lets look at a product specific test suite. Go to wsas directory in your test framework checkout. This is the project used to run WSO2 WSAS specific tests. You will find the following files and directories at the root of this project.

lib
src
pom.xml
runAll.sh
runAll.rb
wsas_test_suites.txt

lib directory is used to store the test artifacts specific to WSAS such as axis2 services, jaxws services etc.
You can invoke tests using two different ways.

• Direct maven invocation - you can run tests individually by passing a system property as follows

mvn clean install -Dtest.suite=usermanagement

• Run tests through a shell script - You can run tests individually or as a whole suite or few tests at once using this way. Here, it is required to uncomment the selected tests in wsas_test_suites.txt. The runAll.sh shell script read the test names from wsas_test_suites.txt and invoke each test.

sh runAll.sh

In any of the above mechanisms, we call a central test suite class which takes care of calling the individual tests. For each project we have a separate AllTests.java class which extends junit.framework.TestSuite parent. In our example, org.wso2.carbon.web.test.wsas.AllTests.java is the TestSuite.

See https://wso2.org/repos/wso2/branches/commons/qa/web-test-framework/2.0.3/wsas/src/test/java/org/wso2/carbon/web/test/wsas/AllTests.java for more details about this class.

You will also notice in AllTests class that initBrowser() method of the BrowserInitializer class is called to launch the browser instance for a particular test as follows.

public synchronized static void initBrowser()throws Exception{


if (browser == null) {
browser = new DefaultSelenium("localhost", 4444, "*"+property.getProperty("browser.version"), "https://" + property.getProperty("host.name") + ":" + property.getProperty("https.port"));
browser.start();
browser.setSpeed("200");

}
}

You can easily try out WSO2 QA test framework once you download and start using any of WSO2 Carbon based product. This is not a detailed explanation of the all features available in our test framework. I will guide you through more information in future posts. If you encounter any issues while using the test framework, please drop us a mail - architecture@wso2.org

Stay tuned.. will post more on automation soon!

How to start WSO2 Carbon AMIs

The latest product versions of WSO2 SOA platform are available for download now. This release addresses various bug fixes and enhancements. In addition to the usual binary, source and document distributions, WSO2 products are now available as cloud virtual machines.
WSO2 cloud virtual machines provides you with the ability to implement your SOA infrastructure in private or public cloud. In this post, we will look at how AMIs of WSO2 Carbon family of products can be used.
I will use WSO2 ESB AMI for the demonstration, however you can follow the same steps to start any of the WSO2 Carbon AMI.

Pre-requisite:
You should have the necessary Amazon EC2 account ready for AMI administration. AMI EC2 API tools should be installed in your computer. You may refer to this tutorial in order to set up the infrastructure.

Step 1
Go to WSO2 ESB home page in Oxygen Tank. Click on WSO2 Cloud Service under the Download icon.
You will find WSO2 ESB cloud virtual machines in that page. Take a note of AMI ID given there (ami-878569ee)

Step2
Open a shell and issue the following command to start 64bit WSO2 ESB AMI instance.

ec2-run-instances ami-878569ee -k --instance-type m1.large

Replace with the name of your public/private keypair. Read http://docs.amazonwebservices.com/AmazonEC2/gsg/2006-06-26/running-an-instance.html if you do not know hot to generate a keypair.


Step 3

The above command starts up an AMI instance with a pre-configured WSO2 ESB server. Also, the above will return the ID of the started instance (instance-id).

e.g:-

INSTANCE i-c28093aa ami-878569ee pending charithakankanamgewso2-keypair 0 m1.large 2010-02-04T06:03:25+0000 us-east-1c aki-a3d737ca ari-7cb95a

After a few seconds, issue the following command to get the public DNS name of WSO2 ESB so that we can access ESB management console.

ec2-describe-instances

e.g:- ec2-describe-instances i-c28093aa

This will return the public DNS name as follows.

INSTANCE i-c28093aa ami-878569ee ec2-174-129-86-140.compute-1.amazonaws.com domU-12-31-39-0C-21-B2.compute-1.internal

Step 4

Open a browser and access URL, https://ec2-174-129-86-140.compute-1.amazonaws.com. You will be able to log in to WSO2 ESB management console.

How to install WSO2 Carbon cluster management feature

I have written and published a tutorial on wso2.org which guides you through the steps of installing WSO2 Carbon cluster management feature.

Running WSO2 Carbon based products on IBM JDK

The next minor release of WSO2 Carbon product platform (2.0.2) will be fully compatible with IBM JDK (IBM J9 VM 2.4). You simply need to update transports.xml as follows in order to run any of the WSO2 carbon based product on IBM JDK.

Open CARBON_HOME/transports.xml

Add the following parameter to HTTPS transport configuration

<parameter name="algorithm">IBMX509</parameter>

Now, restart carbon. Server will be started successfully on IBM JDK.

Carbon-2.0.2 product platform will be released within next week. Stay tuned..

The newest version (2.0.1) of WSO2 carbon family of prodcuts released!!

The latest version of WSO2 carbon based products were released yesterday. More info can be found at http://wso2.org/projects/carbon

New Features in This Release
----------------------------
* Improved transaction support.
* Improved Support for deploying on top of WebSphere, WebLogic, and
JBoss.
* P2 based provisioning for WSO2 Carbon family of products.
* Numerous bug fixes.

How to run multiple WSO2 WSAS instances in a single host

I have published a simple three step tutorial on running multiple WSO2 WSAS instances in a single host in WSO2 Oxygen Tank. Have a look!

WSO2 Carbon is among InfoWorlds best open source software

WSO2 has been named an InfoWorld 2009 Best of Open Source Software (Bossie) Award winner.

http://www.infoworld.com/d/open-source/best-open-source-platforms-and-middleware-758?current=9&last=8?r=76

Proud to be a member of the award winning team :)

Remotely starting OSGI console when WSO2 Carbon runs on an application server

When WSO2 Carbon based product (WSAS, ESB, G-reg, BPS, IS) runs in standalone mode, you can start server with equinox OSGI console just by issuing -DosgiConsole system property.

wso2server.bat -DosgiConsole

How do you connect to OSGI console when you are running Carbon on an application server such as WebSphere or WebLogic (or tomcat, JBoss etc..)?

1. Open WEB-INF/web.xml file of the carbon web application
2. Uncomment the following element

<init-param>

<param-name>osgiConsole</param-name>

<param-value>-console 19444</param-value>

</init-param>

3. Now restart carbon with these settings.
4. Open a new command window/shell and connect to osgiConsole using telnet as follows

telnet localhost 19444

Build your own server using WSO2 Carbon components

Building a customized server with a set of components downloading from a public repository...
It is no longer a hard and complex task. Now you can build your own version of WSO2 carbon platform using its Equinox P2 based provisioning capabilities.
This post guide you through the steps to build your own server platform using WSO2 carbon-2.0.0.

Pre-requisites:
Download the latest version of WSO2 Carbon server. WSO2 Carbon is the base platform on which we build our customized server.

Step 1

Start carbon server with osgiConsole so that we can issue commands to manage OSGI framework.

cd CARBON_HOME/bin (CARBON_HOME is the location where you unzipped wso2carbon-2.0.0.zip)
wso2server.bat -DosgiConsole

Step 2

Next, we need to specify artifact and metadata repositories from which we download several features and their metadata.

In the osgi console, issue the following commands.

osgi>provaddrepo http://dist.wso2.org/p2/carbon/releases/2.0

osgi>provaddartifactrepo http://dist.wso2.org/p2/carbon/releases/2.0



Step 3

Lets check the installable features included in the above artifact repository.

osgi>provlg

This will list down all the features available in the specified artifact repository as follows.


Step 4

Now we have connected to the artifcat and metadata repositories and looked at the installable features. Suppose we need to build our customized server with the following components out of the features included in the artifact repo.

• data services
• service management
• tools

We can install data service feature as follows.

osgi>provinstall org.wso2.carbon.dataservices.feature.group 2.0.0

If the installation is successful, you will get "installation complete" message.

Similarly, you can install service management and tools features.

osgi>provinstall org.wso2.carbon.tools.feature.group 2.0.0

osgi>provinstall org.wso2.carbon.tools.feature.group 2.0.0



Step 5

Now you must restart carbon server to apply the changes you have done to the server.
Issue the following command to shutdown the server.
osgi>shutdownCarbon

Start the server again using wso2server.bat{sh}

After server is started, access management console using http://localhost:9443/carbon

Log in to management console using the default admin credentials (admin/admin)



You should notice that the data services, service management and tools features are added to the carbon core platform. In other words, we have built our own server on top of carbon-core server.
Simple.. isn't it?