Must attend talk in WSO2Con2011 - Security in practice by Prabath Siriwardhana

Time flew in light speed and finally the days have arrived! WSO2Con 2011, a conference which you should not miss. Though you are a seasoned IT professional or a student keen in learning latest technologies, it is the event of the year.

Out of more than 25 sessions which are planned to be held, my attention goes to one particular session first. It is "Security in Practice" by Prabath Siriwardhana which will be held on 14th of September, 2011 at 3.30 PM. Why do I like it than any other session?

Prabath is a brilliant speaker. I like the way he presenting. He has his own style which attract anyone in the audience.

Prabath is the expert of the most complex and hard-to-understand aspect of Service Oriented Architecture, Security.

You can imagine the complexity of delivering a presentation in such a tough, complicated subject matter

But, Prabath has the ability to inject complex subject matters into your head in very nice manner. You should not miss this great session of Prabath if you are attending WSO2Con 2011!

WSO2 Stratos Application Server - Apache Tomcat As a Service

Few days back, one of my friends developed a new web application and he wanted to try it out by deploying on a J2EE server. I was at his place and was observing the steps he followed to deploy the web app.

• Downloaded the latest version of Apache Tomcat Server
• Extracted and installed it on a local server
• Ran the startup script and started the server

This took considerable amount of time. In the world of everything offered as services, my friend was not aware of the free service of hosting web applications in cloud. In other words, my friend has not known of "Apache Tomcat As a Service".

If I was him, I would have tried out my web applications with Apache Tomcat by following the three simple steps given below.



Step 1 - Create a demo user account at WSO2 StratosLive



This is a simple registration procedure and follow the steps given in my previous blog post to create a demo account.



Step 2 - Access WSO2 Stratos Application Server

You can access Application Server from WSO2 Stratos Manager home page or just type https://appserver.stratoslive.wso2.com in your favorite browser.









Step 3 - Upload web application



Click on Manage --> Web Applications --> Add in the left navigation menu. You will be directed to the following screen.







Browse the webapp (*.war) in your local machine and click on upload. I downloaded a sample calendar.war webapp from here for demonstration purposes. Once the file upload is done, the deployed calendar.war will be listed in Running Web Applications page where you can do all web app management tasks such as session handling, reloading webapps, deleting etc..



Isn't this the simplest way to run your web applications? You will not be charged a cent for the steps which I demonstrated above. Once you are satisfied with testing your web apps on Stratos Application Server, you can upgrade your usage plan as needed and have an enterprise ready web application running on cloud!

WSO2 Data As a Service - Have your own data storage in cloud!

WSO2 StratosLive, the most complete open PaaS (platform-as-a-service), includes multiple types of SOA middleware services which can be used to build and host your SOA based solution. One of the newest additions to WSO2 middleware platform is, the functionality which enables you to have your own database in cloud for free!

This post guides you through the steps of creating your own data storage in cloud using WSO2 StratosLive.

Step 1

First, we need to create an account in stratoslive. You can have a demo account free of charge.

Open a browser and access WSO2 StratosLive landing page. Click on Get Started Now for Free

You will be directed to domain registration page.

Fill the form and follow the instructions in registration email to activate the account.

Once you logged into WSO2 StratosLive, you will be shown all of the cloud services which are available for use. Out of those services, click on Data Services Server link.







You will be directed to the home page of WSO2 Data Service Server.



Step 2

Now, we have accessed the home page WSO2 Data service server in cloud. You will notice Databases menu item in the left navigation pane as shown below.







Click on Databases --> Add option in the above screen. You will be directed to the New Database page. In this page, you will be asked to provide Database Server Instance Name and a name for the database which you are going to add. By default, WSO2 Data Services Server provides you with a pre-configured Amazon RDS instance, in which you can create your database. You also add your own database server instance and use that instead as you preferred. But in this example, we will use the default database server instance. Lets select the default DB server instance and enter "qa" as the name of the database.

Once you click on create, you will be prompted a confirmation dialog, "Database has been successfully created".



Your newly created database will be listed in database list as shown below. Note that, the name of the database is suffixed with tenant domain name.







Step 3



We have created our first database successfully in cloud using WSO2 Data Service Server. Lets, add a new user to the database and create table so that we can use the database for any DB related operation.



Click on Manage icon in the above table. Database User Management page will be shown.







Before adding any user, you should add a database user privilege group. Click on Add Privilege Group and add a new privilege group. Specify select, insert, update, delete and create privileges for the group as shown below.







The newly created privilege group will be shown in Database User Privilege Groups page as follows.







Now, go back to the previous page and click on add new database users icon to access Database Users screen where you can add DB users. Click on add new user.

Specify "test" as username and password and select "group1" as the privilege group.







Step 4



Once you create a new DB user, it will be listed in Database Users page with the options to explore database using the given user credentials, edit, drop and create carbon datasource.

Click on explore database. Database console screen will be displayed as follows. This console allows you to issue any SQL statement and act as an SQL editor for your database.

You may type any SQL statement and hit run button so that the query will be executed on the database which has been created in the default amazon RDS instance provided by WSO2.







Now, you can use this DB as your storage media in your SOA platform in cloud. For example, you may deploy a webapp in WSO2 Stratos AppServer which talks to this particular DB. Or you may simply create a data service using this database.

PaaS testing

PaaS (Platform as a Service) is an application delivery model which provides services to design, develop, test, deploy and host applications. It plays a key role in cloud computing space similar to the other services, IaaS (infrastructure as a service) and SaaS (software as a service). Google App engine, vmForce and WSO2 Stratos are some of the popular PaaS offerings.
This post is not about detailed concepts of PaaS or cloud computing. Rather, I'm going to look at the testing and quality assurance aspects of PaaS. PaaS testing is not a widely discussed topic. There is no pre-defined model for PaaS testing. We need to explore the features of PaaS and derive an approach for testing PaaS apps. I will go through some basic components based on my experience of testing WSO2 Stratos opensource cloud platform.

Hosted services testing - Minimized deployment overhead

In simplest terms, PaaS testing is about testing a hosted platform. So, there is no deployment and configuration overhead for the test team. Testers are expected to access the hosted services remotely using web browser and carry out functional and non-functional testing.

Lesser test platform combinations

When you test a standalone product or a combination of products, you should try out the possible platform combinations such as different types of clustering setups, multiple DBMSs, Application Servers or various operating systems. However, when you test the same product suite or platform on cloud (PaaS), you are bound to one optimum configuration stack. You have one or two choices. There is no requirement to try all possible platform setups.

Multi-tenancy aspects

Multi-tenancy allows a single application to emulate multiple application instances. With multi-tenancy, a single application can be shared across many organizations. Therefore, series of tests should be carried out carefully to verify the multi-tenant aspects. For example, if company A logs in to a web application hosting service in your PaaS offering and deploy a web app, then the users of company B should not be able to locate it unless company A made it public. In other words, the configurations done by one tenant should not be exposed to other tenants.
These mission-critical aspects are some of the key requirements which should be taken into account in any PaaS testing model.

Performance and scalability

Any hosted service should conform with the SLAs. Similar to a website, services included in a PaaS can be accessed by multiple users concurrently. Therefore, sufficient amount of performance testing must be carried out. The usual usage pattern of standalone technology platform can be dramaticallty changed when it is hosted as a service in cloud. For example, the general use cases of WSO2 Carbon SOA middleware platform are different from the standalone version when the platform is exposed as a service. WSO2 SOA middleware platform consists of 12 different products and all of them includes a management UI for administration purposes. In non-cloud based deployment, the multiple, concurrent users access to the management UIs are minimum since it is very rare that hundred of administrators accessing management console at once.
However, if this platform is hosted as a service in cloud (PaaS), then the concurrent user access to the management UIs are a highly desirable scenario.

It is a requirement that the hosted platform should be able to handle load seamlessly without affecting consumers. Auto-scaling is a key feature of WSO2 Stratos cloud platform. When new resources are needed, WSO2 Stratos transparently adds services and when load goes down, WSO2 Stratos automatically brings services down. Testers should ensure, with the load, the new EC2 instances spawning up and down correctly. The tools like hybridfox are very useful in these situations.

Data

If your PaaS provides users with the ability to store their data in cloud and various database centric operations, then the testing will be much complicated. As a tester, you must ensure the availability of data sources, accessibility and provisioning aspects. You must also take extra care that, if the data are geographically distributed in cloud, then it adheres to the legal requirements of the users of your PaaS.

Security

Security is the utmost important aspect of any cloud offering. It is recommended to have a separate group of security testers who do penetration testing and ethical hacking in order to ensure secure infrastructure of your PaaS. WSO2 Stratos cloud platform allows users to deploy their webapps, web services and various custom code. Therefore we must ensure that no vulnerable code are deployed and a code deployed by one tenant does not affect the operations of others or whole platform.

Virtualization

Public PaaS offerings are usually based on virtualized infrastructure such as Amazon EC2. Therefore, a full functional test cycle should be carried out on EC2 VMs inorder to make sure that there are no regressions due to virtual servers.

In addition to the above key areas, PaaS testing model should consists of the general web application testing aspects such as cross browser testing, accessibility testing, localization testing etc..

Data driven testing with Jmeter user parameters

This is a follow up to one of my previous posts which explained data driven web service testing using CSV config element in Jmeter. There, we used CSV file to read input data for SOAP/XML-RPC sampler.

In this post, we will look in to using User Parameters pre-processor element as the data source instead of a CSV file.

Step 1

We are going to use the same web service which we used in my previous post, temperature conversion service. Please add the SOA/XML-RPC sampler, the SOAP request and the necessary thread group as described in step 1 and 2 of that post

Step 2

Lets parameterize the payload of SOAP message so that different requests will be sent to the service with each test run. Instead of reading data from a CSV file, we can add a User Parameter pre processor element in Jmeter test plan.

Right click on the thread group of your jmeter test plan and select Add --> Pre Processors ---> User Parameters

Click on Add Variable and specify celcius as the name of variable. Add few users and enter celcius values for each user as shown below.

Step 3

Now, parameterize the payload of SOAP as follows.

<tem:nCelcius>${celcius}</tem:nCelcius>

Step 4

Increase the thread count corresponding to the user count in your user parameters pre-processor element and run the test. You will notice that the Celcius figure will be varied in each request.

Based on your requirements, you can select either CSV config element or User Parameter pre-processor element for data driven testing. If you have large number of variables to be parameterized, CSV config is the best option.

WSO2 Carbon-3.2.0 - The latest with many new features and a release to remember

On 12th of June 2011, the latest version of WSO2 SOA middleware platform, WSO2 Carbon-3.2.0 has been released. Since 2009 January, WSO2 SOA middleware platform has been grown and matured and now it consists of 12 different products.

WSO2 Application Server

WSO2 Enterprise Service Bus

WSO2 Mashup Server

WSO2 Identity Server

WSO2 Governance Registry

WSO2 Data Services Server

WSO2 Business Process Server

WSO2 Business Activity Monitor

WSO2 Gadget Server

WSO2 Business Rules Server

WSO2 Complex Event Processing Server

WSO2 Message Broker

In this latest version, two new members joined into WSO2 Carbon product family. Those are WSO2 Complex Event Processing Server and WSO2 Message Broker.

In this post, I'm not going to explain the new features and enhancements included in each of these products. You can find them out in the respective product pages in wso2.com.

But, I would like to share how we evolve in terms of features, process and release methodology during the last couple of years.

In late 2008, we started the implementation of WSO2 Carbon platform. Before that, we had four isolated products (WSO2 ESB, WSO2 WSAS, WSO2 IS and WSO2 Registry) but those had limitations to work as a platform. The major objective of WSO2 Carbon platform was to build a suite of products which can be integrated easily and run together with minimum configuration overhead. At the end of the first WSO2 Carbon platform release (version 1.5.0), we were able to achieve that objective to some extent. The first release was a VERY different experience for us since we used to do releases with one product at a time so that the test team had sufficient time and resources to try out various scenarios. Because of that, we had to move in to a totally different testing paradigm.

In early 2009, we had to do a patch release of WSO2 Carbon platform (version 1.5.1) because we uncovered some critical issues after the first Carbon release. With this, we badly felt the need of adjusting our testing methodology to align with WSO2 platform vision. We spent hours in each build to test management console UIs manually and our first objective was to cut down that time and effort. In order to do that, we started developing a selenium based test automation framework. In 2009, we did 3 WSO2 Carbon platform releases (version 2.0.0, 2.0.1, 2.0.2) and we were able to make use of selenium tests heavily in all those release cycles.

In 2010, we went through a major UI refactoring process across the whole WSO2 Carbon platform. Unfortunately, we could not keep maintaining selenium tests with the pace of UI changes. Therefore, we wanted to have a test framework with minimum maintenance overhead. We wanted to call methods of various features bypassing UI elements so that the frequent UI changes do not break the tests. We started admin services based test automation framework, which used to call the operations of administration services associated with each of the features programmatically (Java/Junit).

2010 was an extremely busy year for most of us since we did a lot of frequent releases, but there were very limited automated tests. At the end of 2010, WSO2 Carbon became a fully componentized, modular SOA middleware platform which consisted of multi-tenancy and a lot of usability enhancements. As the testing team, we kept on observing the growth of WSO2 Carbon platform more than anyone else since we deal with almost all aspects of each and every product. As I said at the beginning, one of the objectives of WSO2 product platform was to provide users with ease of integration. By end of 2010, we realized how far we achieved that. Installing a new feature on any of the WSO2 Carbon based product became a simple click'n'click process. Most features worked out-of-the-box.

Now, after 2.5 years of the first Carbon release, our product platform has been improved in various aspects with 12 different products. Our development and testing methodology has been changing with each release to accommodate the pace of growth and objectives in WSO2 SOA platform. In the latest 3.2.0 release, our focus shifted towards more platform level testing, which required us to learn and familiar with each product. With the platform aspect, we cannot do individual product releases. We do all 12 products at once, which demands huge work load on both development and testing teams. At WSO2, we do not trust or rely on the number of testers in the team but their individual capabilities. We realize the need of having 80% of test coverage through some kind of automation in order to sustain with the pace. For all these years, we do not adopt or follow any specific process blindly because that works for some other organization so that we do. Instead, we are following a context-driven testing approach. We learn and enhance our processes with our own mistakes and experiences. While WSO2 is revolutionizing the world of enterprise middleware, testing and development methodology of middleware will also be redefined.

WSO2 SOA middleware Deployment Tips - 1 - validationQuery to avoid broken DB connections

I thought to put together some best practices, guidelines on deploying WSO2 SOA middleware platform. This post will remind you a well-known best practice used in most of the product deployments.

When you are maintaining DB connections, it is always recommended to use a validationQuery to check the health of the TCP connection of the connections stay in DB connection pool.

Because the connection opening is an expensive and time consuming operation, after a connection is created, it will be kept open for a specific time in the pool. When re-using these connections from the pool, there can be situations that the TCP connection to the DB is interrupted and the connection consumer gets errors such as communication link failures etc..

In order to avoid that, a validationQuery, an SQL statement specific to DBMS type, can be used which runs before using the connection.



In WSO2 middleware platform, you usually use central governance/configuration registry to store SOA metadata and various governance tasks. Governance registry supports multiple DBs as underlying data stores. You establish the connection to DB in CARBON_HOME/repository/conf/registry.xml as follows.

<dbConfig name="wso2registry">

<url>jdbc:mysql://localhost:3306/config_db</url>

<userName>>regadmin</userName>

<password>regadmin</password>

<driverName>com.mysql.jdbc.Driver</driverName>

<maxActive>50</maxActive>

<maxWait>60000</maxWait>

<minIdle>5</minIdle>

</dbConfig>

This is one of the places where you may experience DB connection issues as I explained above. Therefore, it is always a best practice to use a validationQuery.

If your DBMSs is MySQL or MSSQL, then use the following.

<validationQuery>SELECT 1</validationQuery>

In Oracle;

<validationQuery>SELECT 1 FROM DUAL</validationQuery>

In Postgres,

<validationQuery>SELECT version(); </validationQuery>

Similarly, when you make DB connections in WSO2 Data Services Server, make sure to use validationQuery in data source definition section as follows.

<property name="org.wso2.ws.dataservice.validationquery">SELECT 1</property>