Friday, September 24, 2010

How to call Oracle Stored Procedures from data services

An Oracle stored procedure is a program stored in an Oracle database which allows business logic to be embedded inside database as an API. You can expose the stored procedures as web services using wso2 data services server. This example takes you through creating a simple stored procedure in an Oracle DB and expose it as a data service using WSO2 Data Services Server.

Pre-requisites:

Download wso2 data services server from here
Oracle 10g or later

Step 1: Create and Populate sample database

First, open sqlplus shell and log in to DB as follows.

sqlplus /nolog

SQL*Plus: Release 10.2.0.1.0 - Production on Wed Sep 23 22:52:01 2009

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> connect sys as sysdba;
Enter password:
Connected.

We should create a user and grant him the necessary privileges as follows.

SQL> create user sample identified by sample account unlock;

User created.

SQL> grant connect to sample;

Grant succeeded.

SQL> grant create session, dba to sample;

Grant succeeded.

SQL> exit;

Now logged in as the new schema user as follows.

sqlplus sample/sample@orcl

SQL*Plus: Release 10.2.0.1.0 - Production on Wed Sep 23 22:52:01 2009

Copyright (c) 1982, 2005, Oracle. All rights reserved.

SQL> connect sys as sysdba;
Enter password:
Connected.
SQL>

Create a table and insert data as follows.

SQL> create table employee (id number primary key, name varchar(100), address varchar(100));

SQL> insert into employee(id, name, address) values (1, 'charitha', 'colombo');
SQL> insert into employee(id, name, address) values (2, 'john', 'galle');
SQL> insert into employee(id, name, address) values (3, 'michel', 'otawa');
SQL> insert into employee(id, name, address) values (4, 'carl', 'dallas');
SQL> insert into employee(id, name, address) values (5, 'chanmira', 'colombo');
SQL> commit;

Now we are ready with oracle database and schema.
Lets write a simple stored procedure to insert more data to employee table.

Step 2: Writing a simple stored procedure

SQL> create or replace procedure addEmployeeSP(id number, name varchar, address varchar) is
2 begin
3 insert into employee (id, name, address) values (id, name, address);
4 end;
5 /

Step 3: Copy oracle jdbc driver

In order for wso2 data services server to communicate to oracle DB, we should download oracle jdbc driver (ojdbc14.jar) and copy in to DS_HOME/repository/components/lib directory.

Step 4: Create the data service through UI wizard

First, start wso2 data services server by running wso2server.sh {bat} which can be found at DS_HOME/bin
Then access management console through https://localhost:9443/carbon and log in using default admin credentials (admin/admin)

Click on Data Service --> Create in the left menu which will bring up data service creation wizard.

Enter a name for the data service and click on next. (For this example, I have specified "BlogExampleDataService" as the service name)



In Data Sources screen, click on Add new data source link and specify a data source ID and select RDBMS as the data source type. Then select Oracle as the Database engine and enter the db info as follows.

Driver Class = oracle.jdbc.driver.OracleDriver
JDBC URL = jdbc:oracle:thin:sample/sample@10.100.1.10:1521/orcl
User Name = sample
Password = sample



Click on "Test Connection" to see whether you can connect to oracle server correctly. Then click on save. Now we have created a data source hence we can proceed through the wizard.

Click on Next to move to query definition page. Select Add New Query to create a new query for our data service.

Enter a query ID (ex:- employees) and select the data source we just created from the Data Source drop down.

We can specify our SQL statement in the SQL text area. We need to call the oracle stored procedure which has been created earlier. You can enter "call addEmployeeSP(?,?,?)" as the query to call oracle stored procedure. ? denotes the input parameters which should be passed to the stored procedure.



Next, click on Add new input mapping and add three new input parameters since our query accepts 3 different parameters.



After adding all 3 input parameters, the query will be looked as follows.



We can save the query now. Our stored procedure inserts employee records in to the table therefore it does not return anything.
Because of that, the output mappings are not required for the query.

In order to check whether the records are correctly added to the database, lets create another query, selectEmployees as shown in the following screen.



Now, we have created both queries. Lets add operations which are necessary to run these queries.

Click Next in the Queries screen to proceed through the wizard which will bring up Operations page. Click on Add New Operation link. Specify operation name, addEmployee and select Query ID, employees.



Similarly, add an operation for selectEmployee query and name it as selectAllEmployees.

Click on Finish to deploy the data service. Once it is deployed, the service will be shown in the service list. Click on Try this service link to test the service.
In there, you will find two operations, addEmployee and selectAllEmployees. First invoke addEmployee operation by specifying id, name and address.
id=6
name=bloguser
address=notown

Now, if you invoke selectAllEmployees operation, you will see that the employee table has been updated with a new record.



Thats all! Drop me a mail or post your question at wso2.org forum if you have any questions about WSO2 Data Services Server.





Wednesday, September 22, 2010

Input validation of data services

Input validation is a new feature included in the latest version of WSO2 Data Services Server (WSO2 DSS). With this, the further processing of a data service request message can be stopped at the service layer without reaching the backend data source based on a pre-defined input validation logic.
This is achieved either using a set of built in validators or custom validator implemented by the service author.

There are four different built-in validators.

1. Long range validator
This can be used to validate an integer input parameter value as follows.

<param name="id" paramType="SCALAR" sqlType="INTEGER" type="IN" ordinal="1">
<validateLongRange minimum="1" maximum="40" />
</param>

2. Double range validator
This is useful when a validity of a float value has to be checked.

<param name="distance" paramType="SCALAR" sqlType="DOUBLE" type="IN" ordinal="2">
<validateDoubleRange minimum="1.5" maximum="850.45" />
</param>

3. Length validator
This can be used to check whether the input parameter value conform to the speficied string length.

<param name="name" paramType="SCALAR" sqlType="STRING" type="IN" ordinal="3">
<validateLength minimum="2" maximum="20" />
</param>

4. Pattern validator
This validates the string value of the input parameter against a given regular expression.

<param name="indexno" paramType="SCALAR" sqlType="STRING" type="IN" ordinal="4">
<validatePattern pattern="(?:[a-z0-9]" />
</param>

Finally, you can write your own validator based on your requirements and use it in the data service query definition. In order to do that, you must implement org.wso2.carbon.dataservices.core.validation.Validator interface in your custom validator class as follows.


import org.wso2.carbon.dataservices.core.validation.Validator;
import org.wso2.carbon.dataservices.core.validation.ValidationContext;
import org.wso2.carbon.dataservices.core.validation.ValidationException;
import org.wso2.carbon.dataservices.core.engine.ParamValue;

public class MyCustomValidator implements Validator {
public void validate(ValidationContext validationContext, String s, ParamValue paramValue)
throws ValidationException {
if (!paramValue.getScalarValue().startsWith("2")) {
throw new ValidationException("Not starting with 2!!",s,paramValue);
}


}
}

Then, you can build a jar with the custom validator class and place it in CARBON_HOME/repository/components/lib directory. After restarting the server, you can use it inside query definition as follows.

<param name="id" paramType="SCALAR" sqlType="STRING" type="IN" ordinal="1">
<validateCustom class="org.test.MyCustomValidator" />
</param>

Monday, September 13, 2010

Creating value with testing

Jonathan Kohl discusses about creating value with software testing. You will find very important set of points which we must follow in daily QA/testing activities.

Is my testing work defensible? (Cem Kaner talks a lot about this.) Think of a court case. What would a jury think if you testified and described what you did as a tester and why. How did you determine priority? Why did you test some things and not test others? (100% complete testing is impossible, so you have to make decisions to optimize your work. Are those decisions well thought out, or more subconscious? What sorts of things might you be missing that you haven't thought of?)

Read the full article from here


Sunday, September 12, 2010

How to start multiple WSO2 Carbon server instances as windows services

We can start more than one WSO2 Carbon (WSO2 ESB, WSAS, G-reg, GS, Mashup, IS, BPS, BRS, BAM, DS) instance by updating the transport configuration given in mgt-transport.xml (or transports.xml in 2.X series) as explained in here.
Suppose, you need to start multiple carbon server instances as windows services instead of regular wso2server.sh executable.
Then, there is an additional setting which has to be configured.

  • Find wrapper.conf file inside CARBON_HOME/repository/conf

  • Locate "Wrapper Windows NT/2000/XP Service Properties" section

  • Update the Name of the service and Display name of the service in each instance as follows

# Name of the service
wrapper.ntservice.name=WSO2Carbon

# Display name of the service
wrapper.ntservice.displayname=WSO2 Carbon

Wednesday, August 18, 2010

Process vs Tools and Technologies - What should Sri lankan QA community be concerned with?

I have been thinking about discussing the matters related to local QA community in Sri lanka but never got a chance. Recently I was able to meet a lot of folks who are engaged in software quality assurance in various sri lankan organizations in one place at a quality summit. By listening to the presentations and talking with people, I came out with a few basic questions.

Some of them were;
what the biggest concern of software quality assurance in our country? are those processes or tools/technologies? Do we have the necessary trainings or knowledge sharing mechanism to overcome the issues which we face during daily QA tasks?

In my view, the biggest concern of QA in our country is, not having people with enough technical skills. By interviewing a lot of QA folks for the past few years, I personally have a good experience about the way people are approaching QA. Most fresh graduates believe that QA as a first step towards entering in to software industry! Some people joins QA merely to get some understanding about the product/project then move in to business analysis or sales.
Why is this? IMO, it is totally due to the perception of software QA in sri lanka. We, sri lankan QA community, must be responsible for drawing that image on people's minds about QA.
For the past few years, I never noticed any training (were there any?) for educating QA community about the usage of tools in daily QA tasks to be more productive or technical aspects such as performance/automation testing tools. Instead, whenever there is something about QA, it is about CMMI or process frameworks.
I'm not going to say that those are not important. BUT those are not what our teams need at the moment.
People struggle with configuring application server X on operating system Z. QA folks face in to difficulties when automating AJAX based UIs. How QA should be dealt with the frequent UI changes during UI based test automation? How can we be more productive using linux? Do we use any scripting language for automate repetitive configuration tasks? Are we doing continuous integration? Are QA people familiar with build tools such as Maven or Ant? Do we know about exploratory testing? Do we know how to use test coverage tools? Do we report bugs with the adequate logs and find the root cause of them?

I think these are the questions that most of the QA teams have. We should try to be more productive and be experts as a community. We should try to change the perceptive about QA by empowering everyone with the right set of skills.

If people are comfortable with the tools and technology they are handling in daily work life, educating them about processes and process improvements is not a big thing!

Saturday, August 14, 2010

How to deploy WSO2 ESB-3.X on Apache Tomcat

WSO2 DOES NOT ENCOURAGE INSTALLING WSO2 ESB ON TOP OF OTHER APPLICATION SERVERS. WSO2 HAS DECIDED TO DROP SUPPORT FOR WEBAPP DEPLOYMENT MODE OF THE WSO2 PLATFORM AND PRODUCTS.





I have noticed a lot of different articles, blog posts with instructions on deploying WSO2 ESB on Apache tomcat. But I observed that most of them are obsolete and the guidelines are not applicable for the latest ESB versions. Therefore, I thought to put together the steps of setting up WSO2 ESB-3.X versions on Apache Tomcat-6.X

Step 1:
Download WSO2 ESB-3.X. Extract the downloaded zip and copy repository and resources directories in to a new folder. Say it is esb-repo (i.e:- /home/user/esb-repo)

Step 2:
Lets refer to your tomcat installation directory, CATALINA_HOME. Go to CATALINA_HOME\webapps directory and create a new directory, esb.
Now, copy wso2esb-3.0.0\webapps\ROOT\WEB-INF to CATALINA_HOME\webapps\esb
Also, copy wso2esb-3.0.0\lib\log4.properties file to CATALINA_HOME\webapps\esb\WEB-INF\classes

Step 3:
Next, we need to enable https in tomcat. Therefore, edit CATALINA_HOME\conf\server.xml by adding the following entry.

<Connector port="8443" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" disableUploadTimeout="true"
acceptCount="100" scheme="https" secure="true" SSLEnabled="true"
clientAuth="false" sslProtocol="TLS"
keystoreFile = "/home/user/esb-repo/resources/security/wso2carbon.jks"
keystorePass="wso2carbon"/>

Make sure to give the exact location of wso2carbon.jks as highlighted above.

Step 4
We have done the configurations required in tomcat. Now, we must do the necessary configurations in a set of config files shipped with WSO2 ESB. We will update carbon.xml, axis2.xml, registry.xml and user-mgt.xml which can be found at esb-repo/repository/conf directory.

First, open carbon.xml and update the ServerURL element as follows.

<ServerURL>https://localhost:8443/esb/services/</ServerURL>

Note that we have configured tomcat to run on 8443 port.

When we deploy ESB on an application server, it uses the http/https transport provided by the servlet container when communicating with registry. Therefore, we must update the registry HTTP port in carbon.xml. In order to do that, uncomment and update the following element in carbon.xml.

<RegistryHttpPort>8080</RegistryHttpPort>

Thats all we need to update in carbon.xml, save and close the file.

Next, open registry.xml and update DB URL as follows.

<url>jdbc:h2:/home/user/esb-repo/repository/database/WSO2CARBON_DB</url>

Now, open user-mgt.xml and update user management database URL as follows.

<Property name='url'>jdbc:h2:/home/user/esb-repo/repository/database/WSO2CARBON_DB</Property>

Make sure to specify the absolute path of the WSO2CARBON_DB in both of the above elements.

Next, we need to configure a few elements in esb-repo/repository/conf/axis2.xml file.

Locate NIO HTTPS transport listener (HttpCoreNIOSSLListener) element and specify the absolute path of keystore and truststore locations as follows.

<KeyStore>
<Location>/home/user/esb-repo/resources/security/wso2carbon.jks</Location>

<TrustStore>
<Location>/home/user/esb-repo/resources/security/client-truststore.jks</Location>

Similarly update the keystore and trustore paths of NIO HTTPS transport sender (HttpCoreNIOSSLSender)

We should also specify the absolute path of synapse-config directory as follows.

<parameter name='SynapseConfig.ConfigurationFile' locked='false'>/home/charitha/products/esb/esb-repo/repository/conf/synapse-config</parameter>

Step 5
We have almost completed the required configurations. Now, open a new shell and change the directory to CATALINA_HOME/bin.
Define an environment variable called CARBON_HOME and set the path to your esb-repo directory.

In windows; set CARBON_HOME=C:\esbs\esb-repo
In linux; export CARBON_HOME=\home\user\esb-repo

Start tomcat from the same command window/shell.
catalina.sh run

WSO2 ESB will be started successfully. You can access the management console using https:\\localhost:8443\esb\carbon

Thats all! You could follow the above steps and deploy WSO2 ESB-3.X on Tomcat successfully. If you do not like to follow each of the above steps manually, I have written a ruby script to automate the above procedure and install ESB on tomcat. You can download it from here
You just need to specify three directory paths there in esb3.X_tomcat_install_linux.rb and rest of the installation steps will be done automatically by the script.
esb_repo = Any directory in the local file system
CARBON_BIN_HOME= Home directory of ESB binary distribution
CATALINA_HOME= Home directory of the tomcat binary

Wednesday, July 28, 2010

Message format transformations with WSO2 ESB

With WSO2 ESB, you can easily convert the format of the messages which passes through. There are situations in which you get SOAP requests but the back end server accepts XML. In these situations, you should convert the SOAP request to XML (POX) before forwarding to the endpoint. This post explains how you can change the format of a SOAP request goes through WSO2 ESB.

Pre-requisites:
Download and install WSO2 ESB-3.X

Step 1
I use WSO2 WSAS as the backend, but you could use any server as you preferred. Start WSO2 WSAS and deploy Axis2Service

Step 2
Start WSO2 ESB and add the following configuration. Here, we specify the message format as POX in the endpoint configuration since we need to convert SOAP message to XML when forwarding it to the endpoint.

<sequence xmlns="http://ws.apache.org/ns/synapse" name="main">
<in>
<send>
<endpoint name="axis2service-epr">
<address uri="http://localhost:9763/services/Axis2Service" format="pox" />
</endpoint>
</send>
<</in>
<out>
<send />
</out>
</sequence>

Step 3
Send a SOAP1.1 message using a tool (SOAPUI, Jmeter, AB etc). If you look at the message transmission between ESB and the endpoint, you will notice the following messages

Request SOAP message:

POST / HTTP/1.1
SOAPAction: urn:echoString
Content-Length: 307
Content-Type: text/xml; charset=UTF-8
Host: 127.0.0.1:8281
Connection: Keep-Alive
User-Agent: Jakarta-HttpComponents-Bench/1.1

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://service.carbon.wso2.org">
<soapenv:Body>
<ser:echoString>
<ser:s>test</ser:s>
</ser:echoString>
</soapenv:Body> </soapenv:Envelope>


Message delivered to the endpoint:

POST /services/Axis2Service HTTP/1.1
Content-Type: application/xml; charset=UTF-8
SOAPAction: urn:echoString
Transfer-Encoding: chunked
Host: 127.0.0.1:9765
Connection: Keep-Alive
User-Agent: Synapse-HttpComponents-NIO

86
<ser:echoString xmlns:ser="http://service.carbon.wso2.org">
<ser:s>test</ser:s> </ser:echoString>0

Similarly, you can convert the message to SOAP1.2 or HTTP GET just by specifying the endpoint format.