Working with HTTP multipart requests in soapUI

You can use HTTP request test step in soapUI to submit messages with various Content-Types.  In this post, we will have a quick look into the multipart/form-data requests in soapUI.

multipart/form-data requests usually come into action when you do HTML form submissions with file attachments. For example, have a look at the following HTML form post.

<form action="http://localhost:8090/CKFileUploadApp/UploadServlet" method="post" enctype="multipart/form-data">
    <input type="file" name="file">
    <input type="text" name="paramName">
    <input type="submit" name="Submit" value="Upload File">
</form>

In the forms like above, we have file upload option as a form input where we can upload a file along with the form submission. In that case, the request's HTTP content type will be multipart/form-data. How can we simulate such a request using soapUI?

Pre-requisite:
You can have your own file upload servlet as the backend. But in order to demonstrate the scenario, I have hosted a sample webapp, CKFileUploadApp.war from here. Make sure to change the following context parameter value in WEB-INF/web.xml directory before deploying the web application in Apache tomcat.

<param-name>file-upload</param-name> 
    <param-value>/home/charitha/</param-value> 

Step 1

Start to create a new soapUI project. Specify a name and select Create Web TestCase option.

Add Web TestCase dialog will be displayed. Enter http://localhost:8080/CKFileUploadApp/UploadServlet as the Web Address. Clear Start Recoding Immediately option and click on OK.

Step 2

Once the new HTTP request is added as explained in the first step, change the HTTP method to POST. The request editor should look like the following.

Step 3

Choose multipart/form-data from the Media Type drop down. Now, click on Attachments tab at the bottom of the request editor (see below).

 

Click on + icon at the top left corner of the attachment window to browse and attach a file to request. Browse for a file in your local file system and add it as an attachment. Now, the attachment editor will be similar to the following.

Step 4

We have everything ready to send the request with a file attachment. Click on submit button to send the request. You will get a HTTP 200 response with the message "Uploaded Filename: <filename>" message. Click on the raw view of the request. You will see the Content-Type of the request is set to multipart/form-data. You will also notice the relevant MIME boundaries of the request.

Since I have attached an xml file to the request, you can observe the Content-Type of the attachment part of the message as text/xml.


------=_Part_17_31084605.1374474723633
Content-Type: text/xml; charset=UTF-8; name=registry.xml

Broker trust relationships with WSO2 Identity Server

WS-Trust can be considered as an extension to WS-Security specification which primarily provides methods for managing security tokens and ways to broker the trust relationships. The web services trust model explained under the WS-trust specification defines three key participants.

• Security token service

• Service Consumer

• Service provider (Relying party)

Security token service (STS) is a web service that issues security tokens based on requesters needs. The consumer sends token requests to the STS as well as append tokens into the actual service request and submits them to the provider. The service provider makes the authentication decison on the service based on the token provided by STS. The service provider may also request token validations from the STS.

This post is not yet another detailed explanation about WS-trust and the associated frameworks. You can find many references just by googling the term, ws-trust. Instead, I'm going to take you through a set of step-by-step instructions on how WS-trust concepts can be used with WSO2 SOA middleware. Hope this post will be a starting point for your journey towards learning WS-trust.

Use Case

We are going to work on the basic trust establishment use case in the context of WS-trust specifiction. A SOAP web service consumer requests security token from a STS. We will make use of WSO2 Identity Server as the token provider. WSO2 Identity Server includes a Security Token Service (STS) which is capable of generating security tokens to build trust relationships. 

Once the consumer possesses the necesary security tokens, he presents those tokens to authenticate to a web service deployed in WSO2 Application Server.

Pre-requisites:

1. Download and install WSO2 Identity Server 4.1.0 or later version

2. Download and install WSO2 Application Server 5.0.1 or later version

3. Any Java IDE (Eclipse, Idea)

4. soapUI (optional)

First, we need to figure out how we can start the end-to-end process. Obviously, the consumer initiates the process. But, how does the consumer know that he first needs to talk to a STS in order to consume the web service which is deployed in WSO2 application server? 

The service provider needs to advertise those requirements through a WS-Policy. So, the starting point of our use case is to configure a policy in the web service. 

Then, we need to look into the token provider, WSO2 Identity Server. As I have explained before, WSO2 IS comes with a Security Token Service. We need to configure it to issue tokens to the provider web service and secure STS using a WS-Security policy because the consumers should authenticate themselves to the STS when requesting tokens.

Finally, we will write a client/consumer (or just use a service invocation tool such as soapUI) to request a security token from STS and append it to the actual web service request message to authenticate to the web service which is deployed in WSO2 Application Server.

Step 1

Start WSO2 Application Server and log in to the management console as default admin user. Go to Manage --> Services  --> Add  --> AAR Service and deploy Axis2Service.aar which can be downloaded from here.

Once the service is deployed, we need to associate a security policy for our service. You will find a set of default web service security policies in security configuration wizard. However, we cannot use any of them for our use case. Why?

Our web service should be associated with a trust based security policy. A Security token  which is offered by STS represents a set of claims. A claim defines a specific information about a particular user. For example, firstname or email address of the user.  

In our example, Axis2Service (the web service deployed in service provider, wso2 application server) needs to be made as a claim-aware web service. So that, the users who are going to consume the service should present the claims defined in WS-Security policy.

Thus, we need to secure Axis2Service using a custom policy. You can download the complete policy from here. You might specially be interested in looking into the following element in the custom security policy.

 <sp:SupportingTokens xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
                <wsp:Policy>
                    <sp:IssuedToken
                            sp:IncludeToken="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy/IncludeToken/AlwaysToRecipient">
                        <sp:Issuer>
                            <Address xmlns="http://www.w3.org/2005/08/addressing">https://localhost:9444/services/wso2carbon-sts</Address>
                        </sp:Issuer>
                        <sp:RequestSecurityTokenTemplate>
                            <t:TokenType xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">urn:oasis:names:tc:SAML:2.0:assertion</t:TokenType>
                            <t:KeyType xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">http://schemas.xmlsoap.org/ws/2005/02/trust/Bearer</t:KeyType>
                            <t:KeySize xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">256</t:KeySize>
                            <t:Claims Dialect="http://wso2.org/claims"
                                      xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust"
                                      xmlns:ic="http://schemas.xmlsoap.org/ws/2005/05/identity">
                                <ic:ClaimType Uri="http://wso2.org/claims/givenname"/>
                                <ic:ClaimType Uri="http://wso2.org/claims/emailaddress"/>
                            </t:Claims>
                        </sp:RequestSecurityTokenTemplate>
                        <wsp:Policy>
                            <sp:RequireInternalReference/>
                        </wsp:Policy>
                    </sp:IssuedToken>
                </wsp:Policy>
            </sp:SupportingTokens>

SupportingTokens are used to provide additional claims for a requester (client). We will use SupportingToken policy assertion to describe claims which should be given by a consumer to access this particular web service.

The sp:IssuedToken element defines that the requester must talk to an STS to obtain the token.

The sp:IncludeToken attribute value is defined as ../AlwaysToRecipient, which says the token must be included as part of all messages sent from the initiator (service consumer) to the recipient.

The Address attribute of sp:Issuer element defines the token issuer whom the consumer must be contacted to get the security token. In other words, the endpoint address of the Security Token Service. In our case, it must be the STS url of IS.
[Note: I'm running WSO2 IS with port-offset 1, hence the https port is 9444]

The sp:RequestSecurityTokenTemplate element specifies the structure/type of the token which has to be issued by the STS. Later, you will observe the children of this element will be included as the part of RequestSecurityToken (RST) message which will be sent to the STS by consumer. 

The t:TokenType element represents the type of the token such as UserNameToken, SAML-1.1, SAML-2.0. In our example, we ask STS To generate SAML-2.0 token by specifying SAML-2.0 token type URL.

t:KeyType represents the type of the key desired in security token, can be either public key, symmetric key or bearer. We will ask STS to generate the relatively simple key, bearer token. Bearer token does not require a proof of possession hence it is quite easy to dealt with.

t:Claims element defines the claims that must be included in the security token. We will request First Name and email address claims from the consumer.

You may raise the question why the First Name claim has been mapped to "http://wso2.org/claims/givenname" URI. We will be able to clear ourselves once we configure the token provider in the next step.

OK, now we understood all elements of the SupportingToken assertion of our custom web service policy. Let's upload the policy to embedded registry in WSO2 Application Service.

Locate /_system/config collection in registry browser and click on Add Resource.
Then, browse the axis2service.policy.xml which you have downloaded from the above location and click on Add.

Now, we have our custom policy in WSO2 Application Server. We can associate the policy to Axis2Service.

Click on Unsecured link in Axis2Service in Deployed Services page. You will be directed to Security for the service page. Find Policy From Registry section which is at the bottom of the page. Click on Configuration Registry icon and select the axis2service.policy.xml from /_system/config collection.
Click on next to proceed with the wizard. You will be at the Activate Security page where you can specify a trusted key store for the web service.
We will select wso2carbon.jks as the trusted key store. We have already discussed that STS is responsible for issuing tokens. Therefore, the STS is trusted by both consumer as well as the service provider. In order to do so, the public key certificate of STS should be imported to the trusted key store defined here.  We will not explicitly do key exchanges in this example because we are using the same wso2carbon.jks keystore in both WSO2 Application Server and WSO2 Identity Server.

Now, we have our web service secured with a claim-aware security policy. Let's move in to the STS configurations.

Step 2

Start WSO2 Identity Server and log in to the management console using the default admin credentials. Select Secure Token Service from the left menu under the Manage section.

First, we should secure the STS. There are multiple ways to authenticate to the Secure Token Service. The consumer can just present his username and password as UserNameTokens or X509 certificates to the STS. In this example, we will configure the STS using UserNameToken Authentication policy so that the clients should talk to STS by presenting their credentials.

Click on Apply Security Policy in the above screen. Select UsernameToken from the list of default security policies in the Security for the service page. Click on Next.
Now, we need to select a user group to which the service consumer belongs. We will incorporate our consumer into the admin group and request tokens from STS by presenting the default admin user credentials. Therefore, select admin as the user group and click on finish.

Next, we need to configure STS to add Axis2Service as a trusted service. In the above screen, enter the http endpoint url of the Axis2Service.

Endpoint Address = http://localhost:9765/services/Axis2Service/

Usually, the security token is signed by STS. Thus, we need to select a certificate alias to sign the token. We will select the default wso2carbon certificate alias.

Now, let's look at the user profile of admin user who is going to authenticate to the STS. Click on My Profiles at the left menu. Update Profile form will be displayed where you can enter various user attributes such as First Name, Last Name etc.
Make sure to add some values to First Name and Email address fields since we are going to use those as the required claims.

Click on Configure in the left menu and select Claim Management. You will find a set of claim dialects associated with the internal user store in IS. Click on the default claim dialect, http://wso2.org/claims. Click on First Name claim mapping.
As you can see in the following screen shot, First Name is mapped to givenName attribute.

Now, you should be able to understand why we have specified http://wso2.org/claims/givenName as the claimuri for First Name attribute in the service policy.

Step 3

Now, all what left is to work on the service consumer. Basically, we need to generate the RequestSecurityToken programatically using a client, insert into the web service request and send to Axis2Service.
The wst:RequestSecurityToken element is used to request a security token from STS. 

It will be a child of SOAP body. At the minumum level, the RequestSecurityToken element will be similar to the following. 

<wst:RequestSecurityToken xmlns:wst="http://schemas.xmlsoap.org/ws/2005/02/trust">
            <wst:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</wst:RequestType>

            <wst:TokenType>http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0
            </wst:TokenType>
            <wst:KeyType>http://schemas.xmlsoap.org/ws/2005/02/trust/Bearer</wst:KeyType>
            <wst:Claims xmlns:wsp="http://schemas.xmlsoap.org/ws/2005/02/trust"
                        wsp:Dialect="http://wso2.org">
                <wsid:ClaimType xmlns:wsid="http://schemas.xmlsoap.org/ws/2005/05/identity"
                                Uri="http://wso2.org/claims/givenname"/>
                <wsid:ClaimType xmlns:wsid="http://schemas.xmlsoap.org/ws/2005/05/identity"
                                Uri="http://wso2.org/claims/emailaddress"/>
            </wst:Claims>
        </wst:RequestSecurityToken>

You would like to compare this with the SupportingToken policy assertion which we defined in step 1. In this, we will request a SAML2 token from STS (see wst:TokenType element) and indicate that the keytype is bearer. We also define the two claims we would present to the web service, givenname and emailaddress.

You can include RequestSecurityToken element into a SOAP message body and send to STS. You can use soapUI SOAP request editor as shown below.

Since we are using usernametoken authentication when submitting token request to STS, specify username and password under "Aut" tab in soapUI. Also specify WSS-passwordType as plaintext and WSS timeto live to some integer value. You can find more information about securing requests with username token with soapUI in one of my blog posts. Also, make sure to enable WS-Addressing for the token request as explained in another blog post of mine.
With all these, when you submit the request to STS endpoint (in our case, https://localhost:9444/services/wso2carbon-sts), you will get a response with the generated token (RequestSecurityTokenResponse). You will notice it in the response view of the above screen shot.
Now, you can extract the saml2:Assertion element from the response and embed it with the actual web service request message. For that, you can try adding SAML WSS entry for soapUI request and copy the extracted saml2:Assertion element into the Enter SAML Assertion text area as shown in the following screen shot. However, I was unsuccessful sending a message with bearer confirmation method with soapUI-4.5.2. It seems soapUI still supports sender vouches confirmation method only.

Because of this limitation, we need to follow a programmatic approach to insert the token into the web service request and forward to Axis2Service.
You can find the complete working client in this sourceforge account

Steps to run the client

1. Check out the complete source from https://sourceforge.net/p/charithablogsam/code/ci/master/tree/  and import the source into your IDE
2. Go to IS_HOME/bin and type ant. This will copy the necessary client libraries in to IS_HOME/repository/lib directory
3. Add IS_HOME/repository/lib directory to your IDE class path
4. Also add IS_HOME/repository/components/lib/bcprov-jdk15-132.jar to the class path
5. Modify Axis2ServiceClient according to the file paths in your system wherever necessary.
e.g:-
Policy stsPolicy = loadSTSPolicy("/resources/policies/sts.ut.policy.xml");
6. Run the client


You should see the output similar to the following.

<saml2:Assertion xmlns:saml2="urn:oasis:names:tc:SAML:2.0:assertion" xmlns:xs="http://www.w3.org/2001/XMLSchema" ID="urn:uuid:D6511FA799FD47AC4D1373081003469" IssueInstant="2013-07-06T03:23:23.465Z" Version="2.0"><saml2:Issuer>localhost</saml2:Issuer><ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
<ds:SignedInfo>
<ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"></ds:CanonicalizationMethod>
<ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"></ds:SignatureMethod>
<ds:Reference URI="#urn:uuid:D6511FA799FD47AC4D1373081003469">
<ds:Transforms>
<ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"></ds:Transform>
<ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"><ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="xs"></ec:InclusiveNamespaces></ds:Transform>
</ds:Transforms>
<ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"></ds:DigestMethod>
<ds:DigestValue>MO6PnDsz+lbXsvuFRmMBwa7r0j4=</ds:DigestValue>
</ds:Reference>
</ds:SignedInfo>
<ds:SignatureValue>
i8S8BFxLJ59DpYUF8s8/Glt8x0n2plGPgCJKigB6eopRt7Y52LLnSqRimkWTWx57wdwjOFENMTPa
ZsYgBJ3AlxKRDQoy23OBOjrRA+S0WQ4Pq3EGcmM5XGKxU9pTNnh/xEhT4lDN9QE12Z1rttFz6RkU
AgFt3nmvqNqZvbNMga4=
</ds:SignatureValue>
<ds:KeyInfo><ds:X509Data><ds:X509Certificate>MIICNTCCAZ6gAwIBAgIES343gjANBgkqhkiG9w0BAQUFADBVMQswCQYDVQQGEwJVUzELMAkGA1UE
CAwCQ0ExFjAUBgNVBAcMDU1vdW50YWluIFZpZXcxDTALBgNVBAoMBFdTTzIxEjAQBgNVBAMMCWxv
Y2FsaG9zdDAeFw0xMDAyMTkwNzAyMjZaFw0zNTAyMTMwNzAyMjZaMFUxCzAJBgNVBAYTAlVTMQsw
CQYDVQQIDAJDQTEWMBQGA1UEBwwNTW91bnRhaW4gVmlldzENMAsGA1UECgwEV1NPMjESMBAGA1UE
AwwJbG9jYWxob3N0MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCUp/oV1vWc8/TkQSiAvTou
sMzOM4asB2iltr2QKozni5aVFu818MpOLZIr8LMnTzWllJvvaA5RAAdpbECb+48FjbBe0hseUdN5
HpwvnH/DW8ZccGvk53I6Orq7hLCv1ZHtuOCokghz/ATrhyPq+QktMfXnRS4HrKGJTzxaCcU7OQID
AQABoxIwEDAOBgNVHQ8BAf8EBAMCBPAwDQYJKoZIhvcNAQEFBQADgYEAW5wPR7cr1LAdq+IrR44i
QlRG5ITCZXY9hI0PygLP2rHANh+PYfTmxbuOnykNGyhM6FjFLbW2uZHQTY1jMrPprjOrmyK5sjJR
O4d1DeGHT/YnIjs9JogRKv4XHECwLtIVdAbIdWHEtVZJyMSktcyysFcvuhPQK8Qc/E/Wq8uHSCo=</ds:X509Certificate></ds:X509Data></ds:KeyInfo></ds:Signature><saml2:Subject><saml2:NameID Format="urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress">admin</saml2:NameID><saml2:SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer"></saml2:SubjectConfirmation></saml2:Subject><saml2:Conditions NotBefore="2013-07-06T03:23:23.465Z" NotOnOrAfter="2013-07-06T03:28:23.465Z"></saml2:Conditions><saml2:AuthnStatement AuthnInstant="2013-07-06T03:23:23.465Z"><saml2:AuthnContext><saml2:AuthnContextClassRef>urn:oasis:names:tc:SAML:2.0:ac:classes:Password</saml2:AuthnContextClassRef></saml2:AuthnContext></saml2:AuthnStatement><saml2:AttributeStatement><saml2:Attribute Name="http://wso2.org/claims/emailaddress" NameFormat="http://wso2.org/claims/emailaddress"><saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">charitha@wso2.com</saml2:AttributeValue></saml2:Attribute><saml2:Attribute Name="http://wso2.org/claims/givenname" NameFormat="http://wso2.org/claims/givenname"><saml2:AttributeValue xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">charitha</saml2:AttributeValue></saml2:Attribute></saml2:AttributeStatement></saml2:Assertion>
Response  : <ns:echoStringResponse xmlns:ns="http://service.carbon.wso2.org"><ns:return>Hello world1</ns:return></ns:echoStringResponse>

Comment on or contact me directly at uoccharitha@gmail.com if this does not work for you!

Invoking WSO2 Carbon admin services with soapUI

The management aspects of WSO2 Carbon platform are primarily achieved through SOAP web services interface known as admin services. All Carbon products ship with a management console (front-end user interface) which communicates with these web services and provides users with various administration capabilities.

In some situations, we need to by-pass the management UI and call the backend web services directly. Specially, in test automation, it is important to minimize the risk of frequent UI changes hence focusing on admin service interactions can be considered as a viable solution. WSO2 test automation framework is built upon this approach which programatically calls the backend web services to manage deployment, configuration and various other tasks.

These backend web services are secured to prevent anonymous invocations. WSO2 Carbon server secures these services through multiple methodologies. For example;

• HTTP Basic Authentication over SSL

• WS-Security username token

• Session based authentication

You can use any SOAP client and communicates with the admin services by authenticating through above security protocols.

In this post, I will take  you through consuming an admin service using soapUI since soapUI is the most user-friendly service testing tool out there to test SOAP or RESTful web services.

We will use HTTP basic auth authentication mechanism out of the auth options described above. If you like to use a different approach such as carbon session based authentication, you may refer to Nandika's blog post.

Pre-requisite:
soapUI 4.5.1 or later
WSO2 Carbon 4.X version (You can use any member product of WSO2 Carbon family)

Step 1

By default, the WSDLs of admin web services are hidden from consumers. Therefore, we need to enable them first to import admin service WSDL into a soapUI project.

Open CARBON_HOME/repository/conf/carbon.xml and set HideAdminServiceWSDLs property to false.

<HideAdminServiceWSDLs>false</HideAdminServiceWSDLs> 

Start WSO2 Carbon server.

Step 2

 

There are large number of admin services which serve many administration and management functionalities such as ServiceAdmin, StatisticsAdmin, ProxyAdmin etc.. However, I will use a simplest admin service, UserAdmin for this example.

Open  https://localhost:9443/services/UserAdmin?wsdl in your browser and check whether it is accessible.

Step 3

Create a SOAP web service project in soapUI using the above WSDL

Step 4

We are going to invoke one of the web services operations exposed through UserAdmin web service. Select listAllUsers operation under UserAdminSoap11Binding interface and click on SOAP request.

Step 5

We are going to submit listAllUsers SOAP request using HTTP basic authentication headers. Therefore, click on Aut tab under the bottom of the request editor and specify the admin  user name and password of carbon server.

Enter the following values for the required parameters of the SOAP request.

<xsd:filter>*</xsd:filter>
 <xsd:limit>100</xsd:limit>

Step 6

Now, we can submit the request to UserAdmin web service.You will get a SOAP response with all users available in the user store of Carbon server.

The simplest possible way to simulate a backend service delay

When testing service integrations, we usually want to simulate the delayed responses from backend web services. For example, if you want to test how WSO2 ESB or Apache Synapse reacts when the backend service takes large amount of time for responding, there are many approaches to introduce delay to backend web service. I have observed that, most of the people modify the backend web services to add Thread.sleep() when they need to introduce a delay in service invocation.

If you are not testing the backend web service and just want to test the integration (e.g:- outgoing calls from ESB), I cannot think of a better solution than using soapUI mock services.

Step 1

 

Add a new mock service to soapUI project. This can be done at the time of creating the project or by selecting a particular interface (binding) of a wsdl based project.

Step 2

 

Select the mock service in soapUI navigator and open the mock service editor. In the mock service editor, select OnRequest script.

Step 3

 

Add the following groovy script inside the OnRequest Script editor to introduce 1 minute delay.

sleep(60000)

Now, send a request to the mock service. You will notice that all requests will respond back to the caller after 1 minute of delay.

Read chapter 6 (Web service simulation with soapUI) and 11 (Extending soapUI with Scripting) of Web Services Testing with soapUI book for more information about soapUI scripting capabilities.

Testing one-way operations which do not return HTTP 202 responses

When you invoke a one-way (in-Only) operation of a web service over HTTP, it responds with HTTP 202 accepted message. Many web service clients such as soapUI or Jmeter waits till they receive a response from the web service.
Waiting for HTTP 202 response is always not desirable since there are situations where you do not even get a 202 response. For example, if you invoke one-way JMS operation, it does not send a reply back to the client.
Look at a scenario similar to the following.

A client sends a message over HTTP to a proxy service in WSO2 ESB. The proxy service places the message in a JMS queue and does not expect a response back. In this case, client does not even get a HTTP 202 response hence it waits and eventually timed out. This prevents you using the tools like soapUI, Apache Jmeter in these scenarios. How can we fix this so that the client always get a HTTP 202 response back?

Let's go through the procedure in detail.

Step 1:

Configure Apache ActiveMQ JMS broker with WSO2 ESB as explained here.

Step 2: 

Create a queue in ActiveMQ. You could access ActiveMQ console application through http://localhost:8161/admin and create a new queue. Name it as "onewaytest".

Step 3: 

Create a simple proxy service in WSO2 ESB as shown below. This proxy service just forwards the incoming SOAP messages into the "onewaytest" JMS queue which we have created in the previous step.

<proxy xmlns="http://ws.apache.org/ns/synapse" name="onewayProxy" transports="http" statistics="disable" trace="disable" startOnLoad="true">
   <target>
      <inSequence>
         <property name="OUT_ONLY" value="true"/>
         <send>
            <endpoint>
               <address uri="jms:/onewaytest?transport.jms.ConnectionFactoryJNDIName=QueueConnectionFactory&java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory&java.naming.provider.url=tcp://localhost:61616"/>
            </endpoint>
         </send>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <description></description>
</proxy>
                                

Step 4:

Use soapUI (or SOAP/XML-RPC sampler in Apache Jmeter) and send a SOAP request to the above proxy service. You will get "java.net.SocketTimeoutException: Read timed out" in soapUI log. This explains the behavior of HTTP client waiting for HTTP 202 response.

Step 5:

There is an useful property, FORCE_SC_ACCEPTED which can be used inside the inSequence of the proxy service to send HTTP 202 Accepted response back to the client in case of one-way JMS operations.

Add this property to the inSequence of the above proxy service.

<property name="FORCE_SC_ACCEPTED" value="true" scope="axis2" />

Step 6:

Re-send a SOAP message to the proxy service using soapUI. You will get a HTTP 202 response.


HTTP/1.1 202 Accepted
Content-Type: text/xml; charset=UTF-8
Date: Sun, 05 May 2013 07:42:27 GMT
Server: WSO2-PassThrough-HTTP
Transfer-Encoding: chunked

How to configure soapUI to send HTTP chunked encoded requests

soapUI makes use of Apache HttpClient (based on HTTPComponents project) as the client side HTTP implementation. When submitting requests using soapUI, you may have noticed that, the requests always send Content-Length HTTP header. You can find it out in the Raw View of the request editor.

POST http://localhost:9000/services/SimpleStockQuoteService HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:getQuote"
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:9000
Content-Length: 416

In functional and non-functional testing, we usually need to test the servers by sending HTTP requests with both Content-Length as well as chunked transfer encoding. How can we configure soapUI to send chunked encoded requests instead of Content-Length?

Step 1

Click on Global soapUI Preferences icon in the main tool bar (Or else, click on File --> Preferences)

Step 2

HTTP Settings tab will be selected by default. You will notice Chunking Threshold option. By default this is set to empty so that the chunking is disabled for any HTTP request.

Update this field with non negative integer value (e.g:- 1) and resubmit your request. You will notice chunked encoded request as shown below.

POST http://localhost:9000/services/SimpleStockQuoteService HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:getQuote"
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:9000
Transfer-Encoding: chunked

For more information about various soapUI tips and tricks, read Web services Testing with soapUI book.

JSON pass-through in WSO2 ESB

JSON is a lightweight data exchange format used in many web applications. WSO2 ESB supports sending and receiving JSON messages out-of-the-box. Exposing a SOAP web service over JSON is explained under sample 440 in WSO2 ESB official documentation. This post is to summarize steps of invoking a pass-through proxy service which accepts JSON request, forwards it to a RESTful service and reply back with JSON response (JSON-in, JSON-out).

Pre-requisite:

• Download and extract the binary distribution of WSO2 ESB-4.5.1
• I will use WSO2 Application Server-5.0.1 to deploy RESTful backend web service. You can use any server as the backend. However, in order to follow the steps mentioned in this post, you should use WSO2 Application server.

Step 1:

First, we need to setup the backend web service.  We will deploy a JAX-RS based restful service which consumes and produces JSON messages. You can download this RESTful web service(jaxrs_basic.war) from here.

Step 2:

Start WSO2 Application Server, log in to management console and navigate to Manage --> Applications --> List.
You will notice the web applications which are deployed by default in WSO2 Application Server. Since we already have a JAX-RS web app with the name jaxrs_basic, delete the default jaxrs_basic web application.

Now, we can deploy our jaxrs_basic web application (Note that, jaxrs_basic is a modified version of default sample jax-rs web application which has been customized to consume and produce JSON messages).

Go to  Manage --> Applications -->Add --> JAX-WS/JAX-RS.
Upload JAX-WS/JAX-RS Applications page will be shown. Browse the downloaded jaxrs_basic.war and click on upload. New JAX-RS web service will be deployed in WSO2 Application Server.

Step 3:

The auto-generated WADL of the RESTful web service can be obtained by accessing http://<wso2_application_server_hostname>:<wso2_application_server_port>/jaxrs_basic/services/customers?_wadl
From there, you will identify the POST URL of 'customer' resource;

http://<wso2_application_server_hostname>:<wso2_application_server_port>/jaxrs_basic/services/customers/customerservice/customers

We will forward a message to the above URL from WSO2 ESB.

Step 4:

Now we have the backend web service, ready to process messages. Let's create a pass-through proxy service in ESB. Obviously, the endpoint URL will be the one mentioned above.

<proxy xmlns="http://ws.apache.org/ns/synapse" name="jsonproxy" transports="https,http" statistics="disable" trace="disable" startOnLoad="true">
   <target>
      <outSequence>
         <send/>
      </outSequence>
      <endpoint>
         <address uri="http://localhost:9764/jaxrs_basic/services/customers/customerservice/customers"/>
      </endpoint>
   </target>
   <description></description>
</proxy>

Either using Pass Through Proxy template in WSO2 ESB management console or using file system, deploy the above proxy service.

Step 5:

Let's send a JSON request to our proxy service. I will use soapUI as usual since it is the best tool out there to invoke web services.

Open soapUI project and add a HTTP request test step to any existing test suite. Change the default method as POST. Enter the proxy service endpoint URL (http://<esb_host_name>:8280/services/jsonproxy) as the Request URL.
Set the media type as application/json and add the following JSON request to POST body text area.

{
  "Customer": { "name": "charitha" }
}

Finally, your soapUI request editor will look like the following.

Submit the request. You will get a JSON response back as shown below. WSO2 ESB will seamlessly forward JSON request to RESTful web service and forward the JSON response to soapUI.

HTTP/1.1 200 OK
Content-Type: application/json; charset=UTF-8
Server: WSO2 Carbon Server
Date: Sat, 12 Jan 2013 13:34:31 GMT
Transfer-Encoding: chunked

{"Customer":{"id":"125","name":"charitha"}}

NOTE:
In the previous versions of ESB (4.0.3 etc), you should add JSON message builders and formatters to ESB_HOME/repository/conf/axis2.xml in order to enable JSON message processing capabilities in ESB. (See http://docs.wso2.org/wiki/display/ESB403/ESB+and+JSON)
However, these builder and formatters are enabled by default in WSO2 ESB version 4.5.1 and later.  In addition to that, you may need to set the following property in IN and OUT paths of proxy services and sequences in previous ESB versions.

<property name="messageType" value="application/json" scope="axis2" />

How to secure a SOAP web service in WSO2 Application Server and invoke it using HTTP basic authentication with soapUI

In a previous blog post, I explained the steps to deploy a web service in Apache Axis2, secure it with HTTP basic authentication and invoke it using Apache Jmeter.  In this post, we will repeat the same using different set of tools and frameworks which are comparatively easy-to-use and user-friendly.
We will secure a simple SOAP based web service which is hosted in WSO2 Application Server, configure an authentication security policy and invoke the service using HTTP basic auth through soapUI.

Pre-Requisites: 
Download and install WSO2 Application Server-5.0.0 
Download and install soapUI 4.0.1 or later version

Step 1:

Start WSO2 Application Server by running wso2server.sh startup script. Access the management console url (https://localhost:9443/carbon) and log in with the default administrator credentials (username=admin, password=admin)

Step 2:
Go to the Deployed Services page and you will find "HelloService" is deployed by default. Click on the service and go the the service dashboard. Click on Security to configure a security policy for the service. You will find the default set of security policies in the Security for the service page. In order to protect the service with HTTP basic authentication, we can configure Usernametoken security scenario. Username token security policy secures the service using username and password. Select UsernameToken security scenario and click on Next. Select admin as the user group which need to be authenticated. Click Finish to complete the security policy configuration.

Step 3:
Now, if you check the WSDL of the web service, you will see a WS-Security policy is added to the WSDL. Therefore, if you send a SOAP request, it must include the necessary WS-Security headers.
You can read one of my previous blog posts to see how you can send a secure SOAP request to a service using soapUI.
The beauty of WSO2 Application Server is, it allows you to send a HTTP GET request with basic authentication headers to the same web service. Let's see how we can send a HTTP GET request to invoke the above HelloService which has been secured with Username Token security policy.

Step 4:
Open soapUI.   Click on File --> New soapUI Project. Enter a name for the project. Select Create Web TestCase option (We are not going to deal with any SOAP messages, therefore we do not want to create usual soapUI project using a WSDL). This will open Add Web TestCase dialog. Enter the following URL as the web address.
Web address = https://localhost:9443/services/HelloService/greet?name=soapUI

Note that, this is the url of the GET request which we are going to submit.

Step 5:

Once you click on OK in the above dialog, soapUI will open an HTTP request editor with the above request URL. Click on Request tab to move into the request editor if you ended up in the response tab.
Click on Aut tab where you can specify user credentials. Enter the following credentials of admin user which we used to secure HelloService at the step 2.

Username=admin
Password=admin

Step 6:

Before invoking the service, make sure Authenticate Preemptively option is selected at the HTTP Settings of soapUI Preferences dialog (File --> Preferences --> HTTP Settings --> Authenticate Preemptively).
Now, submit the request. You will see the following in the raw view of request.


GET https://localhost:9443/services/HelloService/greet?name=soapUI HTTP/1.1
Accept-Encoding: gzip,deflate
Authorization: Basic YWRtaW46YWRtaW4=
User-Agent: Jakarta Commons-HttpClient/3.1
Host: localhost:9443

And, the response will be similar to the following.


HTTP/1.1 200 OK
Content-Type: application/xml;charset=UTF-8
Transfer-Encoding: chunked
Content-Encoding: gzip
Vary: Accept-Encoding
Date: Thu, 20 Sep 2012 08:38:31 GMT
Server: WSO2 Carbon Server

Hello World, soapUI !!!

Consuming a SOAP service using WSO2 API Manager

SO2 API Manager is the new kid in the block, the first fully open source API management platform which can be used to create, manage, consume and monitor APIs. WSO2 API manager is based on the award winning SOA middleware platform, WSO2 Carbon thus it inherits the features provided by the modular Carbon platform.

API authors and publishers can publish both REST and SOAP services to external consumers or partners through WSO2 API manager. In this post, we will look in to the steps of publishing and consuming a SOAP based web service using WSO2 API Manager.

pre-requisites:

Download and install alpha version of WSO2 API Manager from http://dist.wso2.org/products/api-manager/1.0.0-alpha/wso2am-1.0.0-ALPHA.zip. Similar to the other WSO2 Carbon products, you just need to have JDK-1.6.0 or later to run WSO2 API Manager. Then you can just extract the binary and start the server.

Step 1:

We will use a simple SOAP based web service hosted in WSO2 StratosLive Application Server. You can find the WSDL of the service at http://appserver.stratoslive.wso2.com/services/t/superqa.com/Axis2Service?wsdl

Step 2:

WSO2 API Manager consists of three different applications which serve for distinct user requirements. The API Gateway which is accessible through https://localhost:9443/carbon, is used as a gateway for API calls. The API Provider web app is used to publish, manage and monitor statistics of APIs. API provider can be accessed through https://localhost:9443/apiprovider. API Store application acts as the API consumer front end where users can subscribe for the published API. API Store is accessible through https://localhost:9443/apistore Ideally, three different user categories access these three applications. The API gateway is usually hosted and managed by API management provider. Therefore, we can assume that the default admin user of WSO2 API Manager as the API management provider thus he/she initiates creating rest of the user roles in the system. The API provider webapp is usually accessed by people who want to publish and manage new APIs. Therefore, we can think of another distinct role related to creating and publishing APIs. Finally, the API consumer user access API store webapp to subscribe for APIs.

Though we can use the default admin user to access all of the above applications, to be much realistic, lets start to create the roles and users associated to the API provider and API store applications.

• Log in to API gateway app as admin user (user name=admin, password=admin).
• Click on Configurein left menu and select Users and Roles
• Click on Add New Role, enter "publisher" (or whatever you want) as the role name and click on next
• Select the following permissions from the permission tree

Admin Permissions --> Configure --> Governance

Admin Permissions --> Login

Admin Permissions --> Manage --> API --> publish

Admin Permissions --> Manage --> API --> create

Admin Permissions --> Manage --> Resources --> Govern

• Click on Finish to complete the creation of new role
• Now access the Users page and click on Add New User. (say 'charitha' as the user name of new user, password is 'charitha')
• Assign the above role (publisher) to the new user

We have just created a new user who is granted permissions to access API provider webapp.

Step 3:

Access https://localhost:9443/apiprovider. The log in page of the API publisher application will be launched. Log in to the application using the credentials of the user which we have created in the previous step.
Click on Add menu item in the left navigation pane to add a new API.

Enter the following values for the mandatory fields in the above form.

Name: Axis2ServiceAPI
Context: /SOAPService
Version: 1.0.0
Endpoint URL: http://appserver.stratoslive.wso2.com/services/t/superqa.com/Axis2Service
Tier Availability: Bronze

Leaving the other default values as they are, click on Create to add the new API. The published API will be shown in the All APIs page and it will be in the CREATED state.

Step 4:

The APIs which are at the CREATED state cannot be consumed by any user. Therefore, we need to publish the API to API store. Click on the Axis2ServiceAPI-V1.0.0 in All APIs page which will direct you to the detailed information page of the selected API. Click on Lifecycles tab. Select the state as PUBLISHED. Select Push To Gateway option so that the API will be registered in the API gateway.

Clicking on Update will publish the API. If you see the log of API gateway, you will notice the following message. INFO - API Initializing API: charitha--Axis2ServiceAPI 

Step 5: 

Now, the API which we have created from the SOAP based web service is in PUBLISHED state so that a consumer can subscribe to the API and invoke it. API store can be used for the consumers to subscribe to APIs. Access https://localhost:9443/apistore/. The landing page of WSO2 API Store application will be shown. You will notice the API which just has been published in available in that page.

Click on Signup at the top right corner to register a new consumer user. Enter 'apiconsumer' as the username and 'apiconsumer123' as the password.

Log in to API store using the new consumer's credentials.

Step 6:

Now, we are inside the API Store web application. We should create an application to make API subscriptions and consumption. Click on My Applications tab.

Add a new application with the name, AxisApp as shown below.

We need to generate an OAuth key to consume any API. Click on the Axis2ServiceAPI - 1.0.0 in the API list or left pane of the API Store web application.  Select Axis2App from the Applications dropdown list at the right of the page and click on subscribe.

The "Subscription Successful" confirmation dialog will be popped up. Click on Go to My Subscriptions button to check our new API subscription.

Click on Generate in the above page to create a new API key. Once the key is generated, click on Show Key and copy the API key.

Step 5: Now, we have everything required to consume the API. API has been published to the store, subscribed to the API and possessed an API key. Lets invoke the API using any client. In this example, I will use soapUI, the world's best and most user-friendly web service invocation utility to invoke the API.

• Create a new soapUI project or use an existing project. 
• Add the following SOAP request to your soapUI project.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://service.carbon.wso2.org">

   <soapenv:Header/>

   <soapenv:Body>

      <ser:echoString>

            <ser:s>charitha</ser:s>

      </ser:echoString>

   </soapenv:Body>

</soapenv:Envelope>

• Add http://localhost:8280/SOAPService/1.0.0/ as a new endpoint url in soapUI request editor. (This url can be found under the overview tab of Axis2ServiceAPI - 1.0.0 in API store)
• Add the following HTTP header 

Authorization: Bearer  <Your API key which has been generated when subscribing to the API >

• Now, your soapUI request editor will be similar to the following.

• Submit the request and check the response. Since we have subscribed under bronze tier, you will be allowed to send 1 request per minute

WS-Addressing with SOAPUI

WS-Addressing is a standard way of including message routing data within SOAP headers without relying on transport specific routing properties. If you are talking to a web service which expects WS-Addressing information in request SOAP messages, you could either follow a programmatic approach to write a client and insert WS-A headers manually or use a commercial or free web service client tool. SOAPUI is the simplest tool which can be used to insert WS-Addressing headers in to request SOAP messages.
In this blog post, we will;

• Talk to a WS-Addressing enabled service which is hosted in WSO2 Application Server using SOAPUI

• Assert response SOAP message to validate WS-A headers

Pre-requisites:

1. Download and install the free version of SOAPUI 3.0.1 or later
2. Download and install WSO2 Application Server 4.0

Step 1

We are going to invoke the default HelloService which is shipped with WSO2 Application Server. Go to WSO2_APPSERVER/home/bin and start the server by running wso2server.sh{bat}
Access management console with https://localhost:9443/carbon and log in using the default administrator credentials (username=admin, password=admin)
Navigate to Manage --> Services --> List in the left menu and select "HelloService"
You will be directed to the dashboard of HelloService. Click on Modules link.



You will notice that "addressing-3.1.0" is listed under "Globel Level". By default WS-Addressing is enabled for all the services hosted in WSO2 Application Server. Therefore, you do not have to configure anything at the server side to enable WS-Addressing.

Step 2

Now, we will create a new SOAPUI project referring to http://localhost:9763/services/HelloService?wsdl as follows. Make sure to select "Create Test Suite" check box as well.



After creating the project, select greet request under HelloServiceSoap11Binding TestSuite



If you click on run just by providing input string, you will notice that WS-A headers will not be included in the request SOAP message which can be observed if you toggle "Raw" option.

Step 3

Lets add WS-A headers in to the request SOAP message. Select greet request and click on WS-A option and select the following properties.

Enable WS-A addressing

Add default wsa:Action

Add default wsa:To

Randomly generated message ID




Keep the rest of the default values and click on run. You will notice the following WS-A headers in the request message if you look at the raw message in SOAPUI.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greet</wsa:Action>
<wsa:MessageID>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:MessageID>
<wsa:To>http://localhost:9763/services/HelloService.HelloServiceHttpSoap11Endpoint/</wsa:To>
</soapenv:Header>

Also, the response headers will be similar to the following.

<soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:Action>urn:greetResponse</wsa:Action>
<wsa:RelatesTo>uuid:1fa996ca-755b-486c-a2ed-aa14bbf663a3</wsa:RelatesTo>
</soapenv:Header>

Simple.. isn't it? we just invoked a service with WS-Addressing headers using SOAPUI.

Here, we manually verified that the response message includes the correct WS-A headers. We looked at the response message and verified each header. If you maintain an automated test suite to verify your web service interactions, the above procedure cannot not be considered as over until you add an assertion so that the test automatically reports the correctness or failures of response/request messages.

Step 4

Now, we will add an assertion to validate response headers. In this example, we are going to add an XPath match assertion to check whether the wsa:Action header of the response equals to urn:greetResponse

Click on Assertions option and select "Add an assertion to this item" icon. Then select, XPAth Match as the assertion.



Next, you should specify the XPath expression and expected result as follows. First, click on Declare button which will automatically define the namespaces. Specify //wsa:Action as the xpath. Also, specify urn:greetResponse as the expected result.



Run the test again. You will notice that the test will be marked in green denoting that the test is successful.

Invoking secure web services using SOAPUI - part1

SOAPUI is a very useful free tool which can be used in SOA testing. Service invocation using SOAPUI is straight forward and you can find a lot of references by surfing web. However, there are limited information about invoking services with various QOS (Quality of Service) features such as WS-Security, WS-Addressing etc.

This post takes you through the simplest QOS scenario, a web service is secured with UserName token policy and how SOAPUI is used to invoke such a service.

I will use the default HelloService hosted in WSO2 WSAS in this demonstration but the service invocation approach is similar with any service provider.

Pre-requisites:

1. Download and install SOAPUI-3.6.1 or later (free version)

2. Download and install WSO2 WSAS-3.*

Step 1

First, we need to secure a web service hosted in WSAS. We will configure HelloService so that only the users in admin group will be allowed to invoke the service. In order to do that, start WSO2 WSAS and log in to management console. Then, select HelloService from Deployed Services page.

You will be directed to the service dashboard of HelloService as follows.

Click on Security link and Select "Enable Security" option. Then select, UserNameToken security scenario.

In the next screen, select admin user group and click on finish to apply the security policy for HelloService.

Step 2
We have configured server side security policy. If you look at the WSDL of HelloService (http://localhost:9763/services/HelloService?wsdl), you will notice that UserNameToken security policy is added to the service.
Now, we need to configure SOAPUI project to talk to HelloService with the required user credentials.

First, start SOAPUI and create a new project. Specify https://localhost:9443/services/HelloService?wsdl as the initial WSDL. Keep the other default settings.
You will see the following project structure.



Replace ? with an input value in the request editor and run the test. You will encounter a SOAPFault, Missing wsse:Security header in request, since we did not send security headers with the request.
Lets configure client side security now. In the Request Properties pane of SOAPUI project, you will find the following properties.

Username
Password
WSS-Password Type
WSS TimeToLive

Specify the following values for the above properties.

Username = admin
Password = admin
WSS-Password Type = PasswordText
WSS TimeToLive = 2000

Run the test. Have a look at the raw request view. You will see security header is added to the request.
Thats all! We will look at more advanced scenarios such as signing and encrypting messages with SOAPUI in future posts.