Wednesday, May 11, 2016

Configure "Secure Vault" to secure plain text passwords in WSO2 config files

If we take a WSO2 product, there are so many config files which contains plain text passwords. This blog post explains how we can secure those passwords.

Let's take WSO2 DAS . analytics-datasource.xml file and secure the password in following configuration.

Step 1:
Go to <WSO2_DAS>/repository/conf/security and add the following line to the file

Create alias with file path, xpath to the element and boolean value true.

Step 2:
Add the following line to the file.

You have to provide the alias with the plain text password

Step 3:
Go to <WSO2_DAS>/bin and execute ./ -Dconfigure

This will,
- Encrypt the password defined in file
- Configure the analytics-datasauces.xml as follows

Step 4: Restart the server.

How to change a password

Configure file with the password you want to change.  You need to remove the encrypted value and replace it with plain text password with  [ ] square brackets.
Execute the ./ -Dconfigure
Start the server.

Sunday, May 8, 2016

Publish a service which provides both XML/JSON via SOAP/REST protocols in WSO2 ESB

Let's say we want to define a service in WSO2 ESB that provides the same information using different formats and protocols. It should provide both XML and JSON formats (for example 2 operations) and should be provided as SOAP and REST.
This service will consume a DSS service we will deploy as SOAP and as REST.

What is the best way to achieve this via WSO2 ESB?

1. The best approach is to use a proxy for support XML and invoke SOAP endpoint and an API for support JSON and invoke REST endpoint.

2.Then the problem is, publishing it as a proxy we get all the facilities provided in the dashboard. If we use an API we do not have the dashboard options available (statistics, mediation tracer...)

As a solution:
a. we can move the content of the API to a separate sequence then we can get the dashboard options.
b. publish them via a BAM

3. Further, we can define a sequence which transform XML to JSON and vice versa in both Proxy and API and share it with both API and Proxy.

Thursday, May 5, 2016

Configure WSO2 DSS to support Cassandra user defined data types

This is about how to configure WSO2 DSS to support Cassandra user defined data types

Cassandra user defined data type and table:

WSO2 DSS configurations:

Request we should send as:

Sending json string within xml payload

Assuming we are getting the following request to the WSO2 ESB.

I want to generate the following xml payload which contains json string and send it to the backend.

Here's the sequence:

Verify the WSO2 DSS config is valid to connect Cassandra

You can try querying column families from system.schema_columnfamilies to check whether DSS configuration is valid and integration happens correctly.

Here's the DSS configuration

You can try as follows
  1. download the WSO2 DSS and start the server
  2. copy the above configuration to a text file and save as .dbs file and copy to the <DSS_HOME>\repository\deployment\server\dataservices
  3. wait until it deploy
  4. get the WSDL from management console and add it to the SOAP UI and invoke the getColumnfamilies operation – it will list all the column families from your database.

Configure BAM to does not check the presence of Cassandra and Hadoop on start up

I want to configure the BAM to does not check the presence of Cassandra, Hadoop, etc...
If I do not configure in master-data source, it keeps throwing error saying Hadoop not configured, Hive error etc.
As per, I want to generate reports without using Hadoop, Cassandra, etc...
How to disable all these and instead configure RDBMS (or for local H2)

This scenario is about how to use BAM for purely reporting purposes without using it to publish data or summarize data. You can start the BAM server with analytics and receiver components disabled along with the cassandra components disabled as well.
You can follow the "BAM profiles" sections in [1] to get more information on different BAM profiles.

you can start the server with the following command executed from within the <BAM_HOME>/bin folder.
./ -Ddisable.receiver=true -Ddisable.cassandra.server.startup=true

In a clustered scenario the hive jobs or hadoop jobs will not be submitted to the nodes started with the analytics components disabled and data will not be published to the nodes with receiver components disabled.


Wednesday, May 4, 2016

Generate a file using the request payload and send to a backend via WSO2 ESB

Let's say WSO2 ESB receives an request with a payload, and we need to create a file using that payload content and send that file to a endpoint.

Here's the required proxy cofiguration.

This sequence builds the message as a multipart/form-data and send it to the endpoint.

If you want to customise MultipartFormDataFormatter, write your own class extending MultipartFormDataFormatter.

Configure it in WSO2_ESB/repository/conf/axis2/axis2.xml file as follows.

This org.wso2.custom.CustomMultipartFormDataFormatter sets the file name as filename.xml, Content-Type as text/xml and payload as the content of the file

Invoke two endpoints in same sequence

Scenario: invoke two different endpoints in the same sequence. The first invoke is only for logging purpose and its result has no impact on the rest of the process.

Here's the sample sequence. Here I have used Clone Mediator

Use the call mediator instead of send mediator in first target, if you do not expect any response from the endpoint it's good to set the OUT_ONLY property before the call mediator.

If you use the Send mediator here, the outSequence will be invok for all the response coming into the ESB.
When the first time outSequence called it invokes logAccessEP endpoint and then logAccessEP returns a reponse to the ESB, then again ESB invokes outSequence and then logAccessEP endpoint .... like wise it goes as a loop.

User Password History Management in WSO2 IS

WSO2 IS does not maintain password history of the users.

1) Keep a history of password for users in a configured number of counts per Role
2) Password history should be kept with a timestamp (in 1 timezone) so that a user does reuse the same password in xx number of days.
3) Have Password History configurable to roles level

To implement password history feature you have to write your own custom user store manager. Refer doc [1]
Checkout the code from [2]

You have to create your own database tables related to password histroty feature and add the implementation to the following methods


This java implementation will be same for any RDBMS.


Building WSO2 MB 2.2.0 from the scratch

Build MB 2.2.0 [1] :

According to the Release Matrix [2] MB 2.2.0 has released with chunk-11
1. Checkout the kernal patches [3] and go to patches folder and build patches up to 0008 one by one (mvn clean install -Dmaven.test.skip=true)
2. Checkout the product [4], comment out the other products except mb-2.2.0 from the pom [5] and build the product here [6] (mvn clean install -Dmaven.test.skip=true)

For this you need JDK 1.6 and Maven 3.0.5 versions


Installation Pre-requisites

You can find the Integration tests in following location for MB 2.2.0 [8]

How to deactivate the proxy deployed in WSO2 ESB

To deactivate the proxy make the startOnLoad=false. But please note this will deactivate the proxy on next server restart on at the deployment of proxy.

In the server start up, you can see the following logs in the WSO2 ESB console.

[2015-06-01 11:07:51,911] INFO - ProxyService Successfully created the Axis2 service for Proxy service : FileSystemVFSProxy
[2015-06-01 11:07:51,911] INFO - Axis2SynapseController Deployed Proxy service : FileSystemVFSProxy
[2015-06-01 11:07:51,914] INFO - ProxyService Stopped the proxy service : FileSystemVFSProxy
This means that "FileSystemVFSProxy" is not started with WSO2 ESB server.

But once you start up the WSO2 ESB server with startOnLoad="true" and change that to false, it won't deactivate your proxy service unless you restart the WSO2 ESB server.If you need to deactivate the VFS proxy without restarting the WSO2 ESB server you need to deactivate the proxy via WSO2 carbon console.

WSO2 products & i18n

Use Case 1:
When I create some registry, like collection, I need input path first, then I want to specified display name for it, so I add a property named 'name' with value 'display name', how to i18n the display name.

All the UI features of the WSO2 products have i18n support [1]. So we can i18n the UI text only.

Use Case 2:
When first start the wso2 server, it will create records into database, by default some data is in the English language, how to i18n the the WSO2 init data

Answer :
Can't i18n the values we enter and the data created at the server startup.

Bundle different WSO2 products into one and run in one runtime environment

Example: Bundle API Manager, ESB and BAM into one bundle and run in single JVM.

We use WSO2 Carbon [1] as the platform and on top of that we have developed reusable features. Each WSO2 product is a collection of features.
So,there are two options to achieve this requirement;
A) Take WSO2 Carbon and install required features related to API Manager, ESB and BAM on it [2]
B) Take any product (eg: ESB) and install required features of other products (eg: API Manager and BAM) [3]

Installing Carbon 4.2.0 based features on Carbon 4.4.0 is wrong and not allowed. Use WSO2 Carbon 4.2.0 feature repo and install the required features on WSO2 Carbon 4.2.0 (or on any WSO2 Carbon 4.2.0 based product as described above).

Features are available on the feature repo only when the products which use those features are released (each WSO2 Product is a collection of features. Please refer [4] for more information).
For example if you consider WSO2 ESB 4.8.1 which is based on WSO2 Carbon 4.2.0, features used for WSO2 ESB 4.8.1 are available on the feature repo when WSO2 ESB 4.8.1 is released.

WSO2 Carbon 4.3.0 and 4.4.0 was released very recently. At the moment not all products are released based on those version.

How to catch the AD password expired/locked situation by WSO2IS

Solution: you have to write your own custom user store.

When the password expired or account locked from the AD level, AD returns an error codes [1] to the IS. That error code you can catch in the customer user store and implement your own logic to fulfilling your requirement.

So, create a new class extending org.wso2.carbon.user.core.ldap.ActiveDirectoryUserStoreManager [2], and it will let you to override the doAuthenticate() method which derived from ReadOnlyLDAPUserStoreManager [3]. Note that you need to implement bindAsUser() method, and catch the AuthenticationException which throws when the password expired or account locked from the AD level and based on that you can implement your own logic.

You can see the error codes thrown from AD to IS when enabling DEBUG logs in org.wso2.carbon.user.core component.
For this you need to navigate to [IS_HOME]/repository/conf directory and add the following entry to file and save it.

Then restart the server and try to login with password expired or account locked user. You can see the error codes in [IS_HOME]/repository/logs/wso2carbon.log file

Refer [4] which will help you on how to write a custom user store manager and deploying the same. After deployment and restarting the server verify whether your scenario works as expected.

Refer [5] the sample CustomUserStoreManager code base which will help you identifying the structure you need to implement your class. ( You can download this source and adhere to this structure when writing your own implementation)

Setup SSO in WSO2 product with other Identity provider except WSO2IS

This blog post will provide instruction on how to setup SSO, WSO2 DAS server with WSO2 IS.

If you want to setup SSO in WSO2 product with different identity provider. you should consider the following.

Let's say we want to configure WSO2 Greg with ADFS.
1. Configure SSO in Greg updating following configurations in <GREG_HOME>/repository/deployment/server/jaggeryapps/publisher/config/publisher.json

2. Then configure SSO in ADFS and import the ADFS signing cert into the Greg's JKS.
3. Once we setup the SSO, when login to the Greg, it will redirect to the ADFS, then ADFS will authenticate the user and return the SAML assertion.
4. Then authorization should be happen in Greg side. But it will fail, since the user does not exist in Greg side.

to solve that issue I could find number of solutions assuming you can't share the ADFS user base with GREG:
a: We can create a replica of the user base of ADFS (without passwords/ with dummy password) and Greg pointing to the replica

b: When adding user to the ADFS side, add the same user (without passwords/ with dummy password) and roles to the Greg side. either via a provisioning app, script or manually.

c: Write a JIT provisioning extension to provision the user in GREG side (when the Greg receive the SAML assertion) and authorize to the application.

Hope this will help

How to set dynamic content for Jmeter HTTP Request

Let's say you want to generate a unique ID for MessageID property in the request

Add BSF PreProcessor and generate the MessageID as follows

Monday, May 2, 2016

How to generate random unique number in SOAP UI request

eg 1:

${=System.currentTimeMillis() + ((int)(Math.random()*10000))}

eg 2:

${=System.currentTimeMillis() + ((int)(Math.random()*10000))} - See more at:
${=System.currentTimeMillis() + ((int)(Math.random()*10000))} - See more at:
${=System.currentTimeMillis() + ((int)(Math.random()*10000))} - See more at: