Monday, June 19, 2017

How to remove a thumbnail from an API

Let's say you have created an API in API cloud and you have added thumbnail image to it. Now you want to remove it.

When you go to the edit api view it allows you to change the thumbnail, but not remove. Let's see how we can remove it.

1. login to the carbon console of gateway node as tenant admin

2. Go to Resource -> Browse under main menu

3. Go to "/_system/governance/apimgt/applicationdata/provider" 

4. Click on the relevant tenant - you will see list of APIs (eg:

5. Select relevant API - you will see api artifact   (eg: api1 under version 1.0.0)

6. Click on "api" - you will see list of meta data for that api

7. Remove the thumbnail value from attribute "Thumbnail"

8. Save the API

9. Then logout from the API publisher UI and login in incognito window, you will see thumbnail has removed from your API.

Friday, June 2, 2017

How to start multiple services as a group in WSO2 Integration Cloud

Let's say, we have a use case which is deployed in Integration Cloud and that involves number of applications.
There can be a PHP/Web application which user interact, ESB which provide integration with number of systems and DSS to manipulate database.

So let's say we want to start/stop these 3 applications as a group. But at the moment, Integration Cloud does not provide any grouping. So you have to login to the Integration Cloud and go to each and every application and start/stop those.

To make this little easier, we can use Integration Cloud REST API and write our own script.

This is the script to start the all applications as a group. You need to provide username, password, organization name and file which contains application list with versions

How to execute this script
./ <username> <password> <orgnaizationName> wso2Project.txt

wso2Project.txt file content should be like this. There you should provide applicationName and version separated with [ | ] pipe character

As shown above you can keep number of project files and start using script.

Monday, April 10, 2017

Add multiple database users with different privileges for the same database

Currently, the WSO2 Integration Cloud supports adding multiple database users for a same database, but does not support changing user privileges.

Let's say someone has a requirement of using same database via two different user, one user has full access, where other user should have READ_ONLY access. How we do this in Integration Cloud?
We are planning to add this as feature to change the user permissions, but until that you can do it as I have mentioned below.


1. Login Create a database with a user

2. Once you create a database you can see it as below, and you can add another user when clicking on the All users icon

3. There you can create new user or you can attach existing user to the same database

I added two users u_mb_2NNq0tjT and test_2NNq0tjT to the database wso2mb_esbtenant1
My requirement is to give full access to the u_mb_2NNq0tjT user and remove INSERT permission from test_2NNq0tjT user.

4. Login to the via mysql client as user u_mb_2NNq0tjT and revoke the INSERT permission of test_2NNq0tjT

first login as test_2NNq0tjT and check grants
mysql -u  test_2NNq0tjT -pXXXXX -h

show grants
| Grants for test_2NNq0tjT@%                                                             |
| GRANT USAGE ON *.* TO 'test_2NNq0tjT'@'%' IDENTIFIED BY PASSWORD <secret>              |
| GRANT ALL PRIVILEGES ON `wso2mb_esbtenant1`.* TO 'test_2NNq0tjT'@'%' WITH GRANT OPTION |

login as u_mb_2NNq0tjT and revoke the insert permission
mysql -u  u_mb_2NNq0tjT -pXXXXX -h

REVOKE INSERT ON wso2mb_esbtenant1.* FROM 'test_2NNq0tjT'@'%';

login again as test_2NNq0tjT and check grants
mysql -u  test_2NNq0tjT -pXXXXX -h

show grants

| Grants for test_2NNq0tjT@%                                                                                                                                                                                                                                   |
| GRANT USAGE ON *.* TO 'test_2NNq0tjT'@'%' IDENTIFIED BY PASSWORD <secret>                                                                                                                                                                                    |
2 rows in set (0.24 sec)

With this approach we can change the permissions of another user who is attached to the same database.

To make an read-only user you need to revoke the permissions as follows

Please note: after you change the user privileges, do not detach/attach the test_2NNq0tjT user to the same or different database. Then it will set the all privileges automatically.

Friday, March 17, 2017

How to run a Jenkins in WSO2 Integration Cloud

This blog post guides you on how to run Jenkins in WSO2 Integration Cloud and configure it to build an GitHub project. Currently the WSO2 Integration Cloud does not support Jenkins as a app type, but we can use Custom docker app type with Jenkins docker image.

First we need to find out, proper Jenkins docker image, which we can use for this  or we have to build it from the scratch.

If you go to you can find official Jenkins images in docker hub, but we can't use this images as it is due to several reasons. So I'm going to create a fork of and do some changes to the Dockerfile.

I use the branch here.

A. You will see it has VOLUMN mount - at the moment WSO2 Integration Cloud does not allow you to upload an image which has VOLUMN mount. So we need to comment it out

#VOLUME /var/jenkins_home

B. My plan is to build Git hub project, so I need enable Git hub Integration plugin. So I add the following line at the end of the file

RUN docker-slaves github-branch-source

C. I want to build projects using Maven, so I add the following segment to the Dockerfile to install and configure Maven.


RUN mkdir -p /usr/share/maven /usr/share/maven/ref/ \
  && curl -fsSL$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz \
    | tar -xzC /usr/share/maven --strip-components=1 \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

ENV MAVEN_HOME /usr/share/maven
COPY settings-docker.xml /usr/share/maven/ref/
RUN chown -R ${user} "$MAVEN_HOME"

D. I don't want to expose slave agent port 50000 to the outside. Just comment it out.

#EXPOSE 50000

E. I want to configure the Jenkins job to build the project periodically, so I need to copy the required configurations to the Jenkins and give the correct permissions.

Note: You can first run a Jenkins on your local machine, configure the job and get the config.xml file.
I configured the Jenkins job to poll the Github project every 2 minutes and build. (You can configure the interval as you wish)

Here's the Jenkins configurations

<?xml version='1.0' encoding='UTF-8'?>
    <com.coravy.hudson.plugins.github.GithubProjectProperty plugin="github@1.26.1">
  <scm class="hudson.plugins.git.GitSCM" plugin="git@3.1.0">
    <submoduleCfg class="list"/>
      <spec>H/2 * * * *</spec>
      <targets>clean install</targets>
      <settings class="jenkins.mvn.DefaultSettingsProvider"/>
      <globalSettings class="jenkins.mvn.DefaultGlobalSettingsProvider"/>

We need to create the following content in the JENKINS_HOME/jobs folder, to configure a job

 --> jobs
         ├── HelloWebApp
         │   └── config.xml

Add the following to the Dockerfile.

RUN mkdir -p $JENKINS_HOME/jobs/HelloWebApp
COPY HelloWebApp $JENKINS_HOME/jobs/HelloWebApp

RUN chmod +x $JENKINS_HOME/jobs/HelloWebApp \
  && chown -R ${user} $JENKINS_HOME/jobs/HelloWebApp

So let's build the Jenkins image and test locally.
Go to the folder where the Dockerfile exist and execute

docker build -t jenkins-alpine .

Run the Jenkins

docker run -p 80:8080 jenkins-alpine

You will see the Jenkins logs in the command line

You can access the Jenkins via http://localhost/ and see build jobs running in every 2 minutes when it detects any changes in GitHub project.

If you click on the HelloWebApp and go to configure, then you will see the Jenkins job configurations.

So now the image is ready and let's push it to the docker hub and deploy it in WSO2 Integration Cloud.

docker images

REPOSITORY               TAG                               IMAGE ID            CREATED             SIZE
jenkins-alpine                 latest                            d7dc03cec1df        51 minutes ago      257.4 MB

docker tag d7dc03cec1df amalkasubasinghe/jenkins-alpine:hellowebapp

docker login

docker push amalkasubasinghe/jenkins-alpine:hellowebapp

When you login to the docker hub you can see the image you push

Let's login to the WSO2 Integration Cloud -> Create Application -> and select Custom Docker Image

add the image providing image URL

Wait until the security scanning finished and then create the Jenkins application selecting scanned image

Here I select Custom Docker http-8080 and https-8443 runtime, as Jenkins run in 8080 port.

Wait until the Jenkins instance fully up and running. Check the logs

Now you can access the Jenkins UI via

That's all :). Now every 2 minutes our Jenkins job will poll the GitHub project and if there are any changes it will pull the changes and build.

This is how you can setup and configure Jenkins in WSO2 Integration Cloud.

You can see the docker file here

Wednesday, February 15, 2017

WSO2 ESB communication with WSO2 ESB Analytics

This blog post is about how & what ports involved when connecting from WSO2 ESB to WSO2 ESB Analytics.

How to configure: This document explains how to configure it

Let's say we have WSO2 ESB  and WSO2 ESB Analytics packs we want to run in same physical machine, then we have to offset one instance. 
But we don't want to do that since WSO2 ESB Analytics by default come with the offset.

So WSO2ESB will run on 9443 port, WSO2 ESB Analytics will run on 9444 port

WSO2 ESB publish data to the WSO2 ESB Analytics via thrift. By default thrift port is 7611 and corresponding ssl thrift port is 7711 (7611+100), check the data-bridge-config.xml file which is in analytics server config directory . 

Since we are shipping analytics products with offset 1 then thrift ports are 7612 and ssl port is 7712.
Here, ssl port (7712) is used for initial authentication purposes of data publisher afterwards it uses the thrift port (7612) for event publishing.. 

Here's a common error people raise when configuring analytics with WSO2 ESB.

[2017-02-14 19:42:56,477] ERROR - DataEndpointConnectionWorker Error while trying to connect to the endpoint. Cannot borrow client for ssl://localhost:7713
org.wso2.carbon.databridge.agent.exception.DataEndpointAuthenticationException: Cannot borrow client for ssl://localhost:7713
        at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(
        at java.util.concurrent.Executors$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$
Caused by: org.wso2.carbon.databridge.agent.exception.DataEndpointSecurityException: Error while trying to connect to ssl://localhost:7713
        at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftSecureClientPoolFactory.createClient(
        at org.wso2.carbon.databridge.agent.client.AbstractClientPoolFactory.makeObject(
        at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(
        at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(
        ... 6 more
Caused by: org.apache.thrift.transport.TTransportException: Could not connect to localhost on port 7714
        at org.apache.thrift.transport.TSSLTransportFactory.createClient(
        at org.apache.thrift.transport.TSSLTransportFactory.getClientSocket(
        at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftSecureClientPoolFactory.createClient(
        ... 9 more
Caused by: Connection refused: connect
        at Method)
        at org.apache.thrift.transport.TSSLTransportFactory.createClient(
        ... 11 more

This comes because people change the thrift port comes in the following configuration files by adding another 1 (7612+1), thinking of we need to add 1, since we have offset in analytics server as 1.


Sunday, September 4, 2016

Add a new app type to the WSO2 App Cloud

This blog explain how you can add a new app type to the WSO2 App cloud.

Step 1: Get a clone of WSO2 app cloud code base
git clone

Step 2: Create the docker files for the runtime
Please refer our existing docker files when creating new docker files for particular runtime.

Following blog post gives you some details about structure of the docker files

Step 3: Required database changes
When adding new app type you need add some database records, following diagram gives you an idea of database schema.

AC_CLOUD defines the cloud types
AC_APP_TYPE defines app types
AC_RUNTIME defines runtimes
AC_CONTAINER_SPECIFICATIONS defines container specs
AC_TRANSPORT defines the ports we expose for end users

-- insert app type
-- insert app type, cloud mapping
-- insert runtime
-- insert app type, runtime mapping
-- insert container spec if required
-- insert runtime, container spec mapping
-- insert transport if required
-- insert runtime, transport mapping

Step 4: Specify app-type meta data in app-types-properties.json file

This json file we used to load the app type details to the App Cloud UI.
Step 5: Add a sample
We need to add a sample to implement deploy sample option.

Commit your sample archive here:

Specify the sample location here with the property <app_type>_sample_artifact_url

Step 7: Implement endpoints section loading to the app home page
Please refer the followign blog post to see how we have developed the "Endpoints" section per app type.

Loading endpoints to the app home page in WSO2 App Cloud

Currently we have 6 app types in WSO2 App Cloud and when we load a app home page of created applications we can see a section called "Endpoints". This blog post is about how we load endpoints per each and every app type and it's implementation

Loading endpoints for Java web application, Jaggery and PHP app types
for these 3 app types, user have to define the application context, when creating an application
as shown in below image.

Then we append that context to the app version url/default url and display in app home page as below.

Loading endpoints for Microservices app type
for microservices app type, microservices 2.0.0 itself provides a swagger url, we just display the swagger url here.

Loading endpoints for WSO2 dataservices app type
for dataservices app type, we need to invoke the ServiceAdmin soap service to get the endpoints. So we developed a axis2service and deployed in dss runtime, which invokes the ServiceAdmin and return the endpoints to the app cloud.

You can see the axis2Service here:

Loading endpoints for WSO2 ESB app type
for ESB app type, we have developed an API and deployed in ESB runtime to get the SOAP and REST endpoints

You can see the code of the API here:

The implementation code we use to load endpoints per app types, you can find here.

per app type we need to define which mechanism we need to load to the app home page, that we have defined in app-types-properties.json file, as endpointExtractorObject property.

enabling or disabling appContext property, will show/hide Application Context field in Create Application page.

providing a value to the defaultContext will show a default context in Application Context field in Create Application page