Monday, June 19, 2017

How to remove a thumbnail from an API

Let's say you have created an API in API cloud and you have added thumbnail image to it. Now you want to remove it.

When you go to the edit api view it allows you to change the thumbnail, but not remove. Let's see how we can remove it.

1. login to the carbon console of gateway node as tenant admin
https://gatewaymgt.api.cloud.wso2.com/carbon

2. Go to Resource -> Browse under main menu

3. Go to "/_system/governance/apimgt/applicationdata/provider" 

4. Click on the relevant tenant - you will see list of APIs (eg: amalka-AT-wso2.com-AT-esbtenant1)


5. Select relevant API - you will see api artifact   (eg: api1 under version 1.0.0)

6. Click on "api" - you will see list of meta data for that api



7. Remove the thumbnail value from attribute "Thumbnail"

8. Save the API

9. Then logout from the API publisher UI and login in incognito window, you will see thumbnail has removed from your API.







Friday, June 2, 2017

How to start multiple services as a group in WSO2 Integration Cloud

Let's say, we have a use case which is deployed in Integration Cloud and that involves number of applications.
There can be a PHP/Web application which user interact, ESB which provide integration with number of systems and DSS to manipulate database.

So let's say we want to start/stop these 3 applications as a group. But at the moment, Integration Cloud does not provide any grouping. So you have to login to the Integration Cloud and go to each and every application and start/stop those.

To make this little easier, we can use Integration Cloud REST API and write our own script.

This is the script to start the all applications as a group. You need to provide username, password, organization name and file which contains application list with versions


How to execute this script
./startProject.sh <username> <password> <orgnaizationName> wso2Project.txt

wso2Project.txt file content should be like this. There you should provide applicationName and version separated with [ | ] pipe character

As shown above you can keep number of project files and start using startProject.sh script.

Monday, April 10, 2017

Add multiple database users with different privileges for the same database

Currently, the WSO2 Integration Cloud supports adding multiple database users for a same database, but does not support changing user privileges.

Let's say someone has a requirement of using same database via two different user, one user has full access, where other user should have READ_ONLY access. How we do this in Integration Cloud?
We are planning to add this as feature to change the user permissions, but until that you can do it as I have mentioned below.

Steps:

1. Login Create a database with a user


2. Once you create a database you can see it as below, and you can add another user when clicking on the All users icon


3. There you can create new user or you can attach existing user to the same database


I added two users u_mb_2NNq0tjT and test_2NNq0tjT to the database wso2mb_esbtenant1
My requirement is to give full access to the u_mb_2NNq0tjT user and remove INSERT permission from test_2NNq0tjT user.

4. Login to the mysql.storage.cloud.wso2.com via mysql client as user u_mb_2NNq0tjT and revoke the INSERT permission of test_2NNq0tjT

first login as test_2NNq0tjT and check grants
mysql -u  test_2NNq0tjT -pXXXXX -h mysql.storage.cloud.wso2.com

show grants
+-----------------------------------------------------------------------------------------+
| Grants for test_2NNq0tjT@%                                                             |
+-----------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'test_2NNq0tjT'@'%' IDENTIFIED BY PASSWORD <secret>              |
| GRANT ALL PRIVILEGES ON `wso2mb_esbtenant1`.* TO 'test_2NNq0tjT'@'%' WITH GRANT OPTION |
+-----------------------------------------------------------------------------------------+


login as u_mb_2NNq0tjT and revoke the insert permission
mysql -u  u_mb_2NNq0tjT -pXXXXX -h mysql.storage.cloud.wso2.com

REVOKE INSERT ON wso2mb_esbtenant1.* FROM 'test_2NNq0tjT'@'%';

login again as test_2NNq0tjT and check grants
mysql -u  test_2NNq0tjT -pXXXXX -h mysql.storage.cloud.wso2.com

show grants

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for test_2NNq0tjT@%                                                                                                                                                                                                                                   |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'test_2NNq0tjT'@'%' IDENTIFIED BY PASSWORD <secret>                                                                                                                                                                                    |
| GRANT SELECT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, SHOW VIEW, CREATE ROUTINE, ALTER ROUTINE, EVENT, TRIGGER ON `wso2mb_esbtenant1`.* TO 'test_2NNq0tjT'@'%' WITH GRANT OPTION |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
2 rows in set (0.24 sec)


With this approach we can change the permissions of another user who is attached to the same database.

To make an read-only user you need to revoke the permissions as follows
REVOKE INSERT, UPDATE, DELETE, CREATE, DROP, REFERENCES, INDEX, ALTER, CREATE TEMPORARY TABLES, LOCK TABLES, EXECUTE, CREATE VIEW, CREATE ROUTINE, ALTER ROUTINE, EVENT, TRIGGER on `wso2mb_esbtenant1`.*  from 'test_2NNq0tjT'@'%'; 

Please note: after you change the user privileges, do not detach/attach the test_2NNq0tjT user to the same or different database. Then it will set the all privileges automatically.

Friday, March 17, 2017

How to run a Jenkins in WSO2 Integration Cloud

This blog post guides you on how to run Jenkins in WSO2 Integration Cloud and configure it to build an GitHub project. Currently the WSO2 Integration Cloud does not support Jenkins as a app type, but we can use Custom docker app type with Jenkins docker image.


First we need to find out, proper Jenkins docker image, which we can use for this  or we have to build it from the scratch.

If you go to https://hub.docker.com/_/jenkins/ you can find official Jenkins images in docker hub, but we can't use this images as it is due to several reasons. So I'm going to create a fork of https://github.com/jenkinsci/docker and do some changes to the Dockerfile.

I use the https://github.com/amalkasubasinghe/docker/tree/alpine branch here.

A. You will see it has VOLUMN mount - at the moment WSO2 Integration Cloud does not allow you to upload an image which has VOLUMN mount. So we need to comment it out

#VOLUME /var/jenkins_home

B. My plan is to build Git hub project, so I need enable Git hub Integration plugin. So I add the following line at the end of the file

RUN install-plugins.sh docker-slaves github-branch-source

C. I want to build projects using Maven, so I add the following segment to the Dockerfile to install and configure Maven.

ARG MAVEN_VERSION=3.3.9

RUN mkdir -p /usr/share/maven /usr/share/maven/ref/ \
  && curl -fsSL http://apache.osuosl.org/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz \
    | tar -xzC /usr/share/maven --strip-components=1 \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

ENV MAVEN_HOME /usr/share/maven
COPY settings-docker.xml /usr/share/maven/ref/
RUN chown -R ${user} "$MAVEN_HOME"


D. I don't want to expose slave agent port 50000 to the outside. Just comment it out.

#EXPOSE 50000

E. I want to configure the Jenkins job to build the https://github.com/amalkasubasinghe/HelloWebApp/ project periodically, so I need to copy the required configurations to the Jenkins and give the correct permissions.

Note: You can first run a Jenkins on your local machine, configure the job and get the config.xml file.
I configured the Jenkins job to poll the Github project every 2 minutes and build. (You can configure the interval as you wish)

Here's the Jenkins configurations https://github.com/amalkasubasinghe/docker/blob/jenkins-alpine-hellowebapp/HelloWebApp/config.xml

<?xml version='1.0' encoding='UTF-8'?>
<project>
  <description></description>
  <keepDependencies>false</keepDependencies>
  <properties>
    <com.coravy.hudson.plugins.github.GithubProjectProperty plugin="github@1.26.1">
      <projectUrl>https://github.com/amalkasubasinghe/HelloWebApp/</projectUrl>
      <displayName></displayName>
    </com.coravy.hudson.plugins.github.GithubProjectProperty>
  </properties>
  <scm class="hudson.plugins.git.GitSCM" plugin="git@3.1.0">
    <configVersion>2</configVersion>
    <userRemoteConfigs>
      <hudson.plugins.git.UserRemoteConfig>
        <url>https://github.com/amalkasubasinghe/HelloWebApp</url>
      </hudson.plugins.git.UserRemoteConfig>
    </userRemoteConfigs>
    <branches>
      <hudson.plugins.git.BranchSpec>
        <name>*/master</name>
      </hudson.plugins.git.BranchSpec>
    </branches>
    <doGenerateSubmoduleConfigurations>false</doGenerateSubmoduleConfigurations>
    <submoduleCfg class="list"/>
    <extensions/>
  </scm>
  <canRoam>true</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers>
    <hudson.triggers.SCMTrigger>
      <spec>H/2 * * * *</spec>
      <ignorePostCommitHooks>false</ignorePostCommitHooks>
    </hudson.triggers.SCMTrigger>
  </triggers>
  <concurrentBuild>false</concurrentBuild>
  <builders>
    <hudson.tasks.Maven>
      <targets>clean install</targets>
      <usePrivateRepository>false</usePrivateRepository>
      <settings class="jenkins.mvn.DefaultSettingsProvider"/>
      <globalSettings class="jenkins.mvn.DefaultGlobalSettingsProvider"/>
      <injectBuildVariables>false</injectBuildVariables>
    </hudson.tasks.Maven>
  </builders>
  <publishers/>
  <buildWrappers/>
</project>

We need to create the following content in the JENKINS_HOME/jobs folder, to configure a job

JENKINS_HOME
 --> jobs
         ├── HelloWebApp
         │   └── config.xml

Add the following to the Dockerfile.

RUN mkdir -p $JENKINS_HOME/jobs/HelloWebApp
COPY HelloWebApp $JENKINS_HOME/jobs/HelloWebApp

RUN chmod +x $JENKINS_HOME/jobs/HelloWebApp \
  && chown -R ${user} $JENKINS_HOME/jobs/HelloWebApp


So let's build the Jenkins image and test locally.
Go to the folder where the Dockerfile exist and execute

docker build -t jenkins-alpine .

Run the Jenkins

docker run -p 80:8080 jenkins-alpine

You will see the Jenkins logs in the command line

You can access the Jenkins via http://localhost/ and see build jobs running in every 2 minutes when it detects any changes in GitHub project.

If you click on the HelloWebApp and go to configure, then you will see the Jenkins job configurations.



So now the image is ready and let's push it to the docker hub and deploy it in WSO2 Integration Cloud.

docker images

REPOSITORY               TAG                               IMAGE ID            CREATED             SIZE
jenkins-alpine                 latest                            d7dc03cec1df        51 minutes ago      257.4 MB

docker tag d7dc03cec1df amalkasubasinghe/jenkins-alpine:hellowebapp

docker login

docker push amalkasubasinghe/jenkins-alpine:hellowebapp

When you login to the docker hub you can see the image you push



Let's login to the WSO2 Integration Cloud -> Create Application -> and select Custom Docker Image


add the image providing image URL


Wait until the security scanning finished and then create the Jenkins application selecting scanned image



Here I select Custom Docker http-8080 and https-8443 runtime, as Jenkins run in 8080 port.


Wait until the Jenkins instance fully up and running. Check the logs


Now you can access the Jenkins UI via http://esbtenant1-jenkinshellowebapp-1-0-0.wso2apps.com/

That's all :). Now every 2 minutes our Jenkins job will poll the GitHub project and if there are any changes it will pull the changes and build.

This is how you can setup and configure Jenkins in WSO2 Integration Cloud.

You can see the docker file here https://github.com/amalkasubasinghe/docker/blob/jenkins-alpine-hellowebapp/Dockerfile






Wednesday, February 15, 2017

WSO2 ESB communication with WSO2 ESB Analytics

This blog post is about how & what ports involved when connecting from WSO2 ESB to WSO2 ESB Analytics.

How to configure: This document explains how to configure it
https://docs.wso2.com/display/ESB500/Prerequisites+to+Publish+Statistics

Let's say we have WSO2 ESB  and WSO2 ESB Analytics packs we want to run in same physical machine, then we have to offset one instance. 
But we don't want to do that since WSO2 ESB Analytics by default come with the offset.

So WSO2ESB will run on 9443 port, WSO2 ESB Analytics will run on 9444 port

WSO2 ESB publish data to the WSO2 ESB Analytics via thrift. By default thrift port is 7611 and corresponding ssl thrift port is 7711 (7611+100), check the data-bridge-config.xml file which is in analytics server config directory . 

Since we are shipping analytics products with offset 1 then thrift ports are 7612 and ssl port is 7712.
Here, ssl port (7712) is used for initial authentication purposes of data publisher afterwards it uses the thrift port (7612) for event publishing.. 

Here's a common error people raise when configuring analytics with WSO2 ESB.

[2017-02-14 19:42:56,477] ERROR - DataEndpointConnectionWorker Error while trying to connect to the endpoint. Cannot borrow client for ssl://localhost:7713
org.wso2.carbon.databridge.agent.exception.DataEndpointAuthenticationException: Cannot borrow client for ssl://localhost:7713
        at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:99)
        at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.run(DataEndpointConnectionWorker.java:42)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.wso2.carbon.databridge.agent.exception.DataEndpointSecurityException: Error while trying to connect to ssl://localhost:7713
        at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftSecureClientPoolFactory.createClient(ThriftSecureClientPoolFactory.java:61)
        at org.wso2.carbon.databridge.agent.client.AbstractClientPoolFactory.makeObject(AbstractClientPoolFactory.java:39)
        at org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1212)
        at org.wso2.carbon.databridge.agent.endpoint.DataEndpointConnectionWorker.connect(DataEndpointConnectionWorker.java:91)
        ... 6 more
Caused by: org.apache.thrift.transport.TTransportException: Could not connect to localhost on port 7714
        at org.apache.thrift.transport.TSSLTransportFactory.createClient(TSSLTransportFactory.java:237)
        at org.apache.thrift.transport.TSSLTransportFactory.getClientSocket(TSSLTransportFactory.java:169)
        at org.wso2.carbon.databridge.agent.endpoint.thrift.ThriftSecureClientPoolFactory.createClient(ThriftSecureClientPoolFactory.java:56)
        ... 9 more
Caused by: java.net.ConnectException: Connection refused: connect
        at java.net.DualStackPlainSocketImpl.connect0(Native Method)
        at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
        at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:427)
        at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:88)
        at org.apache.thrift.transport.TSSLTransportFactory.createClient(TSSLTransportFactory.java:233)
        ... 11 more

This comes because people change the thrift port comes in the following configuration files by adding another 1 (7612+1), thinking of we need to add 1, since we have offset in analytics server as 1.

<ESB_HOME>/repository/deployment/server/eventpublishers/MessageFlowConfigurationPublisher.xml
<ESB_HOME>/repository/deployment/server/eventpublishers/MessageFlowStatisticsPublisher.xml