Sunday, September 4, 2016

Add a new app type to the WSO2 App Cloud

This blog explain how you can add a new app type to the WSO2 App cloud.

Step 1: Get a clone of WSO2 app cloud code base
git clone https://github.com/wso2/app-cloud

Step 2: Create the docker files for the runtime
Please refer our existing docker files when creating new docker files for particular runtime.
https://github.com/wso2/app-cloud/tree/master/modules/resources/dockerfiles

Following blog post gives you some details about structure of the docker files
http://amalkas.blogspot.com/2016/09/add-new-runtime-to-existing-app-type-in.html

Step 3: Required database changes
When adding new app type you need add some database records, following diagram gives you an idea of database schema.

AC_CLOUD defines the cloud types
AC_APP_TYPE defines app types
AC_RUNTIME defines runtimes
AC_CONTAINER_SPECIFICATIONS defines container specs
AC_TRANSPORT defines the ports we expose for end users

-- insert app type
-- insert app type, cloud mapping
-- insert runtime
-- insert app type, runtime mapping
-- insert container spec if required
-- insert runtime, container spec mapping
-- insert transport if required
-- insert runtime, transport mapping


Step 4: Specify app-type meta data in app-types-properties.json file

This json file we used to load the app type details to the App Cloud UI.
Step 5: Add a sample
We need to add a sample to implement deploy sample option.

Commit your sample archive here:

Specify the sample location here with the property <app_type>_sample_artifact_url

Step 7: Implement endpoints section loading to the app home page
Please refer the followign blog post to see how we have developed the "Endpoints" section per app type.
http://amalkas.blogspot.com/2016/09/loading-endpoints-to-app-home-page-in.html

Loading endpoints to the app home page in WSO2 App Cloud

Currently we have 6 app types in WSO2 App Cloud and when we load a app home page of created applications we can see a section called "Endpoints". This blog post is about how we load endpoints per each and every app type and it's implementation

Loading endpoints for Java web application, Jaggery and PHP app types
for these 3 app types, user have to define the application context, when creating an application
as shown in below image.


Then we append that context to the app version url/default url and display in app home page as below.




Loading endpoints for Microservices app type
for microservices app type, microservices 2.0.0 itself provides a swagger url, we just display the swagger url here.



Loading endpoints for WSO2 dataservices app type
for dataservices app type, we need to invoke the ServiceAdmin soap service to get the endpoints. So we developed a axis2service and deployed in dss runtime, which invokes the ServiceAdmin and return the endpoints to the app cloud.

You can see the axis2Service here:
https://github.com/wso2/app-cloud/tree/master/modules/extensions/org.wso2.appcloud.dss.integration



Loading endpoints for WSO2 ESB app type
for ESB app type, we have developed an API and deployed in ESB runtime to get the SOAP and REST endpoints

You can see the code of the API here:
https://github.com/wso2/app-cloud/tree/master/modules/extensions/org.wso2.appcloud.esb.integration



Implementation
The implementation code we use to load endpoints per app types, you can find here.
https://github.com/wso2/app-cloud/blob/master/modules/jaggeryapps/appmgt/src/modules/application/endpoints.jag


per app type we need to define which mechanism we need to load to the app home page, that we have defined in app-types-properties.json file, as endpointExtractorObject property.

enabling or disabling appContext property, will show/hide Application Context field in Create Application page.

providing a value to the defaultContext will show a default context in Application Context field in Create Application page

Add a new runtime to a existing app type in WSO2 App Cloud

This blog post explains how you can add a new runtime to a existing app type. Recently WSO2 Carbon team released WSO2 AS 6.0.0 M3 release. I'm gonna explain how we can add this WSO2 AS 6.0.0 M3 as a new runtime of war app type.

Step 1: get a clone of app-cloud code base
git clone https://github.com/wso2/app-cloud


Step 2: Create required docker files
When creating docker files, please refer the existing docker files we have created for other runtimes and get an idea.

You can find the existing docker files here
https://github.com/wso2/app-cloud/tree/master/modules/resources/dockerfiles

You can find the wso2as docker files here. We need to add dockefiles related to WSO2AS 6.0.0 M3 dockerfiles here
https://github.com/wso2/app-cloud/tree/master/modules/resources/dockerfiles/wso2as



dockerfiles/wso2as/base 
This folder contains docker files required to build wso2as base images

dockerfiles/wso2as/default 
This folder contains docker files which require to build a images with wso2as base image and the war file we upload when we do create application.

dockerfiles/wso2as/url 
This folder contains docker files which require to build a images with wso2as base image and the war file we upload via url when we do create application.


Step 3: Database changes required
When adding a new runtime, you will need to update the database schema, following diagram explains the relationships with AC_RUNTIME table.

AC_RUNTIME defines the runtimes
AC_CONTAINER_SPECIFICATIONS defines the container specs we allow in our App Cloud setup
AC_TRANSPORT defines the ports we expose, to end users to access
AC_APP_TYPE defines the app types


Database queries required to add WSO2 AS 6.0.0 M3 runtime

-- add WSO2 AS 6.0.0 M3 runtime to the AC_RUNTIME table
INSERT INTO `AC_RUNTIME` (`id`, `name`, `repo_url`, `image_name`, `tag`, `description`) VALUES
(10, 'Apache Tomcat 8.0.28 / WSO2 Application Server 6.0.0-M3','registry.docker.appfactory.private.wso2.com:5000', 'wso2as', '6.0.0-m3', 'OS:Debian, JAVA Version:8u72');

-- add app type-runtime mapping
INSERT INTO `AC_APP_TYPE_RUNTIME` (`app_type_id`, `runtime_id`) VALUES
(1, 10);

-- add relavent container spec mappings
INSERT INTO `AC_RUNTIME_CONTAINER_SPECIFICATIONS` (`id`, `CON_SPEC_ID`) VALUES
(10, 3),
(10, 4);

-- add relavent transport mappings
INSERT INTO AC_RUNTIME_TRANSPORT (`transport_id`, `runtime_id`) VALUES
(3, 10),
(4, 10);

Step 4: integration above changes to App Cloud
Once you complete the above changes, you can send us a pull request :)


Generalising WSO2 App Cloud to Implement another cloud perspective (view)

Currently, WSO2 App Cloud follows the container based approach to provide different runtime to deploy application artifacts. Likewise we can follow the same approach to deploy ESB car file as a app type in WSO2 App Cloud. But, our requirement is to provide a separate cloud perspective (view) for end users for integration solutions, so we thought to generalize the app cloud to operate in two modes (as app cloud and integration cloud) on single app cloud deployment.



1. We need two different URLs (https://apps.cloud.wso2.com and https://integration.cloud.wso2.com ) to login to separate clouds.

How SSO works :
Currently, when a user login, it redirect to WSO2IS for SSO and then it comes to app cloud with https://apps.cloud.wso2.com url. Same as when user requests integration cloud it should redirect to WSO2IS and then it comes to the app cloud with https://integration.cloud.wso2.com url. For that we use 2 different issuers and 2 SPs configured in IS side. When the request first come to the app cloud, we find which cloud the user requests based on the host name, and then redirect the request to IS with correct issuer.

Loading relevant cloud view with valid titles, breadcrumbs, navigation buttons, etc ...:
Once the SSO happens we put the requested cloud to the session and then loads the UI with the correct UI elements reading the following json file.

{
    "app-cloud" : {
      "pageTitle": "WSO2 App Cloud",
      "cloudTitle" : "Application Cloud",
      "properties" : {
          "documentationUrl": "AppCloud.Documentation.Url",
          "supportUrl": "AppCloud.Support.Url”
      }
    },
    "integration-cloud" : {
      "pageTitle": "WSO2 Integration Cloud",
      "cloudTitle" : "Integration Cloud",
      "properties" : {
          "documentationUrl": "IntegrationCloud.Documentation.Url",
          "supportUrl": "IntegrationCloud.Support.Url”
      }
    }
}

2. Based on the selected cloud, app cloud should operate as follows.

- We want to differentiate app types per cloud.
- Application home page should list the application which was created in selected cloud.
- Application home page search function should work on application which was created in selected cloud only.
- Separate subscription plans required per cloud. [max number of applications and databases per cloud]
- Separate white listing required per cloud.

So we changed the app cloud database table structure as shown in below diagram and updated the implementation to get per cloud data.
With these changes we can deploy the app cloud as a separate deployment if required in future.




3. Unified UI design

Per app type, we will be require loading different UI components to the app home page.
As an example: How we display endpoints per app type. Different type of application provides different types of endpoints. ESB app types give SOAP and REST endpoints. Web/PHP gives just a web url. JAX-WS gives SOAP endpoint, etc…Likewise we will be required to add more UI components per app types. So we decided to go with unified UI design approach per app type with javascript abstraction layer. https://github.com/wso2/app-cloud/blob/master/modules/jaggeryapps/appmgt/src/modules/application/endpoints.jag

This is how we render endpoints per app type:
When user navigates to the app home page we make a call to the container and get the urls and generate the UI component to display in app home page.
We don’t persist these endpoints in database. So user can’t see the endpoints when the container is not up and running.