Tuesday, July 19, 2016

Create Application in WSO2 App Cloud via REST API

This blog post gives you some examples on how to use createApplication REST API correctly in WSO2 App Cloud https://docs.wso2.com/display/AppCloud/Published+APIs

Before invoke createApplication API, you need to login to the WSO2 App Cloud and get the session cookie.
curl -c cookies -v -k -X POST -k https://apps.cloud.wso2.com/appmgt/site/blocks/user/login/ajax/login.jag -d 'action=login&userName=amalka.wso2.com@<tenant>&password=<password>’

This are some basic example to create applications in WSO2 App Cloud.

Create application via upload from file system

curl -v -k -b cookies -X POST https://apps.cloud.wso2.com/appmgt/site/blocks/application/application.jag -F action=createApplication -F applicationName=SampleFile -F applicationDescription=desc -F runtime=1 -F appTypeName=war -F applicationRevision=1.0.0 -F uploadedFileName=sample.war -F runtimeProperties=[] -F tags=[]  -F fileupload=@/home/amalka/Downloads/sample.war -F isFileAttached=true -F conSpec=3 -F isNewVersion=false -F appCreationMethod=default

Create a version of the application created above

curl -v -k -b cookies -X POST https://apps.cloud.wso2.com/appmgt/site/blocks/application/application.jag -F action=createApplication -F applicationName=SampleFile -F applicationDescription=desc -F runtime=1 -F appTypeName=war -F applicationRevision=2.0.0 -F uploadedFileName=sample.war -F runtimeProperties=[] -F tags=[]  -F fileupload=@/home/amalka/Downloads/sample.war -F isFileAttached=true -F conSpec=3 -F isNewVersion=true -F appCreationMethod=default

Create application via upload from url

curl -v -k -b cookies -X POST https://apps.cloud.wso2.com/appmgt/site/blocks/application/application.jag -F action=createApplication -F applicationName=SampleURL -F applicationDescription=desc -F runtime=1 -F appTypeName=war -F applicationRevision=1.0.0 -F uploadedFileName=war_sample.war -F runtimeProperties=[] -F tags=[]  -F artifactUrl=https://github.com/wso2/app-cloud/raw/master/samples/artifacts/war_sample.war -F conSpec=3 -F isNewVersion=false -F appCreationMethod=url

Create application via clone GitHub repository

curl -v -k -b cookies -X POST https://apps.cloud.wso2.com/appmgt/site/blocks/application/application.jag -F action=createApplication -F applicationName=SampleGitHub -F applicationDescription=desc -F runtime=5 -F appTypeName=jaggery -F applicationRevision=1.0.0 -F runtimeProperties=[] -F tags=[]  -F conSpec=3 -F isNewVersion=false -F appCreationMethod=github -F gitRepoUrl=https://github.com/SabraO/DSSample.git -F gitRepoBranch=master -F projectRoot=/JaggerySample



action=createApplication
The action we are calling
applicationName=
Valid characters a-z A-Z 0-9 -
Space are not allowed
Numbers only names not allowed
Not case sensitive
applicationDescription=
Give a description for the application
appTypeName=
values [war, mss, php, jaggery, wso2dataservice]
runtime=
values [1, 2, 3, 4, 5, 6, 7]
applicationRevision=
format major.minor.patch
uploadedFileName=
Name of the file uploading
runtimeProperties=
Define runtime properties
[{"key":"enableAnalytics","value":"true"}]
tags=
Define tags
[{"key":"lifecycle","value":"development"}]
fileupload=
should starts with @
isFileAttached=
values [true, false]
conSpec=
values [1, 2, 3, 4, 5, 6, 7, 8]
isNewVersion=
values [true, false]
appCreationMethod=
values [default, url, github]
artifactUrl=
Specify url to download the artifact
gitRepoUrl=
Specify GitHub repository url
gitRepoBranch=
Specify GitHub repository branch
projectRoot=
Specify project root
applicationContext=
Specify application context.


Currently, WSO2 App Cloud supports number of app types, runtimes and container specifications. Following tables will help you to identify relevant runtimes, container specifications, supported app creation methods per app types.

Supported runtimes and app creation methods per app types


App type
Supported Runtime
Supported App creation method
1
war
1, 6
default, url
2
mss
2, 8
default, url
3
php
3
default, url, github
4
jaggery
5
default, url, github
5
wso2dataservice
7
default, url

Supported container specifications per runtimes


Runtimes
Supported Container specs
1
Apache Tomcat 8.0.28 / WSO2 Application Server 6.0.0-M1
3, 4
2
OpenJDK 8 + WSO2 MSF4J 1.0.0
2, 3, 4
3
Apache 2.4.10  
1, 2, 3
4
Carbon 4.2.0      

5
Jaggery 0.11.0
3, 4
6
Apache Tomcat 8.0.28 / WSO2 Application Server 6.0.0-M2
3, 4
7
WSO2 Data Services Server - 3.5.0
3, 4
8
OpenJDK 8 + WSO2 MSF4J 2.0.0
2, 3, 4

Container Specifications

Spec Id
Spec Name
CPU
Memory
1
128MB RAM and 0.1x vCPU
100
128
2
256MB RAM and 0.2x vCPU
200
256
3
512MB RAM and 0.3x vCPU
300
512
4
1024MB RAM and 0.5x vCPU
500
1024



Sunday, July 17, 2016

Log user events to audit.log file via Jaggery block layer

Let's say I have a Jaggery application deployed in WSO2 AS, I want to log the user events to audit.log file via Jaggery block layer.

We can log adding following code:

var audit = org.wso2.carbon.CarbonConstants.AUDIT_LOG;
audit.info("log message");

WSO2 App Cloud Architecture Diagram



App Cloud SaaS application provides an user interface & REST API for app cloud users to deploy their application artifacts. The web UI is developed by Jaggery, and it invokes the REST API, which invokes following backend components to provide the app cloud solution.


Docker Client provides an interface to build images and push to the docker registry.
Kubernetes Client provides an interface to deploy applications in kubernetes cluster
DAO provides an interface to manipulate database operations to store meta data required for App Cloud in App Cloud DB
SOAP Client uses to invoke WSO2 Storage Server admin services to create databases and users


WSO2 Application Server provides an hosting environment to deploy App Cloud SaaS application


WSO2 Identity Server provides identity management configuring SSO with App Cloud SaaS application


WSO2 Storage Server provides RSS instances for app cloud developers to store application data.


WSO2 Data Analytics Server collects statistics published by deployed applications and provides dashboards to the app cloud users.


Docker Registry uses to store application images created using the deployable artifacts and runtimes.


Kubernetes provides runtime for deploy applications. Kubernetes namespaces provides tenant isolation and per pod per container per application will be deployed


End users will access the deployed applications via HAProxy. Further it provides Default URL and Custom URL features apart from load balancing.

How commonauth service work with SSO in WSO2 Identity Server

Considering two SAML service providers configured under the same tenant, they are by default in single sing on, so after I login in Service provider A I can use the same browser to access Service provider B without inserting my credentials again. This blog explains in details how this happened with commonauth service during the second login

What happens here is:

On the first login to service provider A, it stores a cookie with a name "commonAuthId"
When the first request comes; DefaultRequestCoordinator.handle() [1] method invokes DefaultAuthenticationRequestHandler.handle() method, see [2]
within the DefaultAuthenticationRequestHandler.handle() it invokes concludeFlow() private method [3], that sets the 'commonAuthId' cookie via setAuthCookie() method [4]

private void setAuthCookie(HttpServletRequest request, HttpServletResponse response, AuthenticationContext context,
                               String sessionKey, String tenantDomain) throws FrameworkException {
        Integer authCookieAge = null;

        if (context.isRememberMe()) {
            authCookieAge = IdPManagementUtil.getRememberMeTimeout(tenantDomain);
        }

        FrameworkUtils.storeAuthCookie(request, response, sessionKey, authCookieAge);
    }

FrameworkUtils.storeAuthCookie() method

public static void storeAuthCookie(HttpServletRequest req, HttpServletResponse resp, String id, Integer age) {

        Cookie authCookie = new Cookie(FrameworkConstants.COMMONAUTH_COOKIE, id);
        authCookie.setSecure(true);
        authCookie.setHttpOnly(true);
        authCookie.setPath("/");

        if (age != null) {
            authCookie.setMaxAge(age.intValue() * 60);
        }

        resp.addCookie(authCookie);
    }


[1] https://github.com/wso2/carbon-identity/blob/v5.0.7/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/handler/request/impl/DefaultRequestCoordinator.java#L80

[2] https://github.com/wso2/carbon-identity/blob/v5.0.7/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/handler/request/impl/DefaultRequestCoordinator.java#L135

[3] https://github.com/wso2/carbon-identity/blob/v5.0.7/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/handler/request/impl/DefaultAuthenticationRequestHandler.java#L120

[4] https://github.com/wso2/carbon-identity/blob/v5.0.7/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/handler/request/impl/DefaultAuthenticationRequestHandler.java#L284

Then, when we access the service provider B, it checks whether the "commonAuthId" is available in cookie list, if yes, then it gets the authentication details from SessionContext and by pass the authentication step.
see [5] findPreviousAuthenticatedSession() method

[5]https://github.com/wso2/carbon-identity/blob/v5.0.7/components/authentication-framework/org.wso2.carbon.identity.application.authentication.framework/src/main/java/org/wso2/carbon/identity/application/authentication/framework/handler/request/impl/DefaultRequestCoordinator.java

Wednesday, July 13, 2016

Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources

Sometimes you might see this WARN log continuously in DAS analyser nodes, This happened, the __spark_meta_table keeps some meta data info about the CAR files we deploy. When those information got corrupted, you can see this WARN log in console.

"Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources"

This is a known issue in spark and PR is already given to the spark.

As a workaround we can remove the __spark_meta_table table and restart the node.
Since the  __spark_meta_table table is encrypted, we just can't delete it from database, we have to use the Analytics data backup and restore tool