Friday, June 10, 2016

CI/CD with WSO2 App Cloud, Jenkins & GitHub

One way to improve a developer’s productivity is to automate repetitive tasks, such as building, committing, deploying, and testing. In this tutorial you will learn how to configure Jenkins and GitHub to achieve continuous integration and continuous deployment with WSO2 App Cloud. A basic knowledge in using Maven, Git, and invoking REST APIs is recommended to follow these steps.

Introduction

The day-to-day life of a developer often involves some mundane and repetitive tasks like build, commit, deploy, test, etc. With evolution in technology, however, we can automate these tasks to make a developer’s life little easier as well as improve productivity.
Testing and deployment are two integral elements of software development. With some automation, those elements become solutions referred to as ‘continuous integration’ (CI) and ‘continuous deployment’ (CD). The ‘continuous’ aspect of these solutions means that your projects will be automatically tested and deployed, allowing you to focus more on writing code and less on setting up servers.
In this tutorial, we'll set up a popular CI server called Jenkins and integrate it with GitHub so it will run tests every time the new code is pushed. Thereafter, we'll set up Jenkins to automatically deploy the built artifacts to our app server, which will eliminate the need to deploy archives manually.
Here, we will be using WSO2 App Cloud to deploy our application. It provides a hosting environment to deploy different types of applications, such as web, PHP, Jaggery, and uploading a built archive, so we can completely eliminate the need of setting up servers to deploy an application.

[Note: If you’re looking to deploy different types of apps other than those supported by WSO2 App Cloud, please contact the WSO2 App Cloud team and we will consider including that app type to our next releases].




Pre-requisites

  1. We need something to test and deploy. We will be using a web app called HelloWebApp. It’s just display Hello World! in the browser.
  2. To do the development you need a development machine with Maven, Java, and Git installed.
  3. We need a GitHub account https://github.com/ with the HelloWebApp source repository https://github.com/amalkasubasinghe/HelloWebApp.
  4. Jenkins installed server with the required Maven configurations and plugins (Git plugin and GitHub plugin) installed https://jenkins.io/.
  5. To deploy the archives, you need an account in WSO2 App Cloud http://wso2.com/cloud/app-cloud/.


Jenkins and GitHub integration

Here, when the developer pushes the code to the GitHub, we need to trigger a build in Jenkins and then Jenkins will checkout the code, build it, and run the tests. How do we trigger a build in Jenkins? We have two options:
  1. GitHub triggers Jenkins via GitHub webhook when the developer pushes the code to GitHub.
  2. Jenkins polls GitHub periodically to check whether there is a new code push; if yes, then trigger Jenkins by itself.
Here, we will configure Jenkins to poll GitHub every hour and build the code and run the tests.

  1. Go to Jenkins > New Item
  2. Give Item name and select Maven project              
Figure 1

3. Give Maven project name, select GitHub and give project URL. Since we’re building the master and working on 1.0-SNAPSHOT, we names the project “HelloWebApp-1.0-SNAPSHOT”



Figure 2

4. Configure the Source Code Management section as shown in the image specifying the GitHub URL and the branch to build. Then enable the Poll SCM in the Build Trigger section and schedule to poll every hour.      

        


Figure 3


5. Configure the build section to build the source using Maven pom.



Figure 4

Note that in the sample project, what has been written does not include any tests, but you can add your tests to the src/test/java location. Then when Jenkins triggers a build, it will execute all your tests after building the code.

Jenkins and WSO2 App Cloud integration

The next step is how to deploy the built archives in WSO2 App Cloud. First, we will see how we can do this manually using WSO2 App Cloud published REST API. Refer here for more information.

1. We need to login to WSO2 App Cloud and get the session cookie. When you access an API via external REST client, first you need to invoke the login API to ensure that the user is authenticated and then the system stores the generated session cookie in a file, which we can use in the next API invocations.

curl -v -k -c cookies -X POST -d 'action=login&userName=amalka.wso2.com@amalka&password=password' https://apps.cloud.wso2.com/appmgt/site/blocks/user/login/ajax/login.jag

Here my username is amalka@wso2.com and the tenant domain is amalka.

2. Then create a new application passing the stored session cookie in the previous step. It will create an application in WSO2 App Cloud uploading HelloWebApp.war file.

curl -v -k -b cookies -X POST https://apps.cloud.wso2.com/appmgt/site/blocks/application/application.jag -F action=createApplication -F applicationName=HelloWebApp -F applicationDescription=desc -F runtime=1 -F appTypeName=war -F applicationRevision=1.0.0 -F uploadedFileName=HelloWebApp.war -F runtimeProperties=[] -F tags=[]  -F fileupload=@/home/amalka/A/MKT/github/HelloWebApp/target/HelloWebApp.war -F isFileAttached=true -F conSpecCpu=300 -F conSpecMemory=512 -F isNewVersion=false

We can simply automate the above application deployment process by using Maven or by using a shell script to execute these curl commands. Here we’re configuring Maven to deploy an application in WSO2 App Cloud. Later we can configure Jenkins to deploy the application using this Maven config.

Please note the latest version of WSO2 App Cloud does not support updating the same version of an already deployed application with the new code changes. Therefore, when Jenkins builds a new artifact, we need to deploy it as a new version of the particular application. In order to clearly identify the application that was deployed using the artifact built by the particular Jenkins build, we thought we’d append the Jenkins build number to the application version, e.g. if the build number is 53, the application version will be 1.0.53.


[Note that the ability to update the same version of an already deployed application will be available with WSO2 App Cloud in future releases].



    <modelVersion>4.0.0</modelVersion>
    <groupId>com.wso2.appcloud</groupId>
    <artifactId>HelloWebApp</artifactId>
    <packaging>war</packaging>
    <version>1.0-SNAPSHOT</version>
    <name>HelloWebApp Maven Webapp</name>
    <url>http://maven.apache.org</url>
<dependencies>
    <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>3.8.1</version>
            <scope>test</scope>
    </dependency>
</dependencies>
<build>
    <plugins>
        <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.2</version>
                <executions>
                <execution>
                    <id>login</id>
                    <phase>deploy</phase>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                    <configuration>
                            <executable>curl</executable>
                            <arguments>
                                <argument>-v</argument>
                                <argument>-k</argument>
                                <argument>-c</argument>
                                <argument>cookies</argument>
                                <argument>-X</argument>
                                <argument>POST</argument>
                                <argument>-F</argument>
                                <argument>action=login</argument>
                                <argument>-F</argument>
                                <argument>userName=${appcloud.username}</argument>
                                <argument>-F</argument>
                                <argument>password=${appcloud.password}</argument>
                                <argument>https://apps.cloud.wso2.com/appmgt/site/blocks/user/login/ajax/login.jag</argument>
                            </arguments>
                    </configuration>
                </execution>
                <execution>
                    <id>create application</id>
                    <phase>deploy</phase>
                    <goals>
                        <goal>exec</goal>
                    </goals>
                    <configuration>
                            <executable>curl</executable>
                            <arguments>
                                <argument>-v</argument>
                                <argument>-k</argument>
                                <argument>-b</argument>
                                <argument>cookies</argument>
                                <argument>-X</argument>
                                <argument>POST</argument>
                                <argument>https://apps.cloud.wso2.com/appmgt/site/blocks/application/application.jag</argument>
                                <argument>-F</argument>
                                <argument>action=createApplication</argument>
                                <argument>-F</argument>
                                <argument>applicationName=${project.artifactId}</argument>
                                <argument>-F</argument>
                                <argument>applicationDescription=${project.artifactId}_${project.version}_${build.number}</argument>
                                <argument>-F</argument>
                                <argument>conSpecMemory=512</argument>
                                <argument>-F</argument>
                                <argument>conSpecCpu=300</argument>
                                <argument>-F</argument>
                                <argument>runtime=1</argument>
                                <argument>-F</argument>
                                <argument>appTypeName=war</argument>
                                <argument>-F</argument>
                                <argument>applicationRevision=${deploy.version}.${build.number}</argument>
                                <argument>-F</argument>
                                <argument>uploadedFileName=${project.artifactId}-${project.version}.war</argument>
                                <argument>-F</argument>
                                <argument>runtimeProperties=[]</argument>
                                <argument>-F</argument>
                                <argument>tags=[{"key":"BUILD_NUMBER","value":"${build.number}"}]</argument>
                                <argument>-F</argument>
                                <argument>fileupload=@${project.build.directory}/${project.artifactId}-${project.version}.war</argument>
                                <argument>-F</argument>
                                <argument>isFileAttached=true</argument>
                                <argument>-F</argument>
                                <argument>isNewVersion=true</argument>
                            </arguments>
                    </configuration>
                </execution>
                </executions>
        </plugin>
        <plugin>
                <artifactId>maven-deploy-plugin</artifactId>
                <version>2.7</version>
                <configuration>
                    <skip>true</skip>
                </configuration>
        </plugin>
    </plugins>
</build>
<properties>
        <deploy.version>1.0</deploy.version>
        <build.number>0</build.number>   
        <appcloud.password>password</appcloud.password>
        <appcloud.username>username</appcloud.username>
</properties>
</project>



  • In the above config, I have used Maven exec plugin to configure login and create application curl commands
  • Here, I have set the phase of the execution as deploy, since we don’t need to execute this commands every time we build the code (mvn clean install). So then login and create application commands will only execute when we run mvn deploy.
  • IBut when running mvn deploy will deploy the artifacts in nexus repo, that I have skipped adding following configuration to the Maven configuration
<plugin>
                <artifactId>maven-deploy-plugin</artifactId>
                <version>2.7</version>
                <configuration>
                    <skip>true</skip>
                </configuration>
</plugin>



  • In the login command we need to pass the app cloud credentials to get the session cookie, but we can’t hard code those values in the Maven config since this code is publicly available in GitHub. Therefore, we’ve made those values configurable and we can set a username and password when running mvn deploy.
  • In create application command we have set the application revision appending the Jenkins build number as explained earlier; we need to pass that too when running mvn deploy.

  • <argument>applicationRevision=${deploy.version}.${build.number}</argument>

    We have set isNewVersion=true in create application command because each time we want to create a new version of the already deployed application. 

    <argument>isNewVersion=true</argument>

    We can set the app version specific information to the application via tags. We have set the Jenkins build number in this Maven config. 

    <argument>tags=[{"key":"BUILD_NUMBER","value":"${build.number}"}]</argument>

    Here is the command we need to execute to deploy an application in WSO2 App Cloud. We can pass the Jenkins build number as -Dbuild.number=${BUILD_NUMBER}

    mvn clean deploy -Dbuild.number=${BUILD_NUMBER} -Dappcloud.password=xxxxx -Dappcloud.username=amalka.wso2.com@amalka   

    Then you need to update the Jenkins config as follows. Update the build section with a new Maven command. Since Jenkins is not publicly accessible, we can configure our real credentials here. 

      


    Figure 5

    We have now successfully updated Maven configs and Jenkins to work with WSO2 App Cloud. Every hour, Jenkins will poll GitHub, and if there a new code push, Jenkins will check the code, build, and deploy a new version of the HelloWebApp application to WSO2 App Cloud.

    Working with WSO2 App Cloud

    Jenkins always creates a new version of HelloWebApp application. Therefore, before we run the first Jenkins build, we need to deploy the first version of the HelloWebApp in WSO2 App Cloud.

    Login to WSO2 App Cloud and create the first version of the HelloWebApp application providing the following details.
    Application Name: HelloWebApp
    Application Version: 1.0.0

    Build the HelloWebApp code locally and upload the HelloWebApp-1.0-SNAPSHOT.war


    Figure 6

    It will give you the Launch URL as http://amalka-hellowebapp-1-0-0.wso2apps.com/ and display Hello World in the browser.


    Figure 7

    Then add more content to the HelloWebApp code and push it to GitHub; then the next Jenkins job will deploy the updated artifact in WSO2 App Cloud as the new version 1.0.1.

    http://amalka-hellowebapp-1-0-1.wso2apps.com/



    Figure 8

    When the new app version is deployed in app cloud, the previous version will stop running. If you want to access the previous version, then you have to manually start the container and the currently running version will stop automatically. This is because WSO2 App Cloud allows to run only one version of the particular application at a time.

    [Please note that running number of versions of a particular application at once will be available in future releases].

    When you go to the HelloWebApp overview page you can see the deployed version of the HelloWebApp app as follows.



    Figure 9

    When you go a particular version you can see the tags we have set as shown in the image below.


    Figure 10

    Conclusion

    WSO2 App Cloud offers a readily available hosting environment to deploy different types of applications like web, PHP, MS4J, Jaggery; many more application types will be available in the future as well. The step-by-step instructions in this tutorial would have helped you to set up CI/CD in your environment with integration of Jenkins, GitHub, and WSO2 App Cloud.

    References

       

    No comments: