Continuous Integration / Continuous Deployment

Martin Fowler on Continuous Delivery

Lesson 1: What Is CI/CD?

Continuous integration (CI) and continuous delivery (CD) are a pair of practices that, when done properly, greatly aid an organization's ability to deliver software frequently and reliably to its customers. We often hear the combination of these practices referred to as "the CI/CD pipeline," and they are among the most important practices a DevOps team can implement. So what do these practices consist of?

CI : Continuous Integration

Continuous Integration means that every developer should be pushing code to the master branch regularly, at the minimum once per day, and that then that code should be tested to ensure that it integrates with all the other code currently in the master branch. The practice of continuous integration eliminates the huge "integration" phase that typically took place in waterfall model projects, where developers would work on isolated pockets of code for months or even a year or more, and only try to join their code to other teams' code at the very end of the project.

On a push to the master branch, various processes can be launched, such as:

  • Running automatic code quality tests.
  • Building an executable from the code.
  • Reporting on code testing coverage.
  • Building a docker container that can be used to deploy the code.

These simple additions allows the developer to focus on writing the code. The central repository of code is there to receive your changes while automated processes can build, test, and scan the code while providing reports.

CD : Continuous Delivery

Continuous delivery means taking the result of the continuous integration process and delivering it to a "place" in which it can be run. That "place" my be a test server, a staging server, or even production. If the continuous delivery phase is actually putting the new software right into production, then it also qualifies as "continuous deployment." It almost goes without saying that only teams experienced with CI/CD should extend that practice to continuous deployment.

There are many free tools to aid in CI/CD, such as Travis CI and Jenkins, both of which we will discuss below.

References :

  • https://www.infoworld.com/article/3271126/what-is-cicd- continuous-integration-and-continuous-delivery-explained.html
  • https://www.atlassian.com/continuous-delivery/principles /continuous-integration-vs-delivery-vs-deployment
  • https://www.mabl.com/blog/what-is-cicd
Lesson 2: Jenkins

Jenkins is an open source automation tool written in Java. It aids the adoption of CI/CD because through WebHooks, Jenkins can be triggered by, for instance, a push to GitHub. Upon being triggered, Jenkins can directly, or through plugins, run a wide variety of processes that can test, package, document, analyze and deploy your application.

Advantages of Jenkins
  • It is an open source tool with great community support.
  • It has 1000+ plugins to ease your work. If a plugin does not exist, you can code it and share with the community.
  • It is free.
  • It is built with Java and hence, it is portable to all major platforms.
Introduction to Jenkins by Denis Petelin

Jenkins on Wikipedia.

References :

Lesson 3: Travis-CI

Travis CI is a relatively easy-to-setup CI/CD tool. It relies on a YAML file for its configuration, and uses a Webhook to tie to GitHub.

The Travis YAML file (.travis.yml) contains a number of directives as to how the build should be performed. The directives can specify languages to install (e.g., Python 3.6), services to employ (e.g., docker), set environment variables (such as PYTHONPATH), a main script to run (e.g., make tests), an install section to install things like your test tools, a service to notify about the state of the build (such as Slack), addons (such as adding a known host to SSH), and things to do if the build succeeded or if it failed. The YAML file is an example of declarative programming.

Here is the Travis configuration file for this website:

    language: python
    python:
        - '3.6'
    
    services:
        - docker
    
    addons:
        ssh_known_hosts: ssh.pythonanywhere.com
    
    before_install:
        openssl aes-256-cbc -K $encrypted_4cefc8a5b2ff_key -iv $encrypted_4cefc8a5b2ff_iv -in id_rsa.enc -out ~/.ssh/id_rsa -d
    
    install:
        - pip install -r docker/requirements-dev.txt
    
    env:
        - PYTHONPATH=/home/travis/build/gcallah/OnlineDevops
    
    script:
        - make tests
    
    after_success:
        - ./deploy.sh
    
    notifications:
      slack: devopsnyu:vAFcJBgAP2f73SqhWgb40Gjf
                

Implementing Travis Deploy to PythonAnywhere

After the Travis YAML file is written, once it is made sure that all the tests are passing, the next step would be to start a cloud deployment. This website is deployed to PythonAnywhere.

For any deployment, one of the ways Travis would establish a connection with the cloud is SSH. But, for an automated deployment by Travis, there is a need for a non interactive SSH connection. One of the ways to set up such a connection is to use a package called SSHPass. This package provides a way to pass the password to the ssh command.

Here are the steps to do that:

  • In the Travis YAML file add the following lines to before_install:

        before_install:
          - sudo apt-get update
          - sudo apt-get install sshpass
                                        

    The first line updates apt-get, and the second line installs the sshpass package on Travis
  • The password to ssh should not be present in any of the scripts. Luckily, there is a way to encript environment variables on Travis. So, the ssh password should be added as an encrypted environment variable (say "pa-pwd") on Travis.(This can be done on the settings tab of Travis.)
  • In the script, where the ssh command is used (say deploy.sh), pass the ssh password from the Travis environment variable set in the previous step by adding the sshpass command as follows:

        sshpass -p $pa_pwd ssh $project_name@ssh.pythonanywhere.com
                                        

    Note: $pa_pwd is available in .travis.yml file. So, if the script " deploy.sh" is being called from .travis.yml file, then make sure that $pa_pwd is passed as an argument to " deploy.sh", which can be used along with the sshpass command.

Lesson 4: Running Jenkins

Note: Below information is about making a Jenkins build triggered by Github changes.

Install the prerequisites
  • Install JAVA if you don't already have it !
  • Install Git if you don't already have it!
  • Install Docker (optional)
  • Create a git repo to test the demo. It should consist of:

      Readme.md

      Jenkinsfile

Install and initialize Jenkins

Jenkins can be installed on various platforms and operating systems. For this demonstration we are using Docker to install Jenkins.

There are several Docker images of Jenkins available. The recommended Docker image to use is the jenkinsci/blueocean image (from the Docker Hub repository). This image contains the current Long-Term Support (LTS) release of Jenkins (which is production-ready) bundled with all Blue Ocean plugins and features.

After downloading the jenkins image you can use below docker command to install and run jenkins on docker.

For macOS/Linux : Open terminal and run the image as container in Docker using below command


                                docker run \
                                -u root \
                                --rm \
                                -d \
                                -p 8080:8080 \
                                -p 50000:50000 \
                                -v jenkins-data:/var/jenkins_home \
                                -v /var/run/docker.sock:/var/run/docker.sock \
                                jenkinsci/blueocean
                            

For Windows : Open command prompt and run below command


                                docker run ^
                                -u root ^
                                --rm ^
                                -d ^
                                -p 8080:8080 ^
                                -p 50000:50000 ^
                                -v jenkins-data:/var/jenkins_home ^
                                -v /var/run/docker.sock:/var/run/docker.sock ^
                                jenkinsci/blueocean
                            

Connect Jenkins to Github
Create a project to build your application

References :

  • https://www.mirantis.com/blog/intro-to-cicd-how- to-make-jenkins-build-on-github-changes/
  • Other Material
    Jez Humble on Continuous Delivery
    What Is Jenkins?
    Quiz

      In Travis if we want to send a message to Slack about a build, it should go in the section called:

      1. add_ons
      2. authorizations
      3. nullifications
      4. notifications

      Travis CI can...?

      1. hook to GitHub and watch your repo
      2. connect to Slack
      3. build your test environment and run your tests
      4. all of the above

      One common way to trigger a Jenkins build is....?

      1. type some new code in an IDE
      2. run make
      3. run Docker
      4. push a commit in a version control system

      A declarative language focuses on...?

      1. saying what needs to be done rather than how it needs to be done
      2. the exact steps necessary to carry out an installation
      3. pure mathematical functions
      4. all of the above

      In Travis, we should deploy our code to the end users in a section called:

      1. before_install
      2. addons
      3. after_success
      4. services

      If we need to set a path that Travis will use in our tests, it should go in a section called:

      1. env
      2. add_ons
      3. services
      4. install

      Jenkins uses a __________ architecture?

      1. master-master
      2. master-agent
      3. peer-to-peer
      4. monolithic

      The scripting language for Jenkins is called...?

      1. Groovy
      2. Movie
      3. Groupie
      4. Snoopy

      Configuration languages are often examples of...?

      1. functional programming
      2. object-oriented programming
      3. procedural programming
      4. declarative programming