Continuous Integration / Continuous Deployment
Lesson 1: What Is CI/CD?
Continuous integration (CI) and continuous delivery (CD) are a pair of practices that, when done properly, greatly aid an organization's ability to deliver software frequently and reliably to its customers. We often hear the combination of these practices referred to as "the CI/CD pipeline," and they are among the most important practices a Software Engineering team can implement. So what do these practices consist of?
CI : Continuous Integration
Continuous Integration means that every developer should be pushing code to the master branch regularly, at the minimum once per day, and that then that code should be tested to ensure that it integrates with all the other code currently in the master branch. The practice of continuous integration eliminates the huge "integration" phase that typically took place in waterfall model projects, where developers would work on isolated pockets of code for months or even a year or more, and only try to join their code to other teams' code at the very end of the project.
On a push to the master branch, various processes can be launched, such as:
- Running automatic code quality tests.
- Building an executable from the code.
- Reporting on code testing coverage.
- Building a docker container that can be used to deploy the code.
These simple additions allows the developer to focus on writing the code. The central repository of code is there to receive your changes while automated processes can build, test, and scan the code while providing reports.
CD : Continuous Delivery
Continuous delivery means taking the result of the continuous integration process and delivering it to a "place" in which it can be run. That "place" my be a test server, a staging server, or even production. If the continuous delivery phase is actually putting the new software right into production, then it also qualifies as "continuous deployment." It almost goes without saying that only teams experienced with CI/CD should extend that practice to continuous deployment.
There are many free tools to aid in CI/CD, such as Travis CI and Jenkins, both of which we will discuss below.
- https://www.infoworld.com/article/3271126/what-is-cicd- continuous-integration-and-continuous-delivery-explained.html
- https://www.atlassian.com/continuous-delivery/principles /continuous-integration-vs-delivery-vs-deployment
Lesson 2: Jenkins
Jenkins is an open source automation tool written in Java. It aids the adoption of CI/CD because through WebHooks, Jenkins can be triggered by, for instance, a push to GitHub. Upon being triggered, Jenkins can directly, or through plugins, run a wide variety of processes that can test, package, document, analyze and deploy your application.
Advantages of Jenkins
- It is an open source tool with great community support.
- It has 1000+ plugins to ease your work. If a plugin does not exist, you can code it and share with the community.
- It is free.
- It is built with Java and hence, it is portable to all major platforms.
Jenkins on Wikipedia.
Lesson 3: Travis-CI
Travis CI is a relatively easy-to-setup CI/CD tool. It relies on a YAML file for its configuration, and uses a Webhook to tie to GitHub.
The Travis YAML file (.travis.yml) contains a number of directives as to how the build should be performed. The directives can specify languages to install (e.g., Python 3.6), services to employ (e.g., docker), set environment variables (such as PYTHONPATH), a main script to run (e.g., make tests), an install section to install things like your test tools, a service to notify about the state of the build (such as Slack), addons (such as adding a known host to SSH), and things to do if the build succeeded or if it failed. The YAML file is an example of declarative programming.
Here is the Travis configuration file for this website:
openssl aes-256-cbc -K $encrypted_4cefc8a5b2ff_key -iv $encrypted_4cefc8a5b2ff_iv -in id_rsa.enc -out ~/.ssh/id_rsa -d
- pip install -r docker/requirements-dev.txt
- make tests
Implementing Travis Deploy to PythonAnywhere
After the Travis YAML file is written, once it is made sure that all the tests are passing, the next step would be to start a cloud deployment. This website is deployed to PythonAnywhere.
For any deployment, one of the ways Travis would establish a connection with the cloud is SSH. But, for an automated deployment by Travis, there is a need for a non interactive SSH connection. One of the ways to set up such a connection is to use a package called SSHPass. This package provides a way to pass the password to the ssh command.
Here are the steps to do that:
In the Travis YAML file add the following
lines to before_install:
before_install: - sudo apt-get update - sudo apt-get install sshpass
- The password to ssh should not be present in any of the scripts. Luckily, there is a way to encript environment variables on Travis. So, the ssh password should be added as an encrypted environment variable (say "pa-pwd") on Travis.(This can be done on the settings tab of Travis.)
In the script, where the ssh command is used
pass the ssh password from the Travis
environment variable set in the previous
step by adding the sshpass command as follows:
sshpass -p $pa_pwd ssh $email@example.com
Lesson 4: Running Jenkins
Note: Below information is about making a Jenkins build triggered by Github changes.
Install the prerequisites
- Install JAVA if you don't already have it !
- Install Git if you don't already have it!
- Install Docker (optional)
Create a git repo to test the demo. It
should consist of:
Install and initialize Jenkins
Jenkins can be installed on various platforms and operating systems. For this demonstration we are using Docker to install Jenkins.
There are several Docker images of Jenkins available. The recommended Docker image to use is the jenkinsci/blueocean image (from the Docker Hub repository). This image contains the current Long-Term Support (LTS) release of Jenkins (which is production-ready) bundled with all Blue Ocean plugins and features.
After downloading the jenkins image you can use below docker command to install and run jenkins on docker.
For macOS/Linux : Open terminal and run the image as container in Docker using below command
docker run \ -u root \ --rm \ -d \ -p 8080:8080 \ -p 50000:50000 \ -v jenkins-data:/var/jenkins_home \ -v /var/run/docker.sock:/var/run/docker.sock \ jenkinsci/blueocean
For Windows : Open command prompt and run below command
docker run ^ -u root ^ --rm ^ -d ^ -p 8080:8080 ^ -p 50000:50000 ^ -v jenkins-data:/var/jenkins_home ^ -v /var/run/docker.sock:/var/run/docker.sock ^ jenkinsci/blueocean
Connect Jenkins to Github
Create a project to build your application
- What Is Continuous Delivery?
- Continuous Integration , and the dangers of branching!
- Don't branch for more than a day!
- The Death of the Release
- A Beginner's Guide To Building DevOps Pipelines With Open Source Tools
- Setting up Jenkins as a webhook in GitHub
- Slack notification plugin integration with Jenkins
- Testing and Continuous Integration Part 1
- Testing and Continuous Integration Part 2