Automate all the things with Jenkins! (II): Create a pipeline to create Docker images

Maybe you have your own Jenkins Server by now. Great! Now, why don’t start automating all the things?

As Sysadmins we want all our proceses running flawlessly when needed, and in an automated way if possible/applicable. Creating, modifying and maintaining a Docker image can be quite a repetitive task, so we want to automate it at all costs. Think about security patches for example!

About this project

Docker has a nice container to test that Docker works, that is, the whalesay container. Basically is an implementation of cowsay (a popular perl script) that replaces the cow template with a whale one. What happens is that there is no raspberry official equivalent image, and the x86 image uses Ubuntu as source image, so the resulting docker image is huge (more than 200MB to just say Hello World…). So we will create an arm image for raspberry pi based on Alpine that does the whalesay thing. You can check this project on github (Dockerfile here). This article is based on this entry from getintodevops.com

For the Jenkins pipeline we have created a file (written in groove) that describes all the steps (stages) and operations that the pipeline will perform in order to carry out the job. In this file we are defining 4 stages:

  • Clone Repository: We will get the source code from git
  • Build Image: We will do the “Docker Build” operation over the source code
  • Test Image: We will test the created image
  • Push Image: We will push the created image to Docker Hub

For this last step we need the Docker Hub credentials on Jenkins to be able to perform the Docker Push operation.

Configuring Docker Hub With Jenkins

To store the Docker image resulting from our build, we’ll be using Docker Hub. You can sign up for a free account at https://hub.docker.com.

Once we have an account created in Docker Hub, we will put it on  Credentials -> System -> Global credentials -> Add Credentials.

Add your Docker Hub credentials as the type Username with password, with the ID docker-hub-credentials. The ID field is how we will refer to this entry in our script.

The Jenkinsfile

We’ll need to give Jenkins access to push the image to Docker Hub. For this, we’ll create Credentials in Jenkins, and refer to them in the Jenkinsfile as ‘docker-hub-credentials’.

node {
 def app

stage('Clone repository') {
 /* Ensure we can clone the repository */
 checkout scm
 }

stage('Build image') {
 /* This builds the actual image; synonymous to
 * docker build on the command line */
 app = docker.build("stealthizer/rpi-whalesay")
 }

stage('Test image') {
 /* No real tests so it just passes this step always OK */
 sh 'echo "Tests passed"'
 }

stage('Push image') {
 /* Finally, we'll push the image with two tags:
 * First, a short hash that identifies the commit in git
 * Second, the 'latest' tag.
 * Pushing multiple tags is cheap, as all the layers are reused. */
 sh "git rev-parse --short HEAD > .git/commit-id"
 def commit_id = readFile('.git/commit-id').trim()
 docker.withRegistry('https://registry.hub.docker.com', 'docker-hub-credentials') {
 app.push("${commit_id}")
 app.push("latest")
 }
 }
}

As you might have noticed in the above Jenkinsfile, we’re using docker.withRegistry to wrap the app.push commands – this instructs Jenkins to log in to a specified registry with the specified credential id (docker-hub-credentials).

The Jenkins Pipeline

Now, we will create the pipeline, that is, an automated expression of the task you want to perform from your code you have written to the very final delivery step (create the image, or put something into production, for example). This will be your contribution the continuous delivery processes.

To create the pipeline we will add a New Item -> Pipeline. We will customize the next field properties:

  • Description: Free text to describe your pipeline project
  • Github Project: the url of the project you are going to build automaticallyI’ll be using https://github.com/stealthizer/rpi-whalesay/.
  • Definition: On the pipeline part of this config page we will choose “pipeline script from SCM”, specify Git as our SCM source, put again the git project url as Repository url (no credentials needed if the repository is public), specify the branch you want to build (*/master can be a good start) and as script path we will just put Jenkinsfile.

Once we press on save we are finished creating out pipeline.

To test it we just press the Build option inside the job and see if all the stages finish with a Success state.

jenkins_stage_view

Automate all the things with Jenkins! (I)

Automate all the Things!

It is normal that when we develop something we need to repeat some steps over and over again, like testing the code, compiling the binaries or create a new version of a docker image. To help us this way we will be using Jenkins, a continuous integration server. On our Raspberry, of course!

I’ve found that in a fresh recent Raspbian installations some steps are needed to make it work. The oracle-java8-jdk that the apt-get command installs is too old (1.8.0_65), leading to some problems, beginning in the installation itself. The solution found in stackexchange needed an additional step as the apt-key command was not working due to a problem with dirmngr not installed (solution found here)

We will begin to prepare the system to install Jenkins. These instructions are performed on a fresh Raspbian installation with no other java installed. We will first get a key from the Ubuntu keyserver:

sudo apt-get install dirmngr
sudo apt-key adv --recv-key --keyserver keyserver.ubuntu.com EEA14886

We will edit the packet source file /etc/apt/sources.list to add the following lines:

deb http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main
deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu trusty main

So we can proceed to install Java 8

sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt-get install oracle-java8-set-default

To check the version we have installed we can execute:

$ java -version
java version "1.8.0_151"
Java(TM) SE Runtime Environment (build 1.8.0_151-b12)
Java HotSpot(TM) Client VM (build 25.151-b12, mixed mode)

At least the java version must be >= 1.8.0_101, or this like this may happen.

To proceed with the Jenkins installation itself:

wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

We will wait until the service has started at port 8080. You will be prompted to unlock your installation by providing a secret initial password located at /var/lib/jenkins/secrets/initialAdminPassword.  You will be prompted to select some plugins (the suggested ones are ok to begin with) and then you will be prompted to create the admin user for this new Jenkins.

And that’s it! Soon we will be automating the creation of our docker images. Stay tuned!