Tutorial: Integrating Gitlab and Jenkins

This is a short tutorial showing how to integrate Jenkins and Gitlab.

There are many uses cases and customized development processes established across teams. For the sake of the discussion, I will reference the following development process that could be used in those environments with high focus in having high quality code on the main branch at all the times.

Typical Development Process

Usually the workflow starts as soon as a developer creates or updates a Pull Request. Some people set up processes by running the build, test & deployment from the feature branch. If all tests are passed, the Pull Request is considered a good candidate to be merged with the main branch.

Although it can work in many cases, this approach has a potential flaw since it could happen that after the Pull Request is merged, the new version on the main branch does not work as expected (e.g. build failure, test failures, deployment failures etc).

The key point is that the feature branch MUST be merged with the main branch before running any build/test/deployment and not using only the branch on its own.

This consideration increases the level of the confidence while performing code integration that it is especially crucial when it is performed automatically.

After the build/test/deployment completes, the pipeline outcome is reported back to the Pull Request by updating the GIT commit status.

If the pipeline failed, the merge feature is disabled to enforce the team to fix the issue by pushing additional commit or re-executing the pipeline if the problem was not code related.

The following diagram shows the example workflow.

 

Prerequisite

The Jenkins Gitlab plugin is the best source of informations for supported versions as well as for community discussions.

My setup is based on the following stack:

  • Jenkins 2.89.1
    • Git Plugin 3.6.4
    • Gitlab Plugin 1.5.2
  • GitLab Community Edition 10.2.3

 

Jenkins Global setup

Configure the Gitlab plugin pointing to your gitlab instance as shown by the following example

Jenkins Job

Create a Jenkins job that will be used as quality gateway. It should be comprehensive so that all the typical stages are executed (build, test, deploy etc). If possible it should be implemented as Jenkins pipeline to leverage all the benefit of Infrastructure as Code paradigma. The following screenshots shows an example of a traditional Maven Jenkins job but the same can be easily translated in Jenkins pipeline.

SCM Git

The SCM must be configured with Git by pointing to the GIT URL where the project is hosted.

In order to build the feature branch and to perform the important branch operation before the build execution, use the following settings

  • Name: origin
  • refspec: +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*
  • branch specifier: origin/${gitlabSourceBranch}
  • Additional behaviour Merge before build
    • name of repository: origin
    • Branch to merge to: ${gitlabTargetBranch}

Build Trigger

The Jenkins job must be configured to start automatically as soon as our events occur. In this example workflow, we want to trigger a Jenkins build whenever of the two following events occur

  1. Create/Update Pull Request. We don’t want to start a build as soon as a push is performed against our branch (unless the Pull Request is already opened and the tile does not contain the WIP prefix) but only upon a Pull Request creation/update.
  2. A specific comment is added in the Pull Request. Sometime happens that a build fails and it must be restarted. Triggering it from Gitlab, will allow us to keep the Pull Request status in synch.

Jenkins should be configured like the following example:

Take a note of the Gitlab CI Service URL since it is required to configure properly Gitlab in the next step.

Update Commit Status

The last configuration is to tell Jenkins to update Gitlab (commit that triggered the build) with the build status.  Simply add the post build action as shown below:

GitLab setup

Project creation

Create a new project and upload it to Gitlab. The build process can be based on any tool (e.g. maven, gradle etc).

Gitlab should contain the source code, as well as the build scripts (e.g. pom.xml) and the Jenkins configuration such as the Jenkins pipeline (e.g. Jenkinsfile). This example shows instead the Jenkins configuration using the traditional web UI approach but the same concept applies if the job config is moved to the Jenkinsfile (reccomended).

Disable Gitlab Pipeline

Since we will be using Jenkins as CI tool, it is recommended to disable the Gitlab CI tool to avoid conflict in setting the commit status.

Navigate to Project / Settings / General / Permissions / Repository / Pipeline and disable it

Gitlab webhook

Once a Jenkins job is configured to kick off upon a Gitlab event, the Gitlab project must be configured as well to send those events to Jenkins.

Navigate to Gitlab project / Settings / Integration and create a new webhook like the following example. Make sure to use the URL shown by Jenkins Gitlab Plugin and select the interested events (In this example only merge requests and comments)

Test

Everything is ready and should work as expected.

To test the setup, create a branch and push some changes. No build should start on Jenkins yet.

By creating a pull request, a Jenkins build should start automatically and the Pull request shows that the pipeline is running.

Gitlab shows under the Pull Request all Jenkins invocations (Gitlab pipeline) along with the status: