SUSE HackWeek12 - YaST: Replacing Travis by Jenkins

Replacing Travis by Jenkins for YaST

I decided to look at the possible Travis replacement as my Hackweek 12 project.

Currently we use both Travis and Jenkins for continuous integration in YaST projects. Unfortunately there are many disadvantages with Travis which require additional work or limit us what we can run in a Travis job. See the hackweek project for the pros and cons summary.

The biggest Travis disadvantage is that the builds are run in a Ubuntu 12.04 system which is 3 years old and it's very difficult to find a recent compiler, Ruby interpreter, libraries,... for it. 

The Jenkins advantage is that it runs on our server and we can run the latest openSUSE very easily and avoid the problem with porting YaST packages to Ubuntu and backporting the development tools.

Jenkins Plugins

I found out that there are several Jenkins plugins which could be used to replace Travis with Jenkins:

• The GitHub plugin - handlers commit hooks sent by GitHub, reports build status back
• The GitHub Pull Request Builder - handles pull request hooks sent by GitHub (similar to the previous plugin, but builds the merged result)
• The Embeddable build status plugin - provides a build status badge (similar to Travis one)

You can install the plugins in your Jenkins instance like this:

• Login into Jenkins
• Go to Manage Jenkins -> Manage Plugins
• In the tab Available select plugins GitHub Plugin, GitHub pull request builder plugin and Embeddable Build Status Plugin and install them

Configuring the GitHub Plugin

• Generate a new access token at GitHub, select public_repo and repo:status. If you want to allow automatic webhook setup select write:repo_hook (you can add/remove the permissions later).
• Add the token to Jenkins -> Manage Jenkins -> Configure System -> GitHub Web Hook section -> OAuth token field
• Put the same token to the "GitHub Pull Request Builder" section, "Access Token" filed.

Create a new Jenkins job for building commits (pull requests are handled separately):

• Select Freestyle project
• Put the Github URL (https://github.com//) to the GitHub project filed
• In the Source Code Management section select Git and put the same URL here
• Make Branch Specifier field empty to build all branches
• In the Build Triggers section - check Build when a change is pushed to GitHub
• Add Build Step -> select Set build status to "pending" on GitHub commit
• Add post-build action - Set build status on GitHub commit
• Configure the other parameters of the build as needed

See more details here.

Configuring the GitHub Pull Request Builder

• Add the created GitHub token to Jenkins -> Manage Jenkins -> Configure System -> GitHub Pull Request Builder section

Create a new job for building pull requests:

• Select Free style project
• Put the Github URL (https://github.com//) to the GitHub project filed
• In the Source Code Management section select Git and put the same URL here
• In Advanced option set Name to origin and Refspec to +refs/pull/*:refs/remotes/origin/pr/*
• Set Branch Specifier to ${sha1}
• In the Build Triggers section - check GitHub Pull Request Builder option
• Check Use github hooks for build triggering option
• Set Commit Status Context to continuous-integration/jenkins-ci/pr (or something like that to have a different ID for pull requests and avoid clashing with the GitHub plugin configured in the previous step

Set Admins or white list users so you do not have to manually trigger builds for trusted developers - the plugin avoids running trusted code at your

Using Both Plugins in One Job?

I tried to use both plugins in a single job to have less jobs but it did not work form me, the pull requests were not processed at all or it did not work at all.

If you find how to solve it let me know...

Coveralls Support?

The question was whether coveralls code coverage can run also outside the Travis environment.

I found out that it is possible, you just need to set the COVERALLS_REPO_TOKEN environment variable and some other variables containing the Git branch, build number, etc... See more details here.

You need to store the token into Jenkins and set it during build.  (You can put the token directly into the .coveralls.yml file, but that's not a good idea for a public Git repository...) Fortunately there is a Jenkins plugin which helps with this, it can inject a secret in to a environment variable. The nice feature is that it can also filter the console output to make sure your password/token does not leak in the build output.

The Current State

I have created some experimental jobs to try the plugins, e.g. yast-registration-github-ci, yast-devtools-github-ci or yast-journal-github-ci.

The jobs are not ready to fully replace the Travis integration, but they work and I my plan is to continue with this project later.

TODO?

There are still many thing to solve:

• Coveralls - pass the Git data (branch name, etc.) from Jenkis, set the  COVERALLS_REPO_TOKEN variable, make sure it's not logged in the console to avoid compromising the value
• Find a way how to define the scripts started in Jenkins. We need to replace the .travis.yml files with something equivalent and share the common parts effectively, very likely as a shared Rake task.
• The current Jenkins jobs use active Git polling, we should switch that to use the GitHub web hooks 

So stay tuned, I'll post updates on this topic...

Bonus Section

During the implementation I found some other interesting possibilities:

Collecting Code Metrics and Code Coverage

The RubyMetrics Jenkins plugin can be used to collect Flog code metrics and RCov code coverage statistics:

That looks nice, but the practical value is low. The code coverage can be collected by Coveralls, which can comment the changes in pull requests, and code quality can be scanned e.g. by CodeClimate which provides more details about the code and better evaluates the code quality in general. More over it offers some hint what and how to fix, not just plain numbers without any clue what's wrong with the code.

So I decided to not use them for YaST.

Writing Jenkins Plugins in Ruby

I found an interesting possibility when playing with Jenkins - it's possible to write a Jenkins plugins in Ruby! There is also an example. Maybe we could simple enhance Jenkins if needed...