Peter Reid
Peter Reid

My ramblings about developer tooling, the Cloud, JavaScript, DevOps, programming and technology.

Peter Reid
Author

Subscribe


Fill in the form below get the latest posts from Peter Reid straight to your inbox!

Share


Peter Reid

What I've learnt about Jenkins Pipelines

Peter ReidPeter Reid

Pipelines were introduced to Jenkins in April 2016, in this article I talk through some of the best pipeline steps and the weaknesses of pipelines.

Since last April I've been using the latest version of Jenkins for a number of test projects and some academic projects, I've found the documentation for performing a number of operations very sparse, and after much experimentation I have some knowledge to share with you.

Tips on writing Pipelines

Sending email notifications

Unfortunately, I haven't yet found a way to send HTML formatted emails in the same way that you can with email-ext for traditonal jobs in Jenkins, judging by discussion on the forums and the docs it appears that email-ext does not support this at present. I've found mailing the build log to the 'culprit(s)' is the best option, but I've also seen instances of simply sending a link to the build on build failure along with the message 'build failed'. To send a log on failed build you can do the following in your jenkinsfile.

def err = null
try {
    node {
        // Main pipeline code goes here
    }
} catch (caughtError) { //End of Try
    err = caughtError
    currentBuild.result = "FAILURE"
} finally {
    (currentBuild.result != "ABORTED") && node("master") {
        // Send e-mail notifications for failed or unstable builds.
        // currentBuild.result must be non-null for this step to work.
        step([$class: 'Mailer', notifyEveryUnstableBuild: true, recipients: emailextrecipients([[$class: 'CulpritsRecipientProvider'], [$class: 'RequesterRecipientProvider']])])
    /* Must re-throw exception to propagate error */
    if (err) {
        throw err
    }
}

}

Update 2017-02-06: I have been informed by 'Yeroc' in the comments that the Pipeline Model Definition Plugin implements better syntax to replace the try...finally block in the above example. I'm currently investigating this.

Sending Slack notifications

If email isn't your thing you may want to send notifications out to Slack when your build commences, fails or completes successfully. I found out how to do this in a great post by Liam Newman over on the Jenkins project blog.

First walk through the install wizard on your Slack team to setup a key for your Jenkins instance to use.

Then go ahead and install the Slack plugin for Jenkins and configure it with the settings you set above. The problem here is that there is no project level configuration option for the Slack plugin, so if you have multiple teams with different Slack teams using the same Jenkins installation then you may not be able to do this in the way in which you could do with traditional jobs.

You can add the following pipeline step to notify your team when the pipeline starts a build.

slackSend (color: '#FFFF00', message: "STARTED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")

To notify your team when your pipeline fails you can add the following

slackSend(color: '#FF0000', message: "FAILED: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")

and to notify them of a successful build you should add

slackSend(color: '#00FF00', message: "SUCCESSFUL: Job '${env.JOB_NAME} [${env.BUILD_NUMBER}]' (${env.BUILD_URL})")

Running tests and archiving the results

One thing I've picked up on that you can quite easily omit when testing out your pipeline is that you should surround your step that runs your tests in a try...finally or it will fail to run the 'results archiver' and you'll have to go digging through a lengthy console log to find the failing tests.

try {
    sh 'gradle test'
} finally {
    ([$class: 'JUnitResultArchiver', testResults: 'app/build/test-results/debug/*.xml'])
}

My thoughts

Strengths

  • 'Configuration-as-code' allows for greater traceability of changes (through version control) to the build process - a much requested feature of traditional jobs.
  • 'Configuration-as-code' allows for administrators to restrict administrator access for a pipeline to only those who absolutely require it, rather than granting administrative permissions to all the users of a pipeline who may not necessarily be trusted to 'not break the pipeline'.
  • 'Configuration-as-code' allows for much easier sharing of build processes between teams, projects and in the open-source community - if you ever had to manually copy over another Jenkins job's configuration you'll know why I love this so much.
  • I find the stage-view GUI much friendlier than a series of jobs chained in sequence. Coupled with the new Blue Ocean user experience plugins Jenkins is finally aesthetically pleasing and the new UI makes it much simpler for non-technical project stakeholders to understand and use.

Blue Ocean UI

Weaknesses

  • There are still plenty of plugins that I and many others will consider 'core to their CI pipeline' missing full or partial support for pipelines. For example the Gradle, Maven, Ant, CucumberTestResultArchiver [1] and CoberturaPublisher plugins; once staple plugins in traditional jobs, do not support pipelines at the time of writing. While you can work around these limitations with shell stages invoking a series of commands manually, if you aren't familiar with the CLI syntax of a tool it can be a chore to determine the commands that Jenkins once executed for you as part of these plugins.
  • The lack of 'per-project-configuration' in pipelines for many plugins that had this for jobs can lead to making certain plugins 'unusable' in your environment, particularly if you operate several projects with different teams who may require different source control or messaging service integrations. I believe this may be an API limitation of pipelines, with plugins simply not being able to do this.
  • I've found a number of plugins which have had very limited documentation of their available options - and have had to spend much time digging through the Jenkins issue tracker, StackOverflow and various user groups.

Conclusions

At this point I'd say that at least in my eyes Jenkins has finally caught up with Travis CI, Gitlab CI and BitBucket Pipelines having been behind for quite some time due to the lack of configuration-as-code. The plugin ecosystem still needs a lot of work to bring all the most popular plugins into the pipeline ecosystem, a fair number of plugins will need updated for pipelines to match the integrations that Travis has and the new UI still needs a lot of work; even with that Jenkins is now once again my go-to CI product, and I'd definitely recommend using it if you can withstand the limitations of Pipelines I've highlighted above.

Updates


  1. Version 0.9.7 of the CucumberTestResultArchiver implements support for pipelines, therefore the statement I made no longer applies to this plugin. Credit /u/steveeurcol (2017-02-06) ↩︎

Peter Reid
Author

Peter Reid

Comments