Bitbucket and Jenkins pull requests
Update: Part 2 Static Analysis and pull requests in bitbucket.
So we are using Bitbucket and Jenkins and moving from GitHub, GitHub and Jenkins have a great integration feature in the pull requests can be setup to build on Jenkins and return the build status to GitHub, this is what we would like to do with Bitbucket.
I found
this which details pretty much what we were required to do, thanks goes to Christian Galsterer for doing this in the first place, but I came across several differences and we needed to extend the behaviour to working with pipeline builds as well.
Process Requirements:
1. User creates a new branch (e.g. feature, bugfix).
2. After completing his development work and pushing his changes to Bitbucket
the user creates a pull request.
3. In order to approve a pull request we require at least one successful
Jenkins build. Thereby we would like to get not only the build result of the
code checked in for the pull request but get the build status after the code
has been merged with the target branch.
4. When a pull request is created/updated Jenkins shall be triggered
automatically for real continuous integration.
5. The source of the pull request shall be automatically merged with the
target branch.
6. Set the build description with the pull request ID and a link back to
the Bitbucket pull request.
7. The build result shall be reported back to Bitbucket.
8. Only if the build was successful and the number of successful builds
configured in Bitbucket is reached the pull request can be approved and merged.
This is the basic requirements that we need to satisfy and we had some dependencies.
System requirements
Jenkins:
Jenkins (2.19.1)
With the following plugins:
Pre SCM build step (0.3)
Groovy plugin (1.29)
Groovy installed and on the
system path.
Git Plugin(3.0.0)
Stash Notifier (1.11.4)
This
requires a username and password for bitbucket to update the pull request.
Pipeline plugin
Starting in Jenkins 2.0 there are some security changes for parameters, and these need to be white listed at the Jenkins Master.
Special startup parameters:
To make the parameters passable from the commit notify to
the jobs the parameters need to be added to the start-up parameters of Jenkins
this is to do with security changes in Jenkins.
“-Dhudson.plugins.git.GitStatus.safeParameters= PULL_REQUEST_URL,
PULL_REQUEST_ID, PULL_REQUEST_FROM_HASH, PULL_REQUEST_TO_BRANCH”
For windows this can be done using the Jenkins.xml in the
Jenkins home DIR. These need to be added before the -jar term as parameters
after this are ignored.
Ubuntu:
File: /etc/default/Jenkins
Property: JAVA_ARGS
RedHat:
File: /etc/sysconfig/Jenkins
Property: JENKINS_JAVA_OPTIONS
BitBucket Plugins:
Configuration
Once installed the plugin needs to be configured for use. First we are going to do a straight commitnotify, they work by taking the GIT repo url and then matching it to a job in Jenkins that isa configured for polling (there does not need to be a poll interval it's just the url that it will match with)
BitBucket Configuration
Pull request notifier in bitbucket is done using the
following:
1.
Install the Pull Request Notifier for Bitbucket add-on
via the Universal Plugin Manager or manually by downloading from the Atlassian
Marketplace.
2.
Administration > Manage Add-ons > Pull
Request Notifier > Configure
3.
Select trigger only if there are no conflicts.
4.
Select the following triggers:
a.
OPENED
b.
REOPENED
c.
RESCOPED
d.
RESCOPED_FROM
e.
RESCOPED_TO
5.
Enter Jenkins URL
6.
Choose GET as the HTTP method
7.
Use the following URL:
a.
(Jenkins_url)/git/notifyCommit?url=${PULL_REQUEST_TO_SSH_CLONE_URL}&branches=pr&sha1=${PULL_REQUEST_FROM_HASH}&PULL_REQUEST_URL=${PULL_REQUEST_URL}&PULL_REQUEST_ID=${PULL_REQUEST_ID}&
PULL_REQUEST_TO_BRANCH=${ PULL_REQUEST_TO_BRANCH}
8.
Save the trigger using save (note the view does
not refresh)
The notifier plug in exposes certain MACROS
PULL_REQUEST_TO_SSH_CLONE_URL : this is the url of the GIT
repo
PULL_REQUEST_TO_BRANCH: name of the pull request branch
destination
PULL_REQUEST_FROM_HASH: SHA1 of the commit to merge into the
branch
PULL_REQUEST_URL: Pull request URL in bitbucket
PULL_REQUEST_ID: pull request ID
General Jenkins Configuration
SCM
Git
needs to have a username and password in the global configuration so that it
can finish some merge requests, this is required if the tip of master is
updated and the GIT on the agent cannot fast forward to the tip of the branch
to be merged.
Global Security settings:
Change the Markup Formatter to be HTML
safe rather than plain text so that this can work.
Stash Notifier Plugin:
Jenkins > Configure System
> Stash Notifier
Enter Root URL, Stash user and
the stash password
If you use self-signed SSL certs you might need to set ignore SSL for testing, this issue should be solved before deploying live.
Job Specific configuration
Parameters
Parameterize the build with the following options:
PULL_REQUEST_URL : string
PULL_REQUEST_ID: string
PULL_REQUEST_TO_BRANCH: string
SCM
SCM GIT configuration, the GIT url must be accessed by SSH,
this is important due to the commit notify is only done if the repository
url is the same.
The branch specifier must be of the form: pr
This is so only pull requests are
built by this job
Tick the option merge before build use the following
options:
Name: origin
Branch: ${PULL_REQUEST_TO_BRANCH}
Merge: default
FF mode : --ff
Triggers
Polling must be enabled, no polling period needs to be set. This is due to the way in which the commitnotify has been done within the git plugin.
Pre-build step
“Run build step before SCM” system Groovy script to set the
build description:
def currentBuild = Thread.currentThread().executable
def PULL_REQUEST_URL = build.buildVariableResolver.resolve('PULL_REQUEST_URL')
def PULL_REQUEST_ID = build.buildVariableResolver.resolve('PULL_REQUEST_ID')
def description = "<a href='$PULL_REQUEST_URL'>PR #$PULL_REQUEST_ID</a>"
currentBuild.setDescription(description)
Pipeline jobs
Pipeline job needs to be configured slightly differently as commit notify does not operate on these. Instead we use buildwithparameters to trigger the job in Jenkins in any case configuration is detailed below.
BitBucket
For the Pull request trigger the following needs to be
configured:
Injection Url:
http://JENKINS_URL/crumbIssuer/api/xml?xpath=//crumb
Injection regexp:
<crumb>([^<]*)</crumb>
Basic authentication:
username and password
URL:
This needs to be a POST action
A header needs to be added:
Header:
Jenkins-Crumb
Value:
${INJECTION_URL_VALUE}
This trigger will now trigger the job in Jenkins to be built
with parameters that are required to merge the pull request in the target
repository.
Below is an example of a script to merge the two and the
notify stash.
Parameters
Parameterize the build with the following options:
PULL_REQUEST_URL : string
PULL_REQUEST_ID: string
PULL_REQUEST_TO_BRANCH: string
stage 'merge'
node {
def description = "<a href='$PULL_REQUEST_URL'>PR #$PULL_REQUEST_ID</a>"
currentBuild.setDescription(description)
sh 'git config --global user.name "Jenkins"'
sh 'git config --global user.email "jenkins@jenkins.com"'
checkout changelog: true, poll: true, scm:
[$class: 'GitSCM',
branches: [[name: PULL_REQUEST_FROM_HASH ]],
doGenerateSubmoduleConfigurations: false,
extensions: [[ $class: 'PreBuildMerge',
options: [mergeStrategy: 'MergeCommand.Strategy', fastForwardMode: 'NO_FF', mergeRemote: 'origin', mergeTarget: PULL_REQUEST_TO_BRANCH]]],
submoduleCfg: [],
userRemoteConfigs: [[credentialsId: 'jdengel', url: 'ssh://git@172.17.23.33:7999/pul/branch_test.git']]]
echo 'done'
}
stage 'notify'
node {
step([$class: 'StashNotifier']) // Notifies the Stash Instance of an INPROGRESS build
try {
// Do stuff
currentBuild.result = 'SUCCESS' // Set result of currentBuild !Important!
} catch(err) {
currentBuild.result = 'FAILED' // Set result of currentBuild !Important!
}
step([$class: 'StashNotifier']) // Notifies the Stash Instance of the build result
}
