Continuous Integration with Jenkins

CI with Jenkins and GitHub is especially interesting for private repositories, as CI with Travis is mainly for free GitHub repos (unless you have Travis Pro, of course).
But since Jenkins is OpenSource and free, it might make sense to set up this on your server and run CI directly in-house.
It integrates nicely with GitHub. So the following tutorial will focus on that example.

So for the beginning, we have our example app in GitHub, e.g. a CakePHP app (optionally including composer dependencies).

The main goals are:

• Continuous test results on each push (especially to master).
• Automatic test results for each PR (very important when working in teams) – linked inside the PR just as Travis would.
• Coverage being analyzed to see where the weak points can be (classes with < 50% test case coverage).

Set Up Jenkins

This is probably the most difficult part of all.

We first set up Jenkins.

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | apt-key add -

nano /etc/apt/sources.list
------------------------------
deb http://pkg.jenkins-ci.org/debian binary/
------------------------------

apt-get update
apt-get install jenkins

Check if Jenkins runs properly

http://hostname.domain.local:8080

Install plugins and all their dependencies

Let’s then include the plugins we need to integrate GitHub: "GitHub Pull Request Builder" and "GitHub plugin".
The first is especially interesting as it makes it possible to have Jenkins "build/pass" statuses directly in each PR (similar to Travis).

Access Data -> Global Access Data -> Add Access Data

Create access data for the repository you want to access. Provide username and password or keyfiles.

Manage Jenkins -> Configure System

• Add your repositories under "Git"
• Set "Manually manage hook URLs" under "GitHub Web Hook"
• Generate token (https://github.com/settings/applications) with Github user which has access to the
  private repository and put the token under "GitHub pull requests builder"

Add Job

1. Select "Free Style" project and provide a Jenkins working space name
2. Put your repository url (e.g. https://github.com/Name/repo) to "GitHub-Project" input field
3. Select Git under "Source-Code-Management"
   Put your repository url again and select the access data from step 5 of this tutorial.
   Click on advanced settings and set "+refs/pull/:refs/remotes/origin/pr/" for Refspec and "origin" for name.
   Set also the branch specifier to "${sha1}"
4. Select "Build when a change is pushed to GitHub" and "GitHub pull requests builder" under "Build trigger"
   Add persons or organizations to "GitHub pull requests builder" which are whitelisted for trigger an PR build request
5. Add shell execution script for your test cases under build proceed (see below)
6. Add "Set build status on GitHub commit" as post build action.

My shell execution script for the CakePHP apps looks like this:

cd /var/lib/jenkins/jobs/projectname/workspace/app
cp /var/www/projectname/app/Config/database.php Config/
mkdir -p tmp && chmod -R 777 tmp && rm -R tmp/*
cp /var/www/projectname/app/composer.phar ./ && php composer.phar update
Console/cake test app AllApp --stderr

It first navigates into the freshly pulled repo code and into the APP dir.
Then it copies over the database.php from the current staging website (could also be some other place where you store them).
It then creates tmp dirs and sets the correct permissions.
The composer.phar file is also copied over and executed to pull all composer dependencies.
At finally the AllApp (group) tests are executed – with –stderr in order for the session to work properly.
One can then also apply -q for silent (=none) output.

Setup GitHub hooks

To trigger the build process on Jenkins, we need to configure service hook on GitHub. In order to achieve this, navigate to GitHub repository settings and configure Jenkins Hook URL for GitHub plugin. The URL format http://<jenkins-username>:<jenkins-password>@<Elastic-IP-Address>:8080/github-webhook/

Running tests

Run first test manually

Click on "Build with Parameters" and hit "Build" and you will see the first manually triggered build in progress.
If all goes well the icon should be blue (pass) – not red (fail).
In case you need to see what is going on, check the "Console Output" of a specific build.

Check automated tests

Commit to master and see if a build is triggered. Also open a PR and check if there is a badge displayed with the current build status.

Sugar

Badge

To have a small "build status: passing" badge in our Readme, we can install the plugin "Embeddable Build Status".
It will create a new menu entry from where you can select your badges including ready-to-use Markdown syntax if needed.

And if it is a private repo you might also want to htaccess the Jenkins web interface or protect it some other way from the public.

Code Coverage

You can install the plugin "Clover PHP Plugin".
Create a folder "coverage" in your workspace.

Add this to the above command:

--log-junit ../coverage/unitreport.xml --coverage-html ../coverage --coverage-clover ../coverage/coverage.xml

And set up a "Publish Clover PHP Coverage Report" Post Build Action with the absolute path to this coverage.xml file in the workspace.
In my case just coverage/coverage.xml as it can be relative to the workspace, as well.
If you also want to publish the HTML coverage report (which is pretty neat), you can simply put coverage in there then.

Make sure you got Xdebug up and running (apt-get install php5-xdebug). It is a direct dependency here.
Note that using code coverage reports will slow the whole process down by "x3" at least. Usually, that shouldn’t really make a difference, though.

In most cases you want to exclude few dirs. In case you are already using a phpunit.xml file which resides in your APP dir, you can easily add

<phpunit>
    <filter>
        <whitelist>
            <directory suffix=".php">*/Controller</directory>
            ...
        </whitelist>
        <blacklist>
            ...
        </blacklist>
    </filter>
</phpunit>

You can also just create this file and also copy it over to the jenkins workspace.

Memory Limit

If you have a project of medium to large size you will soon see the memory usage for the test coverage of 50%+ way above 200MB.
With more coverage and the project getter bigger, or some coverage report generated you will run out of memory with 256MB (which one would think should suffice).
Raise your limit to at least 512 for the PHPUnit testing. You can use the phpunit.xml file:

<phpunit>
    <php>
        <ini name="memory_limit" value="512M" />
    </php>
    ...
</phpunit>

Cleanup

Use the plugin "Discard Old Build plugin" to keep your disk space reasonable. Without it (especially with CodeCoverage) the disk space
will soon exceed several GB. We usually keep the last 40 builds.

More

This article goes more into detail about the installation of Jenkins and also how to use additional tools like CodeSniffer, MessDetector and CO using a build file and Ant. But I didn’t check that out yet.
There is also jenkins-php.org if you want to go all-in 🙂

UPDATE 2020

GitLab CI could also be a valuable tool to check out these days.