Continuous integration with Gulp and TeamCity

At Vizioz we use TeamCity as our continuous integration server for our development projects. It allows us to run a series of build steps, ensure the code compiles and deploy the website. We use it on our development server, deploying to a test website and creating a publish package for the live environment.

Recently, we decided to make full use of task automation for a project and incorporate a task runner in our build process. We decided to use Bower to manage our client components, and Gulp as our automation tool for our front-end tasks such us CSS and JS minification.

In this article, I will start from the basis that this is all up and running on our local solution and explain the steps I had to follow to integrate our automatic front-end tasks with TeamCity.

Setup

First of all, I had to install a series of tools on our build server so they can be used by TeamCity during the build process.

Git

This is the first tool to install. It is directly available from the git website. The installation is quite easy and straightforward. You only need to make sure to check the option "Run Git from the Windows command prompt". This will ensure that git commands are available in the console window.

NodeJs

Next comes NodeJs. It is also available from their site and the installation is quite simple too. NodeJs lets us manage and install a great number of packages via its node package manager, NPM. In our case, we used NodeJs to install Bower, Gulp and other useful libraries to help us with the automation of CSS and JS related tasks.

Bower

Bower is a front-end package manager. Many developers think Bower is no longer necessary since we are able to download the same packages via NodeJs and I partially agree with them (especially when we install Bower via NPM), but that is another story and I am sure it would be an interesting topic to debate. In our case we decided to make a distinction between them, so we use Bower to manage our client libraries, while NodeJs controls the packages in charge of running the automatic tasks.

As I said, we install Bower via NPM. We need to type the following command in the console window:

npm install -g bower

Gulp

Gulp is the task runner used in our project. There are other task runners out there, like Grunt, which is also quite popular amongst developers. Depending on your preferences or your work environment you might opt for Grunt or any other one of them. The process to follow then might be slightly different, but the essentials might still be valid.

As with Bower, in order to install Gulp we use NMP from the console window by typing the following command:

npm install gulp-cli -g

Other packages

These are the strictly necessary packages we need to install on our server. But if you need to compile LESS or SASS files during your build process, you will still need to install some other tools. For instance, if you have SASS files you will need to install Ruby, and through it install Sass, Compass and other necessary gems or packages. Since we had used Sass before we didn't need to install these again.

NodeJs TeamCity plugin

Once all these packages are successfully installed, we still need to configure TeamCity. We need to install the NodeJs TeamCity plugin that will provide node and NPM build runners. You can find the download link and install instructions at GitHub:

https://github.com/jonnyzzz/TeamCity.Node

Follow the download link and log in as a guest when prompted. Download the ZIP file from the latest successful build, copy it into the plugins folder (do not unzip it) within your TeamCity directory and restart the TeamCity server.

In order to restart TeamCity, you can run the scripts at TeamCity/bin/shutdown.bat and TeamCity/bin/startup.bat, or you can opt to restart the whole server. Please note that a restart is necessary for the changes to take effect.

Once the plugin is installed you will be able to see it in the plugin list in the administration panel.

Running Gulp on TeamCity

Everything is set up now. We need to make sure that Gulp will run properly on TeamCity. At this point it is very likely that Gulp will fail if we attempt to run it, complaining that 'gulp' is not recognized as an internal or external command, meaning the build agent cannot find Gulp in the path. To avoid this, we need to edit our package.json file and add Gulp to the build scripts:

{
  "devDependencies": {
    // ...
  },
  "scripts": {
    "gulp": "gulp"
  }
}

Make Gulp tasks synchronous

We might not be aware that Gulp tasks run asynchronously. Most of the times this is not a problem, but some other times the order in which the tasks are executed and completed is crucial. For instance, we might have a compiling task that needs to run after a cleaning task, but when they run asynchronously the compiling might finish before the cleaning, resulting in the later deleting some of the compiled files.

This is not always so obvious to spot. Locally, our task runner never had this problem, and our build server presented this issue only after several successful builds.

First, you need to make sure that all gulp tasks within your gulp.js file indeed return the actual task, so gulp knows when the dependent tasks are done.

gulp.task("clean:css", function (cb) {
    return rimraf(config.dist.css + "*.css", cb);
});

This will not be enough if you have multiple tasks running in parallel, for instance:

gulp.task("default", ["clean", "styles", "scripts", "fonts"]);

The example above is likely to have issues since the "styles" and "scripts" tasks most probably need to execute after "clean" has finished. To make the tasks running in strict sequence I got the help of gulp-sequence package. It can be installed via NPM:

npm install --save-dev gulp-sequence

This package will allow us to modify our gulp tasks so they can be executed synchronously:

gulp.task("default", gulpSequence("clean", "styles", "scripts", "fonts"));

Build steps

To complete the process we just need to add two build steps to TeamCity, one to install NPM dependencies, and another one to execute a gulp default task. In our case, we added the tasks as substeps inside our build.xml file with the following executable commands:

npm install npm run gulp default

As a side note, and in case you are wondering, we use a gulp task to run bower during the build process.

Success

I really love seeing that green success message in the build results in TeamCity, especially after integrating it with NodeJs and Gulp. After implementing the steps above all our builds have run smoothly and we can leave the bundling and minifying of client files to the TeamCity agent.

Hope this article is useful to someone else looking to integrate their continuous build with Gulp or any other task runner.