Using Gulp to Organize and Streamline Any Development Project

Intro

An important aspect of any development project is structuring the source. Even a medium-sized project can quickly become unorganized as features are added or changed. Using a task runner can make developers life so much easier, especially if multiple people are working on the project. Two of the most popular JavaScript task runners are Grunt and Gulp; both can run very similar tasks and the main difference is the syntax. Recently I have become a fan of Gulp's pipeline style syntax so that is what will be covered in this guide.

Before continuing this guide make sure you are familiar with the npm package manager and have Node.js installed.

Code snippets are provided for both Gulp 3.x and 4.x, the main difference is that 4.x uses ES6. Check your current version by running gulp --version, look at the local version.

Keep reading to learn more!

Project Structure

Project root folder structure.Project structure may vary depending on the size of a project, the type of project and even the dependencies of a project. But I generally follow the guidelines below for most projects that use a task runner.

If you are feeling lazy you can clone the sample repository to get a copy of the structure and code this guide will cover. Use one of the commands below to clone the repository.

# get sample repository for Gulp v3.x
git clone -b gulp-v3 --single-branch https://gitlab.com/evil-zebra/gulp-project-structure.git

# get sample repository for Gulp v4.x
git clone -b gulp-v4 --single-branch https://gitlab.com/evil-zebra/gulp-project-structure.git

Create Directories

Start by creating some required directories, the .project directory is optional. The src directory will contain all of your expanded and organized source files, conversely, the dist directory will contain the compiled and minified source files. Many projects will involve files that you wish to exclude from version control, include a .project directory to contain those files if you wish.

mkdir src dist .project

Create Recommended Files

Create three blank files - .gitignore, CHANGELOG.md, README.md. The first file is used to tell git which files to exclude from commits, it should include node_modules and .project directories. The other two files are recommended for all projects, the names are self-explanatory.

touch .gitignore CHANGELOG.md README.md

Generate package.json

Using the npm init command create your package.json file, follow the questions shown in the terminal. This file will save packages installed for this project and is a good place to set your project version.

npm init

Install Gulp & Create gulpfile.js

Use npm to install Gulp and add it to this project. Then create a blank file, gulpfile.js, this file will contain tasks for Gulp to process.

npm install gulp --save-dev
touch gulpfile.js

Source Structure

This is where things can vary widely depending on the languages involved and scale of the project, so I won't get too specific. A small project may require almost no structure at all and larger projects may have hundreds or thousands of files. Many developers will also use frameworks which tend to have their own structure recommendations.

All of your expanded source files should be placed into the src folder. We will use Gulp to concatenate, minify, obfuscate and copy them into the dist folder.

Gulpfile Structure

Random gulpfile example.Every gulpfile will have roughly the same code structure, some will be much more complicated than others. This will cover some common Gulp tasks which can be applied to many different projects. Below is a very simple gulpfile which will copy files from one directory to another. Continue reading to learn how each part of the file functions.

Gulp v3
// require any packages needed
var gulp = require('gulp');

// create a task for gulp to run
gulp.task('move', function() {
    // task logic
    return gulp.src('src')
        .pipe( gulp.dest('dist') )
});
Gulp v4
// require any packages needed
const { src, dest } = require('gulp');

// create a task for gulp to run
function move() {
    // task logic
    return src('src')
        .pipe( dest('dist') );
}

// make task public
exports.move = move;

Require Packages

The file should start by importing any required packages, these are installed via npm. All gulpfiles must include the gulp package and this example is using gulp exclusively.

Gulp v3
var gulp = require('gulp');
Gulp v4
const { src, dest } = require('gulp');

Declare Tasks

Now declare all tasks to be run, these tasks will use the previously imported packages. The name of the task should be descriptive enough to explain what is being done but also short enough to type quickly.

Gulp v3
gulp.task('taskName', function() {
    // logic for this task will go here
});
Gulp v4
function functionName() {
    // logic for this task will go here
}
// this makes the functionName() task public, allowing gulp to run it
exports.taskName = functionName;

Write Task Logic

There are a couple of functions that will be used in almost all tasks, gulp.src() and gulp.dest(). Please check out the documentation for a detailed explanation on both of these functions, they are covered briefly below.

gulp.src()

The first function is used for gathering the specified files to be used and returns a stream which can be used at the beginning or middle of a pipeline. Send a string or an array of strings to tell the function which files to gather - these are called globs.

Gulp v3
// gather all files and folders within the /src directory
var stream = gulp.src('src');

// this stream is then used by other packages
stream.pipe( some_package() );
Gulp v4
// gather all files and folders within the /src directory
const stream = src('src');

// this stream is then used by other packages
stream.pipe( some_package() );

gulp.dest()

gulp.dest() is used to write the files in the pipeline to a specified location. The function requires one parameter, a string specifying the destination directory. Non-existing directories will be created and any existing files with the same path will be overwritten.

Gulp v3
// write all files in the pipeline to the /dist directory
stream.pipe( gulp.dest('dist') )
Gulp v4
// write all files in the pipeline to the /dist directory
stream.pipe( dest('dist') )

Run Gulp

This is the easy part, within the same directory as the gulpfile run gulp with the task name. If following the guide step-by-step then add some empty or miscellaneous files within the src directory. After running gulp the files should be copied into the dist directory.

# run the move task
gulp move

If a task named default exists it will be run when no task name is provided, this is particularly useful for gulpfiles that contain one main task.

# run the default task
gulp

And that wraps up the basics of a Gulp task. The example task will essentially copy all files from the src directory into the dist directory.

Add Packages

npmThis is where Gulp starts to become very useful, there are thousands of useful packages online. An obvious resource to search for packages is npmjs.com, however, simply use your favorite search engine to search - gulp "some task I want to accomplish" - and one will likely find what is needed.

This guide will use the gulp-uglify package as an example. Start by installing the package with npm.

npm install gulp-uglify --save-dev

Next, add the package to the gulpfile.

Gulp v3
var uglify = require('gulp-uglify');
Gulp v4
const uglify = require('gulp-uglify');

And now the package can be used in any task, this task will uglify Javascript files within the pipeline.

Gulp v3
gulp.task('uglifyjs', function() {
    // minify all js
    return gulp.src('src/**/*.js')
        .pipe( uglify() )
        .pipe( gulp.dest('dist') )
});
Gulp v4
function uglifyjs() {
    // minify all js
    return src('src/**/*.js')
        .pipe( uglify() )
        .pipe( dest('dist') )
}

exports.uglifyjs = uglifyjs;

Task Dependencies

Handling dependencies for Gulp v3 versus v4 is quite a bit different so I will cover it in separate sections as opposed to only separate code snippets.

Dependencies in v3

Tasks have the option of being declared with dependencies on other tasks, the required tasks will be run first - these tasks will be run in parallel. To add a dependency the task format should look like this.

gulp.task('someTask', ['requireThis', 'runThisFirst'], function() {
    // task logic
});

To run tasks in a specific order setup the dependencies to require the previous task.

gulp.task('firstTask', function() {});
// this depends on "firstTask"
gulp.task('someTask', ['firstTask'], function() {});
// this depends on "someTask"
gulp.task('doThisLast', ['someTask'], function() {});

Dependencies in v4

In version 4.0 declaring a dependency is quite a bit different but much cleaner and more logical. Instead of declaring a list of required functions, tasks will utilize some new functions - parallel and series. These functions should only be used when declaring public tasks and not within the tasks themselves.

const { series, parallel } = require('gulp');

// this task will require two tasks, which can be run in parallel, to run before the last task
exports.someTask = series( parallel( someTask, anotherTask ), doThisLast );

// this task will run each task listed one after another
exports.someTask = series( runThisFirst, someTask, doThisLast );

Other Ways to use Gulp

While Gulp is targeted towards front-end projects it can be used in many different ways. Move and concatenate files, find and replace text and much more.

Gulp and a Wordpress Plugin

Often when developing a plugin or extension for software the plugin files need to be copied to the appropriate directory within the software. Here are two useful tasks, the first is used to copy files from the development folder into the Wordpress plugins folder. Be sure to change the directory names and paths to match your local environment and install gulp-zip. This same technique could be used in plenty of other settings - Joomla component, Adobe extension, etc.

Gulp v3
var gulp = require('gulp');
var zip  = require('gulp-zip');

// copy plugin files into wordpress install
gulp.task('copy-plugin', function() {
    return gulp.src('src')
        .pipe( gulp.dest('../wordpress/wp-content/plugins/your-plugin-name') )
});

// zip src/ directory and save as dist/plugin-name.zip
gulp.task('package-plugin', function() {
    return gulp.src('src')
        .pipe( zip('your-plugin-name.zip') )
        .pipe( gulp.dest('dist') )
});
Gulp v4
const { src, dest } = require('gulp');
const zip = require('gulp-zip');

// copy plugin files into wordpress install
function copyPlugin() {
    return src('src')
        .pipe( dest('../wordpress/wp-content/plugins/your-plugin-name') )
}

// zip src/ directory and save as dist/your-plugin-name.zip
function packagePlugin() {
    return src('src')
        .pipe( zip('your-plugin-name.zip') )
        .pipe( dest('dist') )
}

// make the functions public
exports.default = copyPlugin;
exports.package = packagePlugin;

Shell Commands with Gulp

With this nifty trick you can do nearly anything by simply utilizing the power of shell commands. Here is an example of how Gulp can be used to create and sign an executable for Windows. This example will require the packages gulp-inno and signcode, it also requires an Inno setup file and an OpenSSL certificate.

Gulp v3
var gulp     = require('gulp');
var inno     = require('gulp-inno');
var signcode = require('signcode');

// create exe using inno
gulp.task('inno', function(){
    return gulp.src('build-assets/inno-setup.iss')
    .pipe( inno({
        args: [ '/Q' ] // compile quietly, only print error messages
    }) );
});

// sign the generated exe using signcode
gulp.task('sign', ['inno'], function(){
  var exePath = ;
  var options = {
      path: 'dist/product-name.exe',
      cert: 'build-assets/openssl-certificate.pfx',
      password:'your-password',
      hash: 'sha256',
      overwrite: true,
      site: 'http://www.your-site.com/'
  };

  return signcode.sign(options, function (error) {
      if (error) {
          console.error('Signing failed', error.message)
      } else {
          console.log(options.path + ' is now signed')
      }
  })
});

// set the default task
gulp.task('default', ['sign']);
Gulp v4
const { src, dest, series } = require('gulp');
const inno     = require('gulp-inno');
const signcode = require('signcode');

// create exe using inno
function inno() {
    return src('build-assets/inno-setup.iss')
    .pipe( inno({
        args: [ '/Q' ] // compile quietly, only print error messages
    }) )
}

// sign the generated exe using signcode
function sign() {
  var options = {
      path: 'dist/product-name.exe',
      cert: 'build-assets/openssl-certificate.pfx',
      password:'your-password',
      hash: 'sha256',
      overwrite: true,
      site: 'http://www.your-site.com/'
  };

  return signcode.sign(options, function (error) {
      if (error) {
          console.error('Signing failed', error.message)
      } else {
          console.log(options.path + ' is now signed')
      }
  })
}

// force tasks to run one after another using series()
exports.default = series( inno, sign );

Useful Links

gulp.js Quick Start
gulp-project-structure repository

February 21st 2019

Nicholas Horlocker

gulp, javascript, npm, task runner, development