Javascript Module Pattern & Modern Tooling

The Module Pattern encapsulates privacy, state and organization using closures, and promotes namespacing, clean interfaces and decoupling, and code maintainability and reuse.

Note that "module" here is a chunk of JavaScript code and not an ES6 or AMD module.

About Todd

Lead Drupal Developer at @meetmiles

So your site's JS file is endless and unmanageable...

WAT? No!

What We'll Cover

  • Module Pattern
  • Quick Intro to Concepts
  • Module Pattern - illustrative code
  • Module Pattern - sub-patterns and variations
  • Module Pattern - real examples
  • JSHint - install and config
  • Code organization
  • Grunt concat
  • Package Managers
    • Bower & grunt-bower-concat
    • npm & Browserify

The Module Pattern

The Module Pattern is a Design Pattern

A design pattern is a general reusable solution to a commonly occurring problem within a given context -Wikipedia

There are plenty of JS patterns but the Module Pattern is easy and effective.

It can be traced back to Richard Cornford, among others, in 2003. Crockford populaized it in his lectures.

The Module Pattern is part of or can be implemented with almost every framework, including YUI, Drupal, Dojo, ExtJS, jQuery, etc.

Why the Module Pattern?

Advantages

  • Privacy: functions create scope; everything is private unless exported
  • State: private variables' values are maintained
  • Organization: code structure is not accessible outside the closure
  • Namespacing: only the module (function) name is available, nothing else is leaked into the global scope
  • Interfaces: returned objects are the public API; everything else is private
  • Decoupling: modules only communicate via their public interfaces
  • Maintainability: modules are leaner and specific, and are bounded by their API
  • Reuse: modules are modular, heh!

Privacy

Functions create scope; everything is private unless exported.

JavaScript has no inherent ability for privacy, but by using an IIFE* in can be emulated. Only that which is returned from the the module (the IIFE) is public, everything else is private.

State

Private variables' values are maintained.

With the use of closures private variables state (value) is maintained.

Organization

Code structure is not accessible outside the closure.

The inner workings, private variables, and structure of the closure code is hidden from the outside world.

Namespacing

Only the module (function) name is available, nothing else is leaked into the global scope.

"Polluting" the global namespace means you're placing quite a few assignments in the global scope, and you run the risk of your global vars will be modified, or you'll modify some other code's vars.

Interfaces

Returned objects are the public API; everything else is private.

The interface provides an "API" that external code has to use in order to interact with the module.

Decoupling

Modules only communicate via their public interfaces.

This is related to Organization and Interfaces, as each module in encapsulated and so has only limited "knowledge" of each other.

Maintainability

Modules are leaner and specific, and are bounded by their API.

By enforcing structure - privacy, interfaces, decoupling, etc - modules are to the point, less easily effected by other code, and so are more maintainable.

You probably break your SASS/LESS into separate files, and your backend code isn't in one huge file, so nor should your JS.

Reuse

"Modules are modular".

By creating encapsulated code with defined interfaces, it's easier to make code reusable.

Why not this Pattern?

Disadvantages

  • Accessing public members requires this.member while private does not
  • Changes in visibility require changing access method
  • Since functions create scope, extending modules with later functions prevents them from accessing earlier private members
  • Unit testing can't access private members
  • You can't hotfix a module's privates by extension (you have to actually fix the code)
  • Private members can't be extended

Quick Intro to Concepts

  • Scope
  • Immediately-Invoked Function Expressions (IIFE)
    • Function Declaration vs Function Expression
    • Forcing Function Expressions
    • Immediately-Invoked
  • Closures

Scope

  • The "global" scope is the top-level scope, AKA window in a browser.
  • A "local" scope is any non-global scope.
  • Only functions create a new local scope*.
  • "Child" scopes inherit the scope of their parent**.
  • Scope chain is the local scope, then their parent's scope, and their parent's parent, and so on, all the way up to the global scope.
  • Parent scopes have no access to their children's scope.
  • * In ECMAScript 2015 (ES6) let and const have block-level scope... ** Except for Function Constructors, which I won't cover.

Immediately-Invoked Function Expressions* (IIFE**)

Let's look at types of function statements:


function myFunc() { /* a function declaration */ };
foo = function() { /* a function expression */  };
foo = function myFunc() { /* a function expression */ };
            

There's also Function Constructors but we're not concerned with them here. Do NOT use old incorrect names, "self-executing anonymous function" or "self-invoked anonymous function". ** Allegedly pronounced "iffy", but I've never heard it not spelled out.

Function Declaration vs Function Expression

Function Declarations:

  • FD's are stand alone constructs and not part of a larger expression.
  • FD's get hoisted* to the top of the scope.
  • FE's are part of an expression, usually a variable assignment.**
  • FE's are not hoisted***

* "hoisted" statements get automatically declared (but not initialized) at the top of the scope. ** This is valid but don't do it: console.log((function(a){return a+1;})(4) + (function(a){return a+2;})(3)); *** In var foo = function /* etc */ the var foo is hoisted but not the FE itself.

Forcing Function Expressions

You can force a function statement to be an expression by wrapping it in "grouping operators" ()'s (aka paren's)


(function myFunc() { /* now a function expression */ });
foo = (function() { /* still a function expression */  });
foo = (function myFunc() { /* still a function expression */ });
            

So ALWAYS wrap function statements in parens to make FE's!*

*Even if not needed, as it's clearer and the preferred syntax. You may see other ways to force a function expression but don't do them - they do not make for readable code! FD's may also become FE's when not used as a "source element" but I won't get into that.

Immediately-Invoked

You execute a function expression by appending () like myFunction()


(function () {})(); // <- this trailing () here
isTheSameAsFunct();
						

This is exactly like calling isTheSameAsFunct(x) but instead of calling function isTheSameAsFunct, you're defining the function in the expression.

}()) vs })(): They are *exactly* the same. Crockford prefers the first, but Drupal, jQuery and most libs prefer second. There's also (function(){}).call(this); but I'm not going to address that here.

Closures

Anytime you use the function keyword* inside a function, you are potentially creating a closure**.

A closure means that the inner (child) function "memorizes" the outer (parent) local scope so that it persists between calls to the inner function.


var myFunc = function (x) {
  var myObj = {};
  myObj.myClosure = function () {
    console.log(x++);
  };
  return myObj;
};
var myInst = myFunc(10);
myInst.myClosure(); // 10
myInst.myClosure(); // 11
						

* Or the => (["fat"] Arrow functions) keyword in ECMAScript 2015 (ES6) ** Some will argue that technically a closure doesn't exist until during run time when a scope is externalized, but we're not going to get that deep.

Introduction to code

A Module Pattern module is code that returns an object, with properties and methods that define it's API.

Let's progressively introduce some Module Pattern though mock code.

Module Imports

All Foreign Modules (or any global) are passed in explicitly*.


(function ($) {
  $().someFunction();
})(jQuery);
						

jQuery is passed in and aliased as $ inside.

* It's bad form and slower to depend on JS' implied globals**. If you use JShint (described later) it won't let you not import globals. ** Implied globals means whenever a variable is used, the interpreter has to walk the scope chain backwards looking for a declaration.

Module (Public) Export

Assign the IIFE to a variable, and return an object. Externally use the "interface": doSomething.publicProperty *.


var doSomething = (function () {
  var my = {},
  my.publicProperty = 2;
  my.publicMethod = function () { /* ... */ };

  return my;
})();
						
If you return an object literal, you're using the Revealing Module Pattern (described later). * JShint will enforce importing this module into others before you can use it there.

Public and Private

privateVariable: using var or not*, it's not global; it's private as long as it's not returned


var doSomething = (function () {
  var privateVariable = 1; 
  function privateMethod() { /* ... */ };

  var my = {},
  my.publicProperty = 2;
  my.publicMethod = function () { /* ... */ };

  return my;
}());
						
*JShint will enforce using var with any assignments.

Internal Member Referencing

To reference publicProperty from within a public method you must use this


var doSomething = (function (console) {
  var my = {},
  my.publicProperty = 2;
  my.publicMethod = function () {
    console.log(privateVariable);
    console.log(this.publicProperty);
  };
  return my;
}(console));
						

This is because publicProperty in not local to the publicMethod (anonymous) scope, but rather is part of the returned object my so JS needs to be told the correct scope, this.

Sub-patterns and Variations

To really see the power of the Module Pattern, there's a number of different ways to structure the code and imports/exports.

Augmentation

You want to extend a previously declared module.


var doSomething = (function (my) {
  my.myProperty = 1;

  return my;
}());
						

var doSomething = (function (my) {
  my.newProperty = 9;

  return my;
}(doSomething));
						

Loose Augmentation

You want to extend a module, but want to be flexible in declaration order. If doSomething is falsy, an empty object "{}" is passed in.


var doSomething = (function (my) {
  my.newProperty = 9;

  return my;
}(doSomething || {}));
						
This || {} structure only works for the module itself; you couldn't do something like jQuery || {} as a second param. Loose Augmentation is also helpful in async and non-blocking loading of individual modules. You must be careful when accessing previously declared public members of the module.

Tight Augmentation

Tight enforces declaration order, but allows overrides of previously declared public members.


var doSomething = (function (my) {
  var oldMethod = my.aMethod; // "save" old method

  my.aMethod = function () {
    /* do something first */
    oldMethod(); /* do old thing second */
  }
  return my;
}(doSomething));
						

Revealing Module pattern

This variation dispenses with object literal notation for public members at the expense of not allowing overrides in later code.


var doSomething = (function (my) {

  var publicProperty = 29;
  var badlyNamedPublicProperty = 39;

  function publicMethod () { /* stuff */ }

  return {
    publicProperty: publicProperty,
    publicMethod: publicMethod,
    nicelyNamedPublicProperty: badlyNamedPublicProperty
 }
}(doSomething));
						
This can also be used to implement the Façade Pattern. Note that doSomethig as my does not happen to be referenced in this example.

Advanced

These can be looked up as there are somewhat esoteric:

  • Cloning and Inheritance: new module inherits from another
  • Cross-File Private State: split modules across files
  • Sub-modules: objects of object

Some Real Examples?

My "main" js file

Is itself an IIFE


(function (_, $, window, document, console) {
  'use strict';

  $('#footer-main > .block').equalHeight();
    
  $(window).resize(_.debounce(
    function() {
      $('#footer-main > .block').equalHeight();
    }
    , 250
  ));
	
}(_, jQuery, window, document, console));
						

Keep it clean and brief!

Just a Module

with Imports


// ie10ie11-check.js
(function (window, document) {
  'use strict';

  if ('onpropertychange' in document && !!window.matchMedia) {
    document.documentElement.className+=' ie10';
  }
  if (!(window.ActiveXObject) && 'ActiveXObject' in window) {
    document.documentElement.className+=' ie11';
  }
  
}(window, document));
						
Perhaps not strictly the Module Pattern as I'm not returning an object.

Loose Augmentation

Adds two methods to _, and if _ library is already loaded, that's OK.


// underscore-debounce.js
// borrowed from http://underscorejs.org/docs/underscore.html#section-71 etc
var _ = (function (underscore) {
  'use strict';

  underscore.now = Date.now || function() { /* ... */ };

  underscore.debounce = function(func, wait, immediate) { /* ... */ };
  
  return underscore;

}(_ || {}));
						
Debounce is key to performant event management!

Using jQuery in Modules

Just like "normal" as long as you Import jQuery


(function ($, window, document, console) {
  'use strict';
  
  $(document).ready(function() { /* do stuff here */ });

}(jQuery, window, document, console));
						
Perhaps not strictly the Module Pattern as I'm not returning an object.

Extending jQuery with a custom Plugin


// equal-heights.js
var jQuery = (function ($, console) {
	'use strict';

  // allHeights and equalHeights are now jQuery methods
  $.fn.allHeights = function() { /* ... */ return something; };

  $.fn.equalHeight = function () { /* ... */ return something; };
    
  return $;

}(jQuery, console));
						
jQuery.fn.equalHeight = (function($) { return function() { }; })(jQuery) would probably also work.

JShint

I've mentioned JShint a few times - take the hint: it's a great way to ensure you're following some best practices, have consistent structure, and catch some errors.

You'll need Grunt (or Gulp - but you're on your own for that) for most of the rest of this presentation so I'll cover it briefly.

Grunt: The JavaScript Task Runner

gruntjs.com

See Getting Started

There's a number of standards you're going to run into, but here's one we've already seen:

  • Put 'use strict'; just inside your function declarations.

JS Hint, install

grunt-contrib-jshint

jshint.com

"JSHint is a community-driven tool to detect errors and potential problems in JavaScript code and to enforce your team's coding conventions."

npm install grunt-contrib-jshint --save-dev
Then add this to your Gruntfile.js:
grunt.loadNpmTasks('grunt-contrib-jshint');

JS Hint, config

Edit your .jshint file so you have a globals key. This defines the properties as external. true means code can modify them, false not.

{
  "globals": {
    "jQuery": true,
    "Drupal": false,
    "window": false,
    "document": false
    }
}
						

JS Hint, grunt config

Add something like this to your Gruntfile.js:

jshint: {
  options: {
    jshintrc: '.jshintrc'
  },
  all: [
    paths.js + '/{,**/}*.js',
    '!' + paths.js + '/{,**/}*.min.js',
    '!' + paths.js + '/compiled_script.js'
  ]
},
						
Note that I'm ! excluding already minified and compiled files.

JS Hint, grunt task

Add something like this to your Gruntfile.js:


grunt.registerTask('js', [
    'jshint',
    /* put before uglify, minify, etc... */
  ]);
						

Now just use grunt js to run the JShint task.

Grunt concat

grunt-contrib-concat allows one to set up concatenation steps within your Gruntfile.js.

There's no dependency management, but the upside is you only need to load one combined JS file in your project.

The downside is you get ALL the JS, ALL the time.

concat, install

npm install grunt-contrib-concat --save-dev
Then add this to your Gruntfile.js:
grunt.loadNpmTasks('grunt-contrib-concat');

We're assuming you already have grunt installed.

concat, config


concat: {
  options: {
    separator: ';' + grunt.util.linefeed, // defensive semicolon
    sourceMap: true, // for debug
    nonull: true,   // warn of missing files
  },
  dist: {
   src: [ 
      paths.js + '/ie10ie11-check.min.js',
      paths.js + '/underscore-debounce.min.js',
      paths.js + '/equal-heights.min.js',
      paths.js + '/script.min.js'
    ],
    dest: paths.js + '/compiled_script.js',
 },
},
						
Define src: []in dependency order - there's no "lifting" among modules. I'm also concatting the minified files.

concat, grunt task

Add something like this to your Gruntfile.js:


grunt.registerTask('js', [
  /* put after uglify, minify, jshint, etc... */
  'concat:dist'
]);
						
Now just use grunt js to run the task that includes concat.

concat, grunt, alternatives

Grunt offers a number of ways to "build" file lists, including multiple build targets, wildcards, dynamic filesnames, exclusions and even code to perform complex file selections.

  • You may wish to make multiple compiled "target" files, and split them by section or function of your website. Just ensure you don't load some dependency twice.

Package Managers

Take a module and add some meta-data, such as version and dependencies, you have a package, so why not use a package manager?

Conceptually, your framework may have some way of managing JS, but as a package the meta-data is maintained with the module, where it belongs.

Package Managers

These are the leading options:

  • Bower is "A package manager for the web".
  • npm is "the package manager for javascript."
  • There are others, lessen known, like those using AMD, or forthcoming like ES6, but I won't be covering here.
  • This is where modularity, interfaces, and reuse really come into play. Why copy & paste code between projects, or worse, rewrite code again and again?

Bower

Bower was intended for client-side, most often JavaScript, but also CSS, HTML, and even images.

Bower uses a flat dependency tree, requiring only one version for each package, reducing page load to a minimum. -Bower

There's been a growing feeling that Bower is "losing" to NPM, but we'll cover it as it's still alive and kicking.

Brace yourselves, this is a long one.

Bower etc., installing

npm install -g bower

add bower_components to .gitignore

We're assuming you already have grunt installed, and are using git.

Bower etc., creating a package

In each of your module (AKA package) directories*:

bower init

Then answer the questions - notable ones below:

  • version: // Semantic versioning, ex 1.9.3-beta
  • main file: // in context, this is your js file
  • type of package: globals // this is Module Pattern**
  • mark this package as private? Yes // probably

* Including your 'main" JS file, as it's dependencies is what pulls in all the others. * See Bower Module Types

Bower etc., git part

By specifying module versions you can also reduce acquired incompatibilities, as unupdated packages will continue to use old compatible versions.

So now that you've recorded the version number in Bower, you need to do so in in that package's own git repo:


git push -u origin master // code push before tag
git tag -a m.n.p -m "tag message"
git push origin m.n.p
						
m.n.p is Semantic versioning: major.minor.patch, but extended versions are also legal. More on Git tagging

Bower etc., registries

Bower searches registries, default and custom, for packages. By default it searches the Bower registry and Github, by "shorthand" name, and it will accept full git URLs.

To add a "shorthand" for your organization's internal git repo, see this example for gitlab. Add/edit .bowerrc:


{
  "registry": {
    "search": [
      "git@gitlab.{your-org}.com"
    ]
  },
  "shorthand-resolver": "git@gitlab.{your-org}.com:{{shorthand}}.git"
}
						

Bower etc., installing packages & other commands

To install a package use it's "shorthand" name:


bower install user/repo-name[m.n.b] -S // *
						
-S flag will save the dependency in bower.json (which must exist prior). Bower will also automatically include dependencies of that package. * m.n.b is the Semantic Versioning, and is optional.

Other commands:


bower info {package} // displays package info and version
bower list {displayes} // local packages and updates
bower update {package} // updates the package
bower uninstall {package} // is self explanatory
						

Using Bower packages

You could stop here and use any of your framework's built-in methods to load your packages, or even plain grunt concat.

But there's a better way: use grunt-bower-concat to concat your packages - in dependency order, automatically!

bower-concat, install

npm install grunt-bower-concat --save-dev
Then add this to your Gruntfile.js:
grunt.loadNpmTasks('grunt-bower-concat');

bower-concat works by reading the bower.json files of all the Bower packages installed, determining their dependencies, and then concat's them in order.

bower-concat, config

The magic works out of the box IF you only want to make a *single* resulting js file. Here's an example Gruntfile.js:


grunt.initConfig({
  bower_concat: {
    dest: {
      options: {
        separator : ';\n' // defensive semicolon
      },
      dest: 'js_cat/dest.js'
    }
  }
});

grunt.registerTask('default', [
  // other tasks first
  'bower_concat:dest'
]);
						

bower-concat, going further

One of the disadvantages of bower-concat is it makes only one target file. There is an include parameter which has the unfortunate side effect of NOT including dependencies.

But when used with the parameter, includeWithDependencies you can now specify multiple targets, with automatic dependency inclusion, but the downside is you must ensure you don't load multiple targets on the same page which have the same dependencies.

Because of this caveat, the best use of this is dividing your targets not by component, but by page. *Provided by a patch by me, now merged.

bower-concat, includeWithDependencies


bandc_js: {
  options: {
    separator : ';\n',
    includeWithDependencies: true
  },
  include: [
    'js-b',
    'js-c'
  ],
  dest: 'js_cat/bandc.js'
}
						

Output two files


bandc_js: {
  options: {
    separator : ';\n',
    includeWithDependencies: true },
  include: [ 'js-b', 'js-c' ],
  dest: 'js_cat/bandc.js'
}
dwithc_js: {
  options: {
    separator : ';\n'
    includeWithDependencies: true },
  include: [ 'js-d' ],
  dest: 'js_cat/dwithc.js'  // js-d has js-c as dependency
}
						

npm

npm is Node Package Manager, original intended to be the package manager for Node on the back end, but has been used increasingly in more ways.

npm uses nested dependencies, so that each package has a copy of its own version of each dependency. This can make for a "heavy" installation that needs to be managed for front-end use.

npm, unlike Bower, has a requirement on your JS code: it must be CommonJS (CJS) module format, or now, as an ES6 module*.

I won't be getting into ES6 Modules here.

npm, CommonJS

CommonJS, for our purposes here, comes down to a couple basic ideas:

  • require function for importing other modules.
  • exports and module variables for exporting objects, or any variable.
  • It dispenses with IIFE's as inherently the CJS modules are run in their own context*.

Because of these, CJS modules have Privacy, State, Organization, Namespacing, etc., inherently, so using the Module Pattern with them is redundant.

* In fact there is no root scope at all.

CommonJS, example


var foo = require(foo);

foo.bar(); // run a method from the imported module

// export our own function
module.exports = function (x) {
  return {
    plusOne: function () {
      return x+1;
    }
  }
}
						

npm, Modular Pattern "adjacent"

Given the future of JavaScript is modules (it already is in the backend), I'll cover some topics to help translate what we've covered of the Module Pattern into Common JS.

npm, Install

At this point I've already assumed you've installed npm as it's needed for Grunt, and Bower.

As it comes packaged with node.js, see the node install page.

npm, creating a package

In each of your module (AKA package) directories*:

npm init
  • name:
  • version: // Semantic versioning, ex 1.9.3-beta
  • main: // this is your js file
  • private: true // probably

This creates and populates the package.json file. So far this parallels Bower, as does the git portion.

* Including your 'main" JS file, as it's dependencies is what pulls in all the others.

npm, registries

npm recognizes the following types of locations for dependencies:

  • local packages by name
  • local packages by path
  • tarballs by URL
  • git repos by URL
  • github repos by shorthand

npm, installing packages & other commands

To install a package use it's "shorthand" name:


npm install short/name[@m.n.b] -S // *
						
-S flag will save the dependency in package.json There's variantions on this to load from git, etc.

Other commands:


npm view {package} // displays package info and version
npm ls {package} // local packages and updates
npm update {package} // updates the package
npm uninstall {package} // is self explanatory
						

npm, dedupe

As was mentioned npm creates a "heavy" deep requirements tree, which can be problematic for client-side JS.

The npm dedupe command, "Reduce duplication", attempts to simplify the dependency tree by moving shared dependencies further up the tree.

npm, bass ackwards usage

At this point if you were just using your own (non compliant) non-CJS packages, you could just use grunt concat and manually specify the locations, versions and depency order of each package.

But I really wouldn't recommend it.

umd, Universal Module Definition

It's possible to write Module Pattern-ish code that's "normalized" (compatible) with Common JS, or even compatible with Common JS and AMD.

In the first case, it looks something like this:


(function (exports) {
  "use strict";

  exports.Foo = 'Bar';

}('undefined' !== typeof exports && exports || new Function('return this')()));
						

In the second case it's far to ugly to show here, Google it.

npm with Browserify

Browserify let's you "require('modules') in the browser by bundling up all of your dependencies".

Browserify requires you write your JS in the required CJS format.

To install:


npm install -g browserify
						

Browserify also has ways of handling some Node concepts that only kick in when trying to make Node code isomorphic.

Browserify, usage

To concat all your code in depency order:


browserify main.js -o bundle.js
						

grunt-browserify, install

Instead of using Browserify from the CLI, you can make it part of your Grunt tasks.

First install the package:


npm install grunt-browserify --save-dev
						

Then include the following line in your Gruntfile.js file:


grunt.loadNpmTasks('grunt-browserify');
						

grunt-browserify, config

Place something like this in your Gruntfile.js


browserify: {
  dist: {
    files: {
      'bundle.js': ['scripts/**/*.js']
    },
    options: {
    }
  }
}
						

And much like our earlier Grunt code, add browserify to a task, and then call the Grunt task from the CLI and you're off!

Further Reading

About Todd

Lead Drupal Developer at @meetmiles

If you can read this you've gone too far.