Monday, 24 June 2013

Asset Pipeline Organisation

MIQly (our soon-to-be flagship product) is a application, but could be classed as 3 large-scale Javascript applications (setting questions, marking assessments, providing student feedback). As it exists in the Rails 3.1+ ecosystem, the Javascript (written one-step removed as ) lives inside the . I can already hear the shudders of dread [1] from many Rails people at this thought, but we managed to make the asset pipeline work for us. 

This is how.


The out-the-box organisation of assets within Rails provides very little control over the ordering and packaging of JavaScript and assets. Problems that follow from the standard and rather painful approach of a default, single application.js manifest file become apparent as an application grows, especially with one default line in application.js:

//=require_tree . 

This one line causes major headaches with indeterminate code ordering. This is especially apparent if the assets are developed on and then precompiled on a machine, as these have different default ordering of files loaded from the filesystem with respect to capitalisation. The result is painful manual resolution of dependency ordering problems, and frequently ends up with (the library that does a lot of the heavy lifting for the asset pipeline) being criticised for what is ultimately a code organisation issue.


We needed to impose a more suitable structure within the asset pipeline to avoid having to repeatedly perform manual resolutions as our code base grew.  We considered the use of a system like , but the integrations were immature at the time of consideration, and the asset pipeline itself provided advantages in terms of an integrated build process that we didn't want to abandon. As such, we wanted a solution that worked well within the asset pipeline rather than attempting to force a different solution on top of the pipeline.


Our solution is mostly organisational and not tooled beyond the Sprockets tooling already available. The key part is in recognising and distinguishing two distinct types of sprocket require directives and two different types of files. These get paired together as follows.


The first type is the 'typical' rails usage, with a manifest that pulls in various files into a precompiled asset. These are something we term an 'assembly' and defines a compilation unit. The items that get pulled into such an assembly are components, not lower-level units of composition. The initial application.js file in a new Rails project is an example of this (albeit a poor one, as it uses require_tree rather than a more stable dependency declaration method). One of our manifest files looks like:

The second type of file is an implementation file. These also have sprocket directives that will pull in required dependencies. For example, a screen widget item will pull in lower-level widgets and components (such as autogrowing text fields, or in-line editor code). The lower level code will also pull in its own dependencies, eventually 'grounding out' in either dependency-less code, or base libraries like jQuery  (n.b. in miqly, jquery is pulled in with a separate sheet to avoid certain issues with inter-manifest conflicts). One of our implementation files looks somewhat like:

This approach lets us assemble various components into compiled files and be sure of the dependency loading and code ordering within these files with very little effort: We have the asset pipeline resolving all our dependency ordering issues for us across several thousand lines of coffeescript and CSS.

[1] These 3 links were just picked after skim reading some results from the query 'asset pipeline criticism'. Feel free to provide more criticisms (or praise) in the comments :)

No comments:

Post a Comment