Building a Webpack plugin to generate localized emails

At Dashlane we build most of our transactional emails and some marketing emails using basic HTML/CSS with a sprinkle of EJS templating for rendering variables and conditional statements on server-side. Unfortunately, HTML emails can be a frustrating thing to work on since email clients typically render using outdated standards and require nested tables and other things that the much more progressive web has moved past in recent years.

Our existing approach to building email templates was lacking from a technical standpoint too. We had a simple template build system that handled some things like i18n and A/B tests, but all of our emails were barebones HTML with no tooling to help developers build out each email. Each template was basically it's own unique file, with poor reusability. We wanted components, we wanted TypeScript, and importantly we wanted a drop-in replacement so we could seamlessly move from the legacy system without a hitch.

The goal: Create a new build pipeline for email templates

We had some important qualities going into our prototype:

After some initial prototyping, we landed on using MJML, an email-centric markup language from Mailjet. It solved a lot of the headaches involved with building specifically for email clients, which is an area where we didn't want to reinvent the wheel or spend too much time worrying about email client rendering differences. We expanded it to a proof-of-concept showing emails built in React, written in TypeScript, with a backend of MJML, and rendering to static HTML with full EJS still intact. This covered most of our goals so we knew this was the way to go.

To put it all together, we felt that Webpack could handle a lot of the compilation steps, such as loading TypeScript sources, gathering dependencies, and outputting the final assets. So we set off to build out a Webpack-powered build system. One problem: At the start of this project, I had never built a plugin for Webpack! I had very little knowledge about Webpack outside of some fighting with configs to get my projects to build correctly. Not only that, we weren't using webpack in one of the more common ways, so finding documentation and examples was difficult.

For our email build system we needed to do a few things:

The goal is to end up with a build system that looks something like the image above. All components will be built on React/MJML with some JSON metadata. Webpack will drive compilation and take that React code and data to build the files and we will end up with static HTML and some other necessities like metadata and plaintext versions of the emails. Most of the work will happen in a custom built Webpack plugin which we call ReactMJMLWebpackPlugin. This plugin will handle a few different phases of compilation:

And with that, we have enough high level info to get started. Here on out we'll be going deep in the technical details of setting up a Webpack plugin to drive the build system including lots of code examples and config files! First, a detour to explain what a Webpack plugin is, for the unfamiliar.

Webpack defines itself as a "static module bundler for modern JavaScript applications". A basic explanation of what it does would be that it takes some files as input, maps out their dependencies, then merges those files into some bundled output.

A simple outline of how Webpack does this:

I recommend reading the Webpack Concepts page in their documentation for a bit more in depth outline of how webpack works from a conceptual level.

Before getting into the cool stuff, there's some setup required to get Webpack to work with our stack. This is probably the area most developers who use Webpack are familiar with. Most projects just need to use existing plugins and setup a few config options to get up and runnning.

In our project, we want to use TypeScript so we'll need to add a TypeScript loader called ts-loader. And we want all our builds to output to the export directory. Let's also add in file-loader and use it on images. This will load all images with their absolute location on the local disk so we can import images and use them in our React components with ease. In production you'd want to specify the URL of your file hosting here or in the case of a web app, use something like url-loader to load Base64 images inline. For testing emails locally in the browser however, a file url works just fine.

This gets us rolling with TypeScript and React! Toss a main.tsx file in the codebase and it'll generate a main.js file just fine! However, we're not building a single typescript file, we'll need to build for a large number of email templates!

Entry points are where Webpack starts building out it's dependency graph.

The simplest example of this would be a single JS file that is the sole bundle for a Single Page Application, where all imported dependencies are loaded into this main entry point.

If we wanted to generate multiple bundles, we would add new entry points. For example a static website might contain a single bundle for each page. The general rule of thumb is "one entry point for each HTML document".

In our case we decided to treat it like a static website since we have many templates each with their own build output and each generates a different HTML page. Since we have potentially multiple variants for each template, each A/B test variant and the control will all be their own entry point. For localization we just insert different content based on language, we'll handle that in the plugin itself, which we'll touch on later.

That's great but its very clunky to write, defining a new entry point for every single template variant gets cumbersome fast, and the config would get massive. Since we use Javascript to define the webpack config we can actually simplify this into a helper function that crawls the source code for all our templates. This makes it really simple to add new templates. A developer can drop in a new template to that folder and Webpack will pick it up with the rest.

With this config we actually get pretty far towards our goal! Webpack takes our entry points and makes a new dependency graph for each. We'd be able to use React and import any other dependencies and images. Unfortunately we're still just generating a .js bundle for each template. That might work well for a website, but we want fully build static HTML with no JS to use in emails.

Our plugin is going to need to handle a few things:

To build a webpack plugin, make a new class that has a method apply that takes a compiler object.

From the apply method, we have access to the entire Webpack compiler lifecycle. Super cool! But also super complex! The Webpack compiler does a lot. It builds the dependency graph, loads source files, optimizes bundle output, and a lot more. For this plugin we're going to focus on a few specific things: loading properties files not normally picked up in the dependency graph, replacing the JS bundle output with static html, rendering extra output (plaintext email, metadata file).

In the apply method, we hook into different stages of the lifecycle. Here's a simple example, hooking into the compilation object, then again into the "optimize assets" stage of compliation.

This example will print "Compilation object created!" then list each of the assets. In our case this will be the already compiled JS bundles, since optimizeAssets occurs after the modules have been bundled, and will look something like:

Notice that the asset keys are actually the files that will be outputted, and they are .js! that's because we transform the tsx files using ts-loader and the final bundle output will be in JavaScript.

For our email templates, even though we're using this new build process, we actually still have more templating to do once the emails are built! We use EJS inserted inside <mj-raw> tags so that the final output of our build can still have variables inserted, such as a user's name or a dynamic link generated on the server before sending. We have a properties file we use to define this contract with the server as well as A/B information and i18n language data. So we need to load that file, parse it's data, and validate we have everything we need.

The main question now is: Where do we do this? What part of the Webpack lifecycle should handle this?

We actually aren't importing the properties json file into each component, so Webpack won't be automatically adding it to the bundle output.

Some solutions considered:

Option 1 required developers to manually add extra code to each template, whereas Options 2-4 handled the properties file automatically. Option 2 felt like it was a misuse of entries, breaking the best practice of "one entry point for each HTML document" (more on that in Addendum on extra dependencies). And Option 3 felt just as hacky as Option 4, but knowing adding manual assets wouldn't have any side-effects, we opted to just load properties manually using node's fs module.

The initial thought was to add this to additionalAssets, but we also wanted to use this data to validate the components. Also we were unsure if compilation.assets had the full list of assets at this point, whereas optimizeAssets did. Since that's where we were doing the rendering (detailed in the next section), we simply added this file loading there.

One thing we'd really like to do is not assume that every JS file is a template and instead flag each one with metadata, but there doesn't seem to be a clean way to do that in Webpack. What we can do, which is how we've built the production version of this, is exclusively store all templates in a templates/ directory, so we know all the JS files we find will be the template files, and only look there when doing transformations in the plugin. That process has been left out for the scope of this post.

Note: We do this phase and the last phase in optimizeAssets, admittedly not a great place to hook into to do this, but there's not really a better place to do so. Webpack v5 introduces a new hook called processAssets that has stages that are made specifically doing what we're doing here where we can add extra assets and made assets from existing assets. We're looking to do a refactor now that Webpack v5 is released.

This is where the atypical Webpack usage begins. So we actually want to render to HTML, we don't want bundled javascript. So while we take advantage of features of webpack throughout the rest of the build process, here we diverge.

We're going to take our bundled React components and run them through eval (can be done using node-eval or jsdom). We then render to static markup using React DOM Server. That'll give us some static mjml content. MJML has it's own renderer, it's a simple function called mjml2html, so we'll call that next.

Note: A word of warning! You don't want to use eval client-side, we only ever run eval as part of the build process of our own React components since we only have them as bundled JS as strings. See: Never use eval!

Our render looks like: Raw JS string to React component to MJML to HTML.

Fantastic! now we have a bunch of new html assets that look something like:

These will all get taken by the final steps of the webpack compiler and give us some lovely static HTML emails rendered out to our export directory!

A few additional things to do before we wrap up. That solution is great but we also want to automate a few things. One is our plaintext emails. Here we simply piggy back off the asset definition we did at the end for the HTML and add a new step.

Finally, we want to handle translations with ease, so we have a bunch of i18n keys at the ready for all text content inside the emails. The way we handle this is simply to loop over each language, and render our output one extra time for each language.

Great! No we have a solid build pipeline all powered by Webpack in one custom plugin. For a high level view of how this plugin is structured take a look at this diagram which shows what different compilation process we hooked into and what steps we took to build each piece of the build system puzzle. There's plenty more to do, and we've built a bunch more on top of this system, like CLI tools for generating and sending emails, reusable components, and more. This should hopefully demystify Webpack plugin development and also show how to tackle some of the less straightforward setups that Webpack can handle, but is lacking in examples or documentation.

There were a few tips and implementation details left out of the main part of the post since it would have complicated the example code a bit too much. We still wanted to include them as they are useful to know for anyone stumbling through Webpack plugin development.

Uncaught errors may cause Webpack to crash, meaning that it won't rebuild automatically when the errors are thrown. You can surface these errors as compilation errors by catching them and adding them to the compilation.errors array, rather then letting them bubble up. We made a little pretty print function to make this clear and encourage our plugin maintainers to provide helpful hints to the other devs!

Any entry file and it's dependencies will be automatically watched by Webpack when you run a watch build. Unfortunately since we do some parsing and file access outside of these dependencies, changes to files like i18n data will not rebuild webpack in dev mode.

Luckily you can add to fileDependencies manually, and Webpack will properly watch those files like any other dependency!

While writing this article I found Advanced entries and feel that would be a cleaner way to do Phase One Option 2 ("Add the properties file to entries"), and probably the best solution. I explored a refactor that uses this instead, but Webpack didn't handle the extra entries very well, and screwed up the way modules were bundled, breaking our React eval step (will go over that in the next section), and I ended up having to ditch the idea.

For more info see: 1, 2, 3, 4

I wouldn't have made it through this without a few really helpful resources and open source projects:

Kenny Goff

Read more