Building a Webpack plugin to generate localized emails

At Dashlane we build most of our transactional emails and some marketing emails using basic HTML/CSS with a sprinkle of EJS templating for rendering variables and conditional statements on server-side. Unfortunately, HTML emails can be a frustrating thing to work on since email clients typically render using outdated standards and require nested tables and other things that the much more progressive web has moved past in recent years.

Our existing approach to building email templates was lacking from a technical standpoint too. We had a simple template build system that handled some things like i18n and A/B tests, but all of our emails were barebones HTML with no tooling to help developers build out each email. Each template was basically it’s own unique file, with poor reusability. We wanted components, we wanted TypeScript, and importantly we wanted a drop-in replacement so we could seamlessly move from the legacy system without a hitch.

The goal: Create a new build pipeline for email templates

Initial proof-of-concept

We had some important qualities going into our prototype:

  • Ideally leverage React and TypeScript, which our other web tech stacks are built on
  • Build static output in HTML with EJS template scripts
  • Simplify process of building for email clients
  • Support i18n/Localization
  • Handle A/B Testing

After some initial prototyping, we landed on using MJML, an email-centric markup language from Mailjet. It solved a lot of the headaches involved with building specifically for email clients, which is an area where we didn’t want to reinvent the wheel or spend too much time worrying about email client rendering differences. We expanded it to a proof-of-concept showing emails built in React, written in TypeScript, with a backend of MJML, and rendering to static HTML with full EJS still intact. This covered most of our goals so we knew this was the way to go.

To put it all together, we felt that Webpack could handle a lot of the compilation steps, such as loading TypeScript sources, gathering dependencies, and outputting the final assets. So we set off to build out a Webpack-powered build system. One problem: At the start of this project, I had never built a plugin for Webpack! I had very little knowledge about Webpack outside of some fighting with configs to get my projects to build correctly. Not only that, we weren’t using webpack in one of the more common ways, so finding documentation and examples was difficult.

The Plan: Build an email compilation engine

For our email build system we needed to do a few things:

  1. Build one html email for each template, for multiple languages
  2. Build zero or more variant templates for A/B tests, each of which may also have custom translation strings
  3. Include metadata information about each template and include that in the build output (useful as a contract with server to know what to expect to render)

The goal is to end up with a build system that looks something like the image above. All components will be built on React/MJML with some JSON metadata. Webpack will drive compilation and take that React code and data to build the files and we will end up with static HTML and some other necessities like metadata and plaintext versions of the emails. Most of the work will happen in a custom built Webpack plugin which we call ReactMJMLWebpackPlugin. This plugin will handle a few different phases of compilation:

  1. Loading email template properties
  2. Evaluating React and rendering to MJML/HTML
  3. Outputting additional assets
  4. Render for each supported language

And with that, we have enough high level info to get started. Here on out we’ll be going deep in the technical details of setting up a Webpack plugin to drive the build system including lots of code examples and config files! First, a detour to explain what a Webpack plugin is, for the unfamiliar.

The Webpack lifecycle: A quick primer

Webpack defines itself as a “static module bundler for modern JavaScript applications”. A basic explanation of what it does would be that it takes some files as input, maps out their dependencies, then merges those files into some bundled output.

A simple outline of how Webpack does this:

  1. Define entry points, where Webpack starts
  2. Specify loaders, which define how to load and transform various files
  3. Integrate plugins, which customize how each file is compiled and bundled
  4. Define output, where the final bundled assets are exported

I recommend reading the Webpack Concepts page in their documentation for a bit more in depth outline of how webpack works from a conceptual level.

Webpack configuration

Before getting into the cool stuff, there’s some setup required to get Webpack to work with our stack. This is probably the area most developers who use Webpack are familiar with. Most projects just need to use existing plugins and setup a few config options to get up and runnning.

In our project, we want to use TypeScript so we’ll need to add a TypeScript loader called ts-loader. And we want all our builds to output to the export directory. Let’s also add in file-loader and use it on images. This will load all images with their absolute location on the local disk so we can import images and use them in our React components with ease. In production you’d want to specify the URL of your file hosting here or in the case of a web app, use something like url-loader to load Base64 images inline. For testing emails locally in the browser however, a file url works just fine.

// webpack.config.js; Basic setup
module.exports = {
  entry: "main.tsx",
  output: {
    // Export to the `./export` directory
    path: path.join(__dirname, "export"),
  },
  // Load in files `ts` and `tsx` files
  extensions: [".ts", ".tsx", ".js", ".jsx"],
  rules: [
    // Use the `ts-loader` loader to transform ts(x) files to javascript
    {
      test: /\.ts(x?)$/,
      exclude: /node_modules/,
      use: [
        {
          loader: "ts-loader",
        },
      ],
    },
    // Use the `file-loader loader to transform images to their export locations
    {
      test: /\.(png|jpe?g|gif|svg)$/i,
      loader: "file-loader",
      options: {
        // Ensure the file gets copied to the exports folder
        emitFile: true,
        // Export as the string "./exports/images/[name].[ext]"
        name: "images/[name].[ext]",
        // In production: specify the URL of production file hosting here
        publicPath: path.join(__dirname, "export"),
      },
    },
  ],
  externals: {
    react: "React",
  },
};

This gets us rolling with TypeScript and React! Toss a main.tsx file in the codebase and it’ll generate a main.js file just fine! However, we’re not building a single typescript file, we’ll need to build for a large number of email templates!

Multi-entry configuration

Entry points are where Webpack starts building out it’s dependency graph.

The simplest example of this would be a single JS file that is the sole bundle for a Single Page Application, where all imported dependencies are loaded into this main entry point.

// webpack.config.js; Single entry config, like a Single Page Application
module.exports = {
  // ...
  entry: {
    main: "main.js",
  },
};

If we wanted to generate multiple bundles, we would add new entry points. For example a static website might contain a single bundle for each page. The general rule of thumb is “one entry point for each HTML document”.

// webpack.config.js; Multi-entry config, like a Static Website
module.exports = {
  // ...
  entry: {
    home: "index.js",
    about: "about.js",
    blog: "blog.js",
    "blog/post-0001": "posts/post-0001.js",
  },
};

In our case we decided to treat it like a static website since we have many templates each with their own build output and each generates a different HTML page. Since we have potentially multiple variants for each template, each A/B test variant and the control will all be their own entry point. For localization we just insert different content based on language, we’ll handle that in the plugin itself, which we’ll touch on later.

// webpack.config.js; Our multi-entry config, simplified
module.exports = {
  // ...
  entry: {
    "templateOne/control": "src/templates/templateOne/TemplateOne.tsx",
    "templateTwo/control": "src/templates/templateTwo/TemplateOne.tsx",
    "templateThree/control": "src/templates/templateThree/TemplateOne.tsx",
    "templateThree/variant":
      "src/templates/templateThree/TemplateOne.variant.tsx",
  },
};

That’s great but its very clunky to write, defining a new entry point for every single template variant gets cumbersome fast, and the config would get massive. Since we use Javascript to define the webpack config we can actually simplify this into a helper function that crawls the source code for all our templates. This makes it really simple to add new templates. A developer can drop in a new template to that folder and Webpack will pick it up with the rest.

// webpack.config.js; Our multi-entry config
const getEntriesForTemplates = require("./plugins/utils/getEntriesForTemplates");
module.exports = {
  entry: getEmailTemplateEntryPoints(path.join(__dirname, "./src/templates")),
};

With this config we actually get pretty far towards our goal! Webpack takes our entry points and makes a new dependency graph for each. We’d be able to use React and import any other dependencies and images. Unfortunately we’re still just generating a .js bundle for each template. That might work well for a website, but we want fully build static HTML with no JS to use in emails.

Building the React MJML Plugin

Our plugin is going to need to handle a few things:

  1. Load and validate email properties (per template)
  2. Render each template component from React to MJML to HTML (one per variant)
  3. Render each HTML output to a plaintext version
  4. Output a metadata file (subject, from email, etc)

Structure of a Webpack plugin

To build a webpack plugin, make a new class that has a method apply that takes a compiler object.

// ReactMjmlWebpackPlugin.js
class ReactMjmlWebpackPlugin {
  apply(compiler) {
    console.log("Hello from ReactMjmlWebpackPlugin!");
  }
}
 
// webpack.config.js
module.exports = {
  // ...
  plugins: [
    new ReactMjmlWebpackPlugin({
      templatesPath: path.join(__dirname, "src/templates"),
    }),
  ],
};

From the apply method, we have access to the entire Webpack compiler lifecycle. Super cool! But also super complex! The Webpack compiler does a lot. It builds the dependency graph, loads source files, optimizes bundle output, and a lot more. For this plugin we’re going to focus on a few specific things: loading properties files not normally picked up in the dependency graph, replacing the JS bundle output with static html, rendering extra output (plaintext email, metadata file).

In the apply method, we hook into different stages of the lifecycle. Here’s a simple example, hooking into the compilation object, then again into the “optimize assets” stage of compliation.

// ReactMjmlWebpackPlugin.js
class ReactMjmlWebpackPlugin {
  constructor(options) {
    this.templatesPath =
      options.templatesPath || path.join(process.cwd(), "src/templates");
  }
 
  apply(compiler) {
    compiler.hooks.thisCompilation.tap(
      "ReactMjmlWebpackPlugin",
      (compilation) => {
        console.log("Compilation object created!");
 
        compilation.hooks.optimizeAssets.tapAsync(
          "ReactMjmlWebpackPlugin",
          (assets, callback) => {
            // Print out the asset keys
            console.log("All the assets", Object.keys(assets));
 
            // optimizeAssets is async, we need to call the callback when done
            callback();
          }
        );
      }
    );
  }
}

This example will print “Compilation object created!” then list each of the assets. In our case this will be the already compiled JS bundles, since optimizeAssets occurs after the modules have been bundled, and will look something like:

// webpack.config.js; Entries in webpack
modules.export = {
  entry: {
    "templateOne/control": "src/templates/templateOne/TemplateOne.tsx",
    "templateTwo/control": "src/templates/templateTwo/TemplateOne.tsx",
    "templateThree/control": "src/templates/templateThree/TemplateOne.tsx",
    "templateThree/variant":
      "src/templates/templateThree/TemplateOne.variant.tsx",
    // ...
  },
  // ...
};
 
// Compilation object during optimizeAssets
compilation.assets = {
  "templateOne/control.js": { _value: "/* JS module 1 */" },
  "templateTwo/control.js": { _value: "/* JS module 2 */" },
  "templateThree/control.js": { _value: "/* JS module 3 */" },
  "templateThree/variant.js": { _value: "/* JS module 3 variant */" },
  // ...
};

Notice that the asset keys are actually the files that will be outputted, and they are .js! that’s because we transform the tsx files using ts-loader and the final bundle output will be in JavaScript.

Phase One: Loading email template properties

For our email templates, even though we’re using this new build process, we actually still have more templating to do once the emails are built! We use EJS inserted inside <mj-raw> tags so that the final output of our build can still have variables inserted, such as a user’s name or a dynamic link generated on the server before sending. We have a properties file we use to define this contract with the server as well as A/B information and i18n language data. So we need to load that file, parse it’s data, and validate we have everything we need.

The main question now is: Where do we do this? What part of the Webpack lifecycle should handle this?

We actually aren’t importing the properties json file into each component, so Webpack won’t be automatically adding it to the bundle output.

Some solutions considered:

  1. Import the properties file, just to export it within each template
  2. Add the properties file to entries
  3. Dynamically modify the file dependencies in Webpack plugin to include properties files
  4. Load the properties file within the plugin, add to assets manually

Option 1 required developers to manually add extra code to each template, whereas Options 2-4 handled the properties file automatically. Option 2 felt like it was a misuse of entries, breaking the best practice of “one entry point for each HTML document” (more on that in Addendum on extra dependencies). And Option 3 felt just as hacky as Option 4, but knowing adding manual assets wouldn’t have any side-effects, we opted to just load properties manually using node’s fs module.

The initial thought was to add this to additionalAssets, but we also wanted to use this data to validate the components. Also we were unsure if compilation.assets had the full list of assets at this point, whereas optimizeAssets did. Since that’s where we were doing the rendering (detailed in the next section), we simply added this file loading there.

// inside compiler.hooks.thisCompilation callback...
compilation.hooks.optimizeAssets.tapAsync(
  "ReactMjmlWebpackPlugin",
  (assets, callback) => {
    // Get the template names as an array from the assets,
    // parsing the format `templateName/variantName.js`
    const templateNames = Object.keys(assets)
      .filter((assetKey) => assetKey.endsWith(".js"))
      .map((assetKey) => assetKey.replace(".js", "").split("/")[1]);
 
    templateNames.forEach((templateName) => {
      const propertiesFilePath = path.join(
        this.templatesPath,
        templateName,
        "properties.json"
      );
      const propertiesFile = fs.readFileSync();
      const properties = JSON.parse(propertiesFile);
 
      // Make sure the parsed JSON is valid, has the right info
      // throws if there is an issue, returns valid properties and removes
      // extraneous data the server doensn't need
      const finalProperties = validateTemplateProperties(properties);
 
      // Add the properties.json file to the assets
      // One per template
      const propertiesAssetKey = `${templateName}/properties.json`;
      assets[propertiesAssetKey] = JSON.stringify(finalProperties);
    });
 
    callback();
  }
);

One thing we’d really like to do is not assume that every JS file is a template and instead flag each one with metadata, but there doesn’t seem to be a clean way to do that in Webpack. What we can do, which is how we’ve built the production version of this, is exclusively store all templates in a templates/ directory, so we know all the JS files we find will be the template files, and only look there when doing transformations in the plugin. That process has been left out for the scope of this post.

Phase Two: Evaluating React and rendering to MJML/HTML

Note: We do this phase and the last phase in optimizeAssets, admittedly not a great place to hook into to do this, but there’s not really a better place to do so. Webpack v5 introduces a new hook called processAssets that has stages that are made specifically doing what we’re doing here where we can add extra assets and made assets from existing assets. We’re looking to do a refactor now that Webpack v5 is released.

This is where the atypical Webpack usage begins. So we actually want to render to HTML, we don’t want bundled javascript. So while we take advantage of features of webpack throughout the rest of the build process, here we diverge.

We’re going to take our bundled React components and run them through eval (can be done using node-eval or jsdom). We then render to static markup using React DOM Server. That’ll give us some static mjml content. MJML has it’s own renderer, it’s a simple function called mjml2html, so we’ll call that next.

Note: A word of warning! You don’t want to use eval client-side, we only ever run eval as part of the build process of our own React components since we only have them as bundled JS as strings. See: Never use eval!

Our render looks like: Raw JS string to React component to MJML to HTML.

// Inside compiler.hooks.thisCompilation callback...
compilation.hooks.optimizeAssets.tapAsync(
  "ReactMjmlWebpackPlugin",
  (assets, callback) => {
    // ... loading properties, implemented above
 
    // Remove any asset that's not JS
    const templateAssets = Object.keys(assets).filter(
      (assetKey) => !assetKey.endsWith(".js")
    );
 
    templateAssets.forEach((assetKey) => {
      // Remember the asset keys are the relative export path and filename
      const [templateName, variantName] = assetKey
        .replace(".js", "")
        .split("/");
      const rawJS = assets[assetKey]._value;
 
      // Run eval on the raw JS so we can actually use the React component
      // All of our templates are functional React components using the default export
      const TemplateComponent = _eval(rawJS)["default"];
 
      // Render to mjml, then to html
      const mjml = ReactDOMServer.renderToStaticMarkup(
        React.createElement(TemplateComponent)
      );
      const { html } = mjml2html(mjml);
 
      // Create the new HTML file as an asset
      const templateFilename = `${templateName}/${variantName}.html`;
      assets[templateFilename] = new RawSource(html);
 
      // Finally, delete the original JS asset since we don't need it in the final output
      delete assets[assetKey];
    });
 
    callback();
  }
);

Fantastic! now we have a bunch of new html assets that look something like:

// Compilation object during optimizeAssets
compilation.assets = {
  "templateOne/control.html": { _value: "/* Static HTML 1 */" },
  "templateTwo/control.html": { _value: "/* Static HTML 2 */" },
  "templateThree/control.html": { _value: "/* Static HTML 3 */" },
  "templateThree/variant.html": { _value: "/* Static HTML 3 variant */" },
  // ...
};

These will all get taken by the final steps of the webpack compiler and give us some lovely static HTML emails rendered out to our export directory!

Phase Three: Outputting additional assets

A few additional things to do before we wrap up. That solution is great but we also want to automate a few things. One is our plaintext emails. Here we simply piggy back off the asset definition we did at the end for the HTML and add a new step.

const templateFilename = `${templateName}/${variantName}`;
 
// Same as before
assets[`${templateFilename}.html`] = new RawSource(html);
 
// Additional plaintext asset
const plaintext = htmlToText.fromString(html);
assets[`${templateFilename}.txt`] = new RawSource(plaintext);

Phase Four: Handling i18n

Finally, we want to handle translations with ease, so we have a bunch of i18n keys at the ready for all text content inside the emails. The way we handle this is simply to loop over each language, and render our output one extra time for each language.

templateAssets.forEach((assetKey) => {
  // Nesting the template rendering in a loop of all languages
  allLanguages.forEach((lang) => {
    // Mostly the same template rendering we did before except...
 
    // All of our templates accept a lang prop, and we have an I18nContext that allows us to write all our emails
    // in React without thinking too much about translations, we just need the keys! Easy!
    const mjml = ReactDOMServer.renderToStaticMarkup(
      React.createElement(TemplateComponent, { lang })
    );
 
    // We actually generate a single html and plaintext for each language, for each variant
    const templateFilename = `${templateName}/${variantName}/${lang}`;
    assets[`${templateFilename}.html`] = new RawSource(html);
    assets[`${templateFilename}.txt`] = new RawSource(plaintext);
  });
 
  delete assets[assetKey];
});

Wrap-up

Great! No we have a solid build pipeline all powered by Webpack in one custom plugin. For a high level view of how this plugin is structured take a look at this diagram which shows what different compilation process we hooked into and what steps we took to build each piece of the build system puzzle. There’s plenty more to do, and we’ve built a bunch more on top of this system, like CLI tools for generating and sending emails, reusable components, and more. This should hopefully demystify Webpack plugin development and also show how to tackle some of the less straightforward setups that Webpack can handle, but is lacking in examples or documentation.

Final notes

There were a few tips and implementation details left out of the main part of the post since it would have complicated the example code a bit too much. We still wanted to include them as they are useful to know for anyone stumbling through Webpack plugin development.

Error handling

Uncaught errors may cause Webpack to crash, meaning that it won’t rebuild automatically when the errors are thrown. You can surface these errors as compilation errors by catching them and adding them to the compilation.errors array, rather then letting them bubble up. We made a little pretty print function to make this clear and encourage our plugin maintainers to provide helpful hints to the other devs!

function createPrettyError(message, suggestion) {
  return new Error(`ReactMjmlWebpackPlugin: ${message}\n→ ${suggestion}`);
}
 
// Inside compilation hook
const error = createPrettyError("oh no", "You forgot to turn Webpack on!");
compilation.errors.push(error);

Watching files

Any entry file and it’s dependencies will be automatically watched by Webpack when you run a watch build. Unfortunately since we do some parsing and file access outside of these dependencies, changes to files like i18n data will not rebuild webpack in dev mode.

Luckily you can add to fileDependencies manually, and Webpack will properly watch those files like any other dependency!

Addendum on extra dependencies

While writing this article I found Advanced entries and feel that would be a cleaner way to do Phase One Option 2 (“Add the properties file to entries”), and probably the best solution. I explored a refactor that uses this instead, but Webpack didn’t handle the extra entries very well, and screwed up the way modules were bundled, breaking our React eval step (will go over that in the next section), and I ended up having to ditch the idea.

For more info see: 1234

References

I wouldn’t have made it through this without a few really helpful resources and open source projects:

  1. Inspiration from Mark Dalgleish’s static-site-generator-webpack-plugin
  2. Official docs for Writing a Plugin
  3. Webpack Compiler Hooks documentation
  4. Webpack Compilation Hooks documentation
  5. Sean Larkin’s Everything is a plugin! talk