Technology Evaluation - Rendering Markdown

From wiki.gpii
Jump to: navigation, search

Introduction

In our work we make extensive use of the Markdown language. For example, we use it to:

  1. Create documentation (in "GitHub Flavored Markdown") viewable directly from GitHub repositories. Nearly all the projects we host on GitHub include a README.md file like this one from gpii-handlebars.
  2. Generate [docs.fluidproject.org/infusion the Infusion documentation] from the associated GitHub repo.
  3. Render API docs, for example, in the UL API docs.
  4. Safely render arbitrary content provided by end users and database vendors.

The last three use cases make use of the marked library. Although this has met our needs to date, it seems to be largely unmaintained, and is increasingly a source of inherited security vulnerabilities. We use the security vulnerability scanner Snyk to monitor our projects. Projects that depend on marked currently inherit a range of high and medium impact vulnerabilities from marked. Previous vulnerabilities have taken months to resolve. In this case, a fix was accepted and merged nine months ago, but there is apparently no one able or willing to cut a new release. This puts us in the position of either:

  1. Creating and maintaining a fork of the marked library ourselves.
  2. Making use of snyk's patches as part of our dependency installation and release process.
  3. Making use of a better-maintained library with less security vulnerabilities.

Although we can certainly discuss the benefits of the first two approaches, they imply ongoing maintenance work beyond simply upgrading our dependencies periodically. Since there are many many other libraries that support markdown rendering, it seemed fair to evaluate them and see if we can find a suitable replacement.

Requirements

As far as I know, as I write this our primary uses of marked as a dependency come from:

  1. Our use of the docpad-marked-plugin in our documentation project.
  2. Our use of marked within the gpii-handlebars package.

Here is a combined list of requirements based on how we use both. An ideal solution:

  1. Must be able to render markdown as HTML.
  2. Must support mixing raw HTML with markdown, as we commonly do in the Infusion documentation.
  3. Should provide security controls to automatically strip problematic HTML (script tags, et cetera).
  4. Should be well maintained, i. e. we need clear evidence that security issues and bugs will be addressed in a timely manner.
  5. Should support both GitHub-flavored Markdown and "normal" markdown. If possible, should support the the CommonMark spec and the relatively new GfM spec.
  6. Should introduce as few security vulnerabilities as possible (ideally none).

Candidates

Docpad Plugin Candidates

I compiled my initial list of candidates from the list of renderer plugins on the Docpad site and from searching npm.

docpad-plugin-marked

This is our current solution. It depends on a very old version of marked with even more security vulnerabilities than the latest version. The plugin itself has a small community that is infrequently active.

docpad-plugin-markit

This plugin is based on markdown-it, which is well-maintained. The plugin itself has a smaller community than docpad-plugin-marked, and is also infrequently active.

docpad-plugin-robotskirt

Although this plugin used another rendering library (robotskirt) which is well-maintained, the plugin itself is no longer maintained.

docpad-plugin-markdown

Although this plugin used yet another rendering library (github-flavored-markdown) which is well-maintained, the plugin itself is also not maintained.

Conclusion

As there is really only one other viable candidate, docpad-plugin-markit should be evaluated in depth to confirm whether it meets our needs.

Node and Browser Candidates

I compiled my initial list of candidates by reviewing the list of CommonMark implementations and by searching npm for both "markdown" and "gfm".

Solution Node Browser Markup Variants HTML Vulnerabilities Community Notes
commonmark.js Yes Yes CommonMark only All or nothing "safe" mode to strip HTML altogether and sanitise links. No vulnerabilities at time of writing Reasonably sized community, but moderately slow release cadence (1-2 releases a year).
markdown-it Yes Yes Supports CommonMark. Supports some GfM conventions (tables, strikethrough) natively, others via plugins. Has an "html" option to enable HTML tags in markup. No vulnerabilities in latest version at time of writing Good sized community, healthy release cadence, seem to have good turnaround on issues and pull requests. A key benefit of markdown-it is that it uses the same parser as docpad-markit-plugin, which means we'd only be supporting one renderer across our holdings.
markdown Yes Yes Very unclear which variant of markdown it supports, but no clear support for GfM. Not clear if it supports mixed-in HTML. No vulnerabilities at time of writing No releases in over 4 years, seems like this is no longer actively maintained.
marked Yes Yes Supports "original" markup and pre-spec GfM. It is possible to sanitise all HTML, but fine-grained controls require writing your own customised functions. Multiple high vulnerability security issues. Very poor release cadence, seems like the project is abandoned at this point. Our current solution.
remark.js Yes No "Original", CommonMark, and GfM. Seems to support embedded HTML, but unclear how this is controllable (very deep stack to wade through). No vulnerabilities at time of writing. Active community, moderately frequent releases.
showdown Yes Yes Markup Variants Supports GfM and "original" markdown. No vulnerabilities at time of writing Several dozen committers, good release cadence and healthy turnaround on issues and pull requests.

Only two projects (other than "marked" itself) meet all the requirements:

  1. markdown-it
  2. showdown

Of these, markdown-it seems like the best supported and best aligned with our need to work with DocPad.

Conclusion

I propose proceeding to submit pull requests to use markdown-it in gpii-handlebars and to use the associated Docpad plugin in the Infusion docs project.