The Marked help site, and the copy that ships inside Marked, is generated from a pile of Markdown in a single folder. For a long time that pile was fed to MultiMarkdown. It now goes through Apex.

I’ve been building this documentation for over a decade, so there’s a lot to take into consideration when switching the build process so drastically. But the switch ended up being pretty painless. I pointed the help builder at the Apex binary, dropped a project config next to the sources, and the pages I already had kept rendering. The interesting part was everything I could add once that was in place.

Using Apex, I was able to integrate a lot of the build process I’d developed in custom Ruby scripts into the processor itself, giving me all the liquid syntax shortcuts and plugins I needed. The advanced image and attribute syntax made things really easy, and a little search and replace let me take full advantage of Apex’s advanced features.

If you haven’t checked out Apex yet, the Apex site has the overview, and the Apex wiki has the details.

The build just shells out

HelpDocs/config.yaml still has a Processor key. The builder reads a page, pipes the Markdown to that command, and wraps the HTML in the help template. The line looks like this:

Processor: "env XDG_CONFIG_HOME=/var/empty /opt/homebrew/bin/apex"

XDG_CONFIG_HOME=/var/empty keeps my personal Apex config out of the docs build. Apex then picks up HelpDocs/.apex/config.yml, because the build runs with HelpDocs as the working directory. For each page, the builder also sets HELP_CURRENT_PAGE to the page’s filename. One of the plugins needs that.

What I turned on

Unified mode, MultiMarkdown-style header ids, and plugins. The rest is there because the existing docs already depended on it.

id-format: mmd
plugins: true
widont: true
wikilinks: true
wikilink-space: underscore
wikilink-extension: html
image-captions: true
markdown-in-html: true
obfuscate-emails: true

mode: unified
standalone: false
pretty: true

language: en
quoteslanguage: english

autolink: true
includes: true
relaxed-tables: true
sup-sub: true
indices: false

code-highlight: pygments
code-line-numbers: false
highlight-language-only: true

id-format: mmd keeps header ids in the old MultiMarkdown style, without dashes, so in-page anchors did not all move. standalone: false matters because the help builder already supplies the HTML shell. markdown-in-html is on so a plugin can emit a <div markdown="1"> or a blockquote and Apex will still parse the Markdown inside it. highlight-language-only stops Pygments from guessing a language for every unlabeled fence.

Wiki links are configured for this site specifically: spaces become underscores, and links get an .html extension. [[Custom Processor]] becomes a link to Custom_Processor.html.

Explicit [[wiki links]] are only half of it. Most of the cross-linking in the docs is automatic, and it comes from the same config.yaml that defines the sidebar.

Each page entry can carry a title and a wiki_keywords list:

- title: Choosing a Processor
  file: Choosing_a_Processor
  wiki_keywords:
    - which processor
    - markdown processor
    - choose processor

A pre-parse plugin, doc-wiki-links, reads that file and builds a map of titles and keywords to File.html. Before Apex parses the page, the plugin walks the text and turns the first mention of each destination into a Markdown link. Longer phrases win, so “markdown processor” is linked before a bare “processor” can steal part of it. It only links a given page once per paragraph.

It skips the current page (that’s what HELP_CURRENT_PAGE is for), headings, fenced and indented code, inline code, links that are already there, and the little section table of contents another plugin inserts. If anything goes wrong it prints the original text and the build continues.

The plugin is not a public project, but if you’re interested, just contact me. It lives in HelpDocs/.apex/plugins/doc-wiki-links/, with a plugin.yml that marks it pre_parse. Apex loads it because plugins: true is set. The help builder never calls it directly. It just runs Apex.

The protocol is small. Apex writes a JSON object to the plugin’s stdin (phase, text, and a plugin id) and expects transformed Markdown on stdout. The readable implementation is Ruby. The plugin.yml points at a compiled binary (Crystal) so a full site build is not spawning Ruby for every page.

The shortcut plugin

The other local plugin is kbd. The docs have used a Liquid-ish shorthand for years, and I did not want to rewrite hundreds of pages just to change processors. kbd runs in the same pre-parse phase and rewrites that shorthand into HTML before Apex sees it.

A few of the things it still handles:

Press {% kbd cmd shift O %} to open Quick Open.

{% appmenu File, Print ({{cmd}}P) %}

See {% prefspane Export %}.

T> This shows up as a tip.
W> This shows up as a warning.

{% apponly %}Only in the in-app help.{% endapponly %}
{% browseronly div %}
Only on the website.
{% endbrowseronly %}

{{cmd}}, {{shift}}, {{opt}}, and the rest become the usual Mac key entities. {% kbd %} wraps combos in <kbd> tags and orders the modifiers the way Apple likes them. {% appmenu %} renders a menu path. {% prefspane %} links into Marked’s settings via the x-marked-3://pref/ URL scheme. T>, W>, E>, and I> become classed blockquotes. {% apponly %} and {% browseronly %} wrap chunks so the in-app help and the website can hide what does not belong. Notes and TODOs get stripped. If a page has more than one ## heading, it also injects a section contents list.

markdown-in-html is what makes the block form useful. The plugin can wrap a div and Apex will still render the Markdown inside it.

Images, including the modern formats

Help pages have always used reference-style images with an @2x token, a width, and a class:

[prefs]: images/screenshots/preferences-Processor.jpg width=689px class=preferencepane-scroll

That still works. My previous build system automatically checked for @2x versions and generated retina srcsets, but Apex allows me to just add @2x to an image tag. Adding modern formats is the same kind of token. webp and avif sit next to @2x:

[prefs]: images/screenshots/preferences-Processor.jpg @2x webp avif width=689px class=preferencepane-scroll

Apex emits a <picture>. AVIF first, then WebP, each with a 1x and a 2x candidate, and the original JPG stays on the <img> as the fallback:

<picture>
  <source type="image/avif" srcset="images/screenshots/preferences-Processor.avif 1x, images/screenshots/preferences-Processor@2x.avif 2x">
  <source type="image/webp" srcset="images/screenshots/preferences-Processor.webp 1x, images/screenshots/preferences-Processor@2x.webp 2x">
  <img src="images/screenshots/preferences-Processor.jpg" alt="" width="689px" class="preferencepane-scroll">
</picture>

image-captions: true still wraps titled images in a figure. I did not have to invent a new image syntax to get retina assets and a JPG fallback in the same reference.

MultiMarkdown still just works

This is the part I was most ready to fight with, and then didn’t. Metadata, definition lists, footnotes, reference links, superscripts, and those image attributes all came along. I did not run a conversion pass on the docs. Unified mode already speaks the dialect I had been writing.

The lock-in caveat

That convenience has a flip side, and it is one I keep running into while developing Apex.

The point of the processor is that a document from somewhere else should render without a rewrite. CommonMark, GitHub Flavored Markdown, MultiMarkdown, Kramdown: pick a mode and Apex sticks to that flavor.

apex input.md --mode commonmark
apex input.md --mode gfm
apex input.md --mode mmd
apex input.md --mode kramdown

Unified mode is the other direction. It turns on the union of those flavors, plus things none of them have. Wiki links, picture tokens, callouts, plugins, metadata variables, the whole list.

So it is entirely possible to use all of that in one file and produce a document that no single other processor can read. A MultiMarkdown file still works. A file that mixes MultiMarkdown metadata, Kramdown attribute lists, GFM tables, [[wiki links]], and webp avif @2x image tokens does not survive a round trip through Pandoc or MultiMarkdown or cmark-gfm.

The restrictive modes exist so you can refuse that. Choosing not to use them can lock you into Apex. The Marked docs are fine with that. They are built with Apex on purpose, and the plugins only exist inside this repo. If I were publishing the source and expecting someone else’s processor to consume it, I would pin a mode and stay inside it.

See it in action

Take a look through the Marked help to see the end result. Then check out Apex and start building!