Creating Filters

Overview

If the base features of Pandoc and Quarto don’t do exactly what you need, you can very likely create a Pandoc Filter that bridges the gap.

Pandoc consists of a set of readers and writers. When converting a document from one format to another, text is parsed by a reader into pandoc’s intermediate representation of the document—an “abstract syntax tree” or AST—which is then converted by the writer into the target format. The pandoc AST format is defined in the module Text.Pandoc.Definition in the pandoc-types package.

A “filter” is a program that modifies the AST, between the reader and the writer.

INPUT --reader--> AST --filter--> AST --writer--> OUTPUT

Pandoc’s built-in citation processing is implemented as a filter, as are many of Quarto’s internal extensions (e.g. cross-references, figure layout, etc.).

You can write Pandoc filters using Lua (via Pandoc’s built-in Lua interpreter) or using any other language using a JSON representation of the Pandoc AST piped to/from an external process. We strongly recommend using Lua Filters, which have the following advantages:

  • No external dependencies
  • High performance (no serialization or process execution overhead)
  • Access to the Pandoc and Quarto libraries of Lua helper functions.

See the Pandoc documentation on Writing Lua Filters for additional details. If you aren’t familiar with Lua you may want to checked out the article on Developing with Lua for pointers on getting started.

If you want to write a JSON filter, see the documentation on Writing JSON filters.

Activating Filters

If you’ve developed a filter and want to use it within a document you need to add it to the list of filters for the document. For example, here we arrange for the spellcheck filter to run:

---
filters:
  - spellcheck.lua
---

By default, user filters are run after Quarto’s built-in filters. For some filters you’ll want to modify this behavior. For example, here we arrange to run spellcheck before Quarto’s filters and lightbox after:

filters:
  - spellcheck.lua
  - quarto
  - lightbox

You’ll notice that one of the extensions (spellcheck.lua) has a file extension and the other (lightbox) does not. This difference stems from how the extensions are distributed: an extension distributed as a plain Lua file uses .lua whereas a filter distributed as a Quarto Extension does not. The next section explores how to distribute filters as extensions.

Filter Extensions

If you have developed a filter that you’d like to distribute to others you can bundle the filter as a Quarto Extension. Bundling a filter as an an extension makes it much easier for others to install, update, and use your filter.

Here is what the source code repository of an an extension named output-folding might look like:

README.md
LICENSE
example.qmd
_extensions/
  output-folding/
    _extension.yml
    output-folding.lua

Note that the only thing strictly required is the _extensions directory (anything above that is for your own purposes and is ignored during installation). Even so, it’s good practice to include a README.md and LICENSE file, and the example.qmd will be useful for developing your extension.

Here’s what the contents of the files in _extensions/output-folding/ might look like:

_extensions/output-folding/_extension.yml
title: Output Folding
author: Cooltools
version: 1.0.0
contributes:
  filters:
    - output-folding.lua
_extensions/output-folding/output-folding.lua
function Div(el)
  -- code required to implement output folding
end

Finally, the example.qmd file would typically have code that exercises the extension. For example:

example.qmd
---
title: "Output Folding Example"
filters:
  - output-folding
---

```{python}
#| output-fold: true

1 + 1
```

To develop your extension, just make changes to output-folding.lua and render example.qmd to test them out.

Distribution

if your extension source code it located within a GitHub repository, then it can be installed by referencing the GitHub organization and repository name. For example:

Terminal
# install the current HEAD of the extension
quarto install extension cooltools/output-folding

# install a tagged release of the extension
quarto install extension cooltools/output-folding@v1

Note that it is possible to bundle and distribute extensions as simple gzip archives (as opposed to using a GitHub repository as described above). See the article on Distributing Extensions for additional details.

Examples

You might also find it instructive to examine the source code of these filter extensions authored by the Quarto team:

Extension name Description
code-filename Add a filename header to code blocks
grouped-tabsets Add grouped tabsets, which remember the active tab across multiple HTML documents.
latex-environment Quarto extension to output custom LaTeX environments.
lightbox Create lightbox treatments for images in your HTML documents.