docspell/website/site/content/docs/addons/control.md

+++ title = "Control Runtime" insert_anchor_links = "right" description = "Control how addons are run" weight = 30 template = "docs.html" +++

Control runtime of addons

Addons are run by the joex component as background tasks in an external process. Depending on the machine it is running on, the addon can be run

• inside a docker container
• inside a systemd-nspawn container
• directly on the machine

Addons can be provided as source packages, where the final program may need to be built. They also can depend on other software. In order to not prepare for each addon, it is recommended to install nix with flakes and docker on the machine running joex.

Please also look at addon section in the default configuration for joex.

You need to explicitly enable addons in the restserver config file.

Docspell uses "runners" to execute an addon. This includes building it if necessary. The following runner exist:

• docker: uses docker to build an run the addon
• nix-flake: builds via nix build and runs the executable in $out/bin
• trivial: simply executes a file inside the addon (as specified in the descriptor)

In the joex configuration you can specify which runners your system supports.

Prepare for running addons

Depending on how you want addons to be run, you need to install either docker and/or systemd-nspawn on the machine running joex. Additionally, the user running joex must be able to use these tools. For docker it usually means to add the user to some group. For systemd-nspawn you most likely want to configure sudo to run passwordless the systemd-nspawn command.

Without this, an addon can only be run "directly" on the machine that hosts joex (which might be perfectly fine). The addon then "sees" all files on the machine and could potentially do harm.

It is recommended to install nix and docker, if possible. Addons may only run with docker or only without, so supporting both leaves more options.

Prepare for building addons

Addons can be packaged as source or binary packages. For the former, joex will build the addon first. There are two supported ways to do so:

• via docker build when the addons provides a Dockerfile (use runner docker)
• via nix build when the addon provides a flake.nix file (use runner nix-flake)

Both build strategies will cache the resulting artifact, so subsequent builds will be (almost) no-ops.

{% infobubble(title="Note") %} Building addons requires to be connected to the internet! Running them may not require a network connection. {% end %}

If the addon is packaged as a binary, then usually the trivial runner (possibly in combination with systemd-nspawn) can be used.

Runtime

Cache directory

Addons can use a "cache directory" to store data between runs. This directory is not cleaned by docspell. If you have concerns about space, use a cron job or systemd-timer to periodically clean this directory.

"Pure" vs "Impure"

Addons can talk back to Docspell in these ways: they can use the http api, for example with dsc, or they can return data to instruct Docspell to apply changes.

The former requires the addon to be connected to the network to reach the Docspell restserver. This allows the addon to do arbitrary changes at any time - this is the "impure" variant.

The second approach can be run without network connectivity. When using docker or systemd-nspawn, Docspell will run these addons without any network. Thus they can't do anything really, except return data back to Docspell.

The pure way is much preferred! It allows for more consistent behaviour, because Docspell is in charge for applying any changes. Docspell can apply changes only if the addon returned successfully. Addons can also be retried on error, because no changes happened yet.

It's the decision of the addon author, how the addon will work. It should document whether it is pure or impure. You can also look into the descriptor and check for a networking: false setting. As the server administrator, you can configure Docspell to only accept pure addons.

Runners

nix flake runner

For addons providing a flake.nix this runner can build it and find the file to execute. With this flake.nix file addons can declare how they should be build and what dependencies are required to run them.

The resulting executable can be executed via systemd-nspawn in a restricted environment or directly on the machine.

{% infobubble(title="Requires") %} You need to install nix and enable flakes to use this runner. {% end %}

docker

Addons can provide a Dockerfile or an image. If no image is given, docker build will be run to build an image from the Dockerfile. Then docker run is used to run the addon.

{% infobubble(title="Requires") %} You need to install docker to use this runner. {% end %}

trivial

Addons can simply declare a file to execute. Docspell can use systemd-nspawn to run it in an restricted environment, or it can be run directly on the machine. This variant is only useful for very simple addons, that don't require any special dependencies.

{% infobubble(title="Requires") %} You need to check each addon for its requirements and prepare the machine accordingly. {% end %}

Choosing runners

The config addons.executor-config.runners accepts a list of runners. It specifies the preferred runner first. If an addon can be executed via docker and nix, Docspell will choose the runner first in the list.

If you don't have nix installed, remove the nix-flake runner from this list and same for docker, of course.

systemd-nspawn

The systemd-nspawn can be used to run programs in a lightweight ad-hoc container. It is available on most linux distributions (it is part of systemd…). It doesn't require an image to exist first; this makes it very convenient for running addons in a restricted environment.

If you enable it in the config file, then all addons are either run via systemd-nspawn or docker - and thus always in a restricted environment, where they can only access their own files and the files provided by Docspell.

The downside is that systemd-nspawn needs to be run as root (as far as I know). Therfore, configure sudo to allow the user that is running joex to execute systemd-nspawn non-interactively.

{% infobubble(title="Requires") %} Install systemd-nspawn and enable the user running joex to use it password-less via sudo. {% end %}

Within Docker

If joex itself is run as a docker container, things get a bit complicated. The default image for joex does not contain nix, so the nix-flake runner cannot be used out of the box.

In order to use the docker runner, the container must be configured to access the hosts docker daemon. On most systems this can be achieved by bind-mounting the unix socket (usually at /var/run/docker.sock) into the container. Here is a snippet from the provided docker-compose file:

  joex:
    image: docspell/joex:latest
    # ... left out for brevity
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp:/tmp

Additionally to /var/run/docker.sock, it also bind mounts the /tmp directory. This is necessary, because docker will be invoked with bind mounts from inside the continer - but these must be available on the host, because the docker client in the container actually runs the command on the host.

The addon executor uses the systems temp-directory (which is usually /tmp) as a base for creating a working and cache directory. Should you change this in joex config file (or your system uses a different default temp-dir), then the bind mount must be adapted as well.

Another variant is to extend the default joex image and add more programs as needed by addons and then use the trivial runner.

Summary / tl;dr

When joex is not inside a container:

• (optional) Install systemd-nspawn - it is provided on many GNU/Linux distributions
• Configure sudo to allow the user running the joex component to execute systemd-nspawn non-interactively (without requiring a password)
• Install docker
• Install nix and enable flakes
• Allow the user who runs the joex component to use docker and nix. If you install nix as multi-user, then this is already done.
• Check the section on addons in the default configuration for joex