docspell/website/site/content/docs/tools/smtpgateway.md

+++
title = "SMTP Gateway with Exim"
description = "Start a SMTP server that forwards all mails to docspell."
weight = 50
+++

One possible use case for the [integration
endpoint](@/docs/webapp/uploading.md#integration-endpoint) is a SMTP
server that forwards all local mail to docspell. This way there is no
periodic polling involved and documents (e-mails) get into docspell
without delay.

The `tools/exim` folder contains a docker file and a sample
`exim.conf` to help start with this setup. Note that these files
provide a minimal setup, you might want to add tls and spam protection
when opening it to the public.


# What you need

You need to own a domain and add the appropriate MX records to point
to your server. In this document, the domain `test.org` is used.

You need to enable the [integration
endpoint](@/docs/webapp/uploading.md#integration-endpoint) in the
docspell configuration.

# Exim

[Exim](http://exim.org/) is a popular smtp server (message transfer
agent). It is used here only because of previous knowledge, but same
can be achieved with other MTAs.


# The Config File

Here is the example config file for exim:

{{ incl_conf(path="templates/shortcodes/sample-exim.conf") }}


Exim has good [documentation](https://www.exim.org/docs.html), look
there for more info. The following is only a quick summary of the file
above.

The `domainlist local_domains` should list your domain. Only mails to
this domain are allowed, as specified in the first rule in
`acl_check_rcpt`. So mails to `name@test.org` are ok, but
`name@someother.org` not.

Another rule in `acl_check_rcpt` executes a `GET` request against the
integration endpoint. If that fails, the recipient is wrong (or the
endpoint disabled) and the mail is rejected right away.

Then the `routers` define how a mail is handled. There is only one
router that accepts all mails (that have not been rejected by a rule
in acls) and uses the `docspell` transport to deliver it. The
transport specifies a command via the `pipe` driver that is run with
the mail. The mail itself is provided via stdin. So a simple `curl`
command can upload it to the integration endpoint. Here are some quick
notes about the used options (see `man curl`):

- `--silent` and `--out /dev/null` don't print upload progress
  information and no output to stdout
- `--fail` return non-zero if http status code is not success
- `-F` use a multipart/form-data request (defaults to a POST request)
- `"file=@-;filename=\"$_subject:\""` add one part with name `file`
  and take the data from stdin (`@-`). Since there is no filename, we
  use the subject of the mail. This is [supported by
  exim](http://exim.org/exim-html-current/doc/html/spec_html/ch-string_expansions.html)
  by expanding the subject mail header via `$h_subject:` (the colon is
  required).
- `$local_part` this is expanded by exim to the recipient address,
  only the part until the `@` sign.
- `${env{DS_HEADER}{$value} fail}` looks up an environment variable by
  key `DS_HEADER`. This is usually defined in `docker-compose.yml`.
  The value must be the "secret" header value as defined in docspell's
  configuration file.
- `${env{DS_URL}{$value} fail}` the url to docspell. It is looked up
  from the environment with key `DS_URL`, which is usually defined in
  `docker-compose.yml`. Adding the `$local_part` at the end means that
  mails to `somename@test.org` are uploaded to the collective
  `somename`.


# Install with Docker

Go into the `tools/exim` directory and build the docker image:

``` bash
docker build -t ds-exim:latest -f exim.dockerfile .
```

Then start docspell somewhere and configure the integration endpoint
to use http-header protection; i.e. set this in the config file:

``` conf
docspell.server {
  integration-endpoint {
    enabled = true
    http-header = {
      enabled = true
      header-value = "test123"
    }
  }
}
```

Then edit the `docker-compose.yml` and change the environment
variables as needed.

Finally start the container:

``` bash
docker-compose up
```


# Test Run

Now it is possible to send mails to this MTA which will be immediatly
uploaded to docspell for the collective corresponding to the
`$local_part` of the recipients address. Here is a quick telnet
session (the collective is named `family`):

```
~> telnet localhost 25
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 test.org ESMTP Exim 4.93 Sun, 14 Jun 2020 19:03:51 +0000
ehlo localhost
250-test.org Hello localhost [::1]
250-SIZE 31457280
250-8BITMIME
250-PIPELINING
250-CHUNKING
250 HELP
mail from:<me@test.org>
250 OK
rcpt to:<family@test.org>
250 Accepted
data
354 Enter message, ending with "." on a line by itself
From: me@test.org
To: family@test.org
Subject: This is a test

Test,

this is just a test mail.
.
250 OK id=1jkXwf-000007-0d
quit
221 test.org closing connection
Connection closed by foreign host.
~>
```

The mail is processed and results in an item:

{{ figure(file="exim-mail.png") }}

However, if a mail is to an unknown collective or not to the
configured local domain, the server rejects it immediately:

``` bash
~> telnet localhost 25
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 test.org ESMTP Exim 4.93 Sun, 14 Jun 2020 19:07:04 +0000
ehlo localhost
250-test.org Hello localhost [::1]
250-SIZE 31457280
250-8BITMIME
250-PIPELINING
250-CHUNKING
250 HELP
mail from:<me@test.org>
250 OK
rcpt to:<family22@test.org>
550 Recipient unknown
rcpt to:<family@gmail.com>
550 Administrative prohibition
quit
221 test.org closing connection
Connection closed by foreign host.
~>
```