Add documentation about docker setup

This commit is contained in:
eikek
2021-05-31 13:25:57 +02:00
parent 3e7b66fd42
commit bdc7822f50
22 changed files with 568 additions and 299 deletions

View File

@ -0,0 +1,242 @@
+++
title = "Docker"
weight = 20
+++
## Docker Images
The docker images are at
[hub.docker.com](https://hub.docker.com/u/docspell). The `latest` tag
always points to the latest *release*. The releases are also tagged
with their respective version number. Additionally, there are images
tagged with `nightly` which are built from the `master` branch.
Therefore the `nightly` packages should be used with care, because
things might break in between. But they are useful for trying out
something.
There are images for all components that are available from the github
release page. The images contain all the necessary
[prerequisites](@/docs/install/prereq.md).
- `docspell-restserver` this images contains the http server
- `docspell-joex` this image contains the job executor and all
required software (ocrmypdf, unoconv etc) mentioned in
[prerequisites](@/docs/install/prereq.md).
- `docspell-tools` this image simply contains all the scripts from the
`tools/` folder. They are installed into a location in `$PATH`. It
doesn't specify a `CMD` or `ENTRYPOINT`, so you must choose which
script to run. The scripts are all prefixed by `ds-`. So to run the
`consumedir.sh` script, execute `ds-consumedir`.
### Examples
These examples use `docker run` to start the restserver and
jobexecutor. Both must be connected to the same database. For this
example, a shared directory is used and the in-process database H2.
For a real setup, using PostgreSQL is recommended.
This requires to change the default config. This example creates a new
config file. Please refer to the [configuration
page](@/docs/configure/_index.md) for more details.
``` bash
$ cat > /tmp/docspell.conf <<-"EOF"
# common settings
db_url = "jdbc:h2:///var/docspell/db;MODE=PostgreSQL;DATABASE_TO_LOWER=TRUE;AUTO_SERVER=TRUE"
# job executor settings
docspell.joex.jdbc.url = ${db_url}
docspell.joex.base-url = "http://10.4.3.2:7878"
docspell.joex.bind.address = "0.0.0.0"
# restserver settings
docspell.server.backend.jdbc.url = ${db_url}
docspell.server.bind.address = "0.0.0.0"
docspell.server.integration-endpoint {
enabled = true
http-header {
enabled = true
header-value = "test123"
}
}
EOF
```
This sets the db url to the same value for both components; thus we
can use the same file for both components. It also sets the bind
address to bind the server socket on all interfaces. Another thing to
note is the `base-url` setting for joex. This publishes joex by this
ip, such that the server component can notify the job executor for new
work. The `integration-endpoint` setting is explained later.
After creating a common network, we start the rest server and joex:
```
$ docker network create --subnet 10.4.3.0/24 dsnet
$ docker run -d --name ds-restserver \
--network dsnet --ip 10.4.3.3 \
-p 127.0.0.1:7880:7880 \
-v /tmp/testdb:/var/docspell \
-v /tmp/docspell.conf:/opt/docspell.conf \
docspell/restserver:latest /opt/docspell.conf
$ docker run -d --name ds-joex \
--network dsnet --ip 10.4.3.2 \
-v /tmp/testdb:/var/docspell \
-v /tmp/docspell.conf:/opt/docspell.conf \
docspell/joex:latest /opt/docspell.conf
```
After this `docker ps` should show these two containers. Go to
`http://localhost:7880` and sign up/login and start playing around.
When signing up, use the same name for collective and user and then
login with this name.
For the last part, we use the `docspell/tools` image to create another
container that watches a directory and pushes files to docspell.
``` bash
$ docker run -d --name ds-consume \
--network dsnet --ip 10.4.3.4 \
-v /tmp/inbox:/var/inbox \
docspell/tools:latest ds-consumedir -imdv --iheader "Docspell-Integration:test123" \
--path /var/inbox "http://10.4.3.3:7880/api/v1/open/integration/item"
```
This starts the [consumedir](@/docs/tools/consumedir.md) script that
watches a directory and uploads arriving files to docspell server.
This requires the value from the `integration-endpoint` setting to be
allowed to upload files. It also requires you to explicitely enable
this: go to *Collective Profile → Settings* and enable the
*Integration Endpoint*. Then create a subdirectory in `/tmp/inbox`
with the name of the *collective* that you registered and place a file
into the `/tmp/inbox/[collective]` directory. The file is pushed to
docspell and processed shortly after.
To see all available options, run the script with the `--help` option:
``` bash
$ docker run docspell/tools:latest ds-consumedir --help
```
Note that this is just an example and is only to demonstrate how to
use the docker images. For instance, this setup does not provide
fulltext search. For a more sophisticated docker setup, use
appropriate tools, for example `docker-compose` which is explained
below.
## Docker Compose
There is a [docker-compose](https://docs.docker.com/compose/) setup
available in the `/docker/docker-compose` folder. This setup is
similiar to the example above, adding fulltext search and a PostgreSQL
database by using just one command. It's only 3 steps to get started:
### Start Docspell
#### Get the docker-compose files
Either via cloning the whole repository:
```bash
$ git clone https://github.com/eikek/docspell
```
This downloads all sources. What you need is only one subdirectory. So
if you don't have git or don't want to clone the whole repo, use these
steps instead:
``` bash
$ mkdir -p docspell/docker/docker-compose
$ cd docspell/docker/docker-compose
$ wget https://raw.githubusercontent.com/eikek/docspell/master/docker/docker-compose/docker-compose.yml
$ wget https://raw.githubusercontent.com/eikek/docspell/master/docker/docker-compose/docspell.conf
$ wget https://raw.githubusercontent.com/eikek/docspell/master/docker/docker-compose/.env
```
You can choose any directory instead of
`docspell/docker/docker-compose`, of course.
Change into the new `docker-compose` directory, for example:
```bash
$ cd docspell/docker/docker-compose
```
#### Run `docker-compose up`
```bash
$ export DOCSPELL_HEADER_VALUE="my-secret-123"
$ docker-compose up
```
The environment variable defines a secret that is shared between some
containers. You can define whatever you like. Please see the
[consumedir.sh](@/docs/tools/consumedir.md#docker) docs for additional
info.
Goto `http://localhost:7880`, signup and login. When signing up, you
choose the same name for collective and user. Then login with this
name and the password.
(Optional) Create a folder `./docs/<collective-name>` (the name you
chose for the collective at registration) and place files in there for
importing them.
The directory contains a file `docspell.conf` that you can
[modify](@/docs/configure/_index.md) as needed.
### Override this setup
If you want to change this setup, you can simply use your own compose
file or add a `docker-compose.override.yml` that allows to amend
certain configs. Look [here](https://docs.docker.com/compose/extends/)
to find more about it.
As an example, here is a `docker-compose.override.yml`:
``` yaml
version: '3.7'
services:
consumedir:
volumes:
- importdocs:/opt/docs
volumes:
docspell-postgres_data:
driver: local
driver_opts:
type: nfs4
o: addr=192.168.x.y,rw,noatime,rsize=8192,wsize=8192,tcp,timeo=14
device: ":/mnt/FreeNas/docker_vol1/docspell/postgres_data"
docspell-solr_data:
driver: local
driver_opts:
type: nfs4
o: addr=192.168.x.y,rw,noatime,rsize=8192,wsize=8192,tcp,timeo=14
device: ":/mnt/FreeNas/docker_vol1/docspell/solr_data"
importdocs:
driver: local
driver_opts:
type: nfs4
o: addr=192.168.x.y,rw,noatime,rsize=8192,wsize=8192,tcp,timeo=14
device: ":/mnt/FreeNas/archiv/gescannt/output"
```
### Upgrading
Since [downgrading](@/docs/install/downgrading.md) is not supported,
it is recommended to backup your database before upgrading. Should
something not work as expected, restore the database backup and go
back to the previous version.
The latest release is always tagged with `latest`. Should you use this
tag, then run these commands to upgrade to newer images:
``` bash
$ docker-compose down
$ docker-compose pull
$ docker-compose up --force-recreate --build -d
```

View File

@ -0,0 +1,121 @@
+++
title = "Download & Run"
weight = 22
+++
You can install via zip or deb archives. Please see the
[prerequisites](@/docs/install/prereq.md) first.
## Using zip files
You need to download the two files:
- [docspell-restserver-{{version()}}.zip](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-restserver-{{version()}}.zip)
- [docspell-joex-{{version()}}.zip](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-joex-{{version()}}.zip)
1. Unzip both files:
``` bash
$ unzip docspell-*.zip
```
2. Open two terminal windows and navigate to the the directory
containing the zip files.
3. Start both components executing:
``` bash
$ ./docspell-restserver*/bin/docspell-restserver
```
in one terminal and
``` bash
$ ./docspell-joex*/bin/docspell-joex
```
in the other.
4. Point your browser to: <http://localhost:7880/app>
5. Register a new account, sign in and try it.
Note, that this setup doesn't include watching a directory nor
fulltext search. Using zip/deb files requires to take care of the
[prerequisites](@/docs/install/prereq.md) yourself.
The provided scripts in
[docspell-tools-{{version()}}.zip](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-tools-{{version()}}.zip)
must be extracted and installed manually somewhere in your `$PATH`.
## Using deb files
Packages are also provided at the release page:
- [docspell-restserver_{{version()}}_all.deb](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-restserver_{{version()}}_all.deb)
- [docspell-joex_{{version()}}_all.deb](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-joex_{{version()}}_all.deb)
The DEB packages can be installed on Debian, or Debian based Distros:
``` bash
$ sudo dpkg -i docspell*.deb
```
Then the start scripts are in your `$PATH`. Run `docspell-restserver`
or `docspell-joex` from a terminal window.
The packages come with a systemd unit file that will be installed to
autostart the services.
The provided scripts in `docspell-tools-{{version()}}.zip` must be
extracted and installed manually somewhere in your `$PATH`. There are
no deb files provided.
## Running
Run the start script (in the corresponding `bin/` directory when using
the zip files):
```
$ ./docspell-restserver*/bin/docspell-restserver
$ ./docspell-joex*/bin/docspell-joex
```
This will startup both components using the default configuration.
Please refer to the [configuration page](@/docs/configure/_index.md)
for how to create a custom config file. Once you have your config
file, simply pass it as argument to the command:
```
$ ./docspell-restserver*/bin/docspell-restserver /path/to/server-config.conf
$ ./docspell-joex*/bin/docspell-joex /path/to/joex-config.conf
```
After starting the rest server, you can reach the web application
`http://localhost:7880/`.
You should be able to create a new account and sign in. When creating
a new account, use the same name for collective and user and then
login with this name.
## Upgrading
Since [downgrading](@/docs/install/downgrading.md) is not supported,
it is recommended to backup your database before upgrading. Should
something not work as expected, restore the database backup and go
back to the previous version.
When using the zip or deb files, either install the new deb files via
your package manager or download and unpack the new zip files. You
might want to have a look at the changelog, since it is sometimes
necessary to modify the config file.
## More
### Fulltext Search
Fulltext search is powered by [SOLR](https://solr.apache.org). You
need to install solr and create a core for docspell. Then cange the
solr url for both components (restserver and joex) accordingly. See
the relevant section in the [config
page](@/docs/configure/_index.md#full-text-search-solr).
### Watching a directory
The [consumedir](@/docs/tools/consumedir.md) script can be used for
this. Using systemd or something similar, it is possible to create a
system service that runs the script in "watch mode".

View File

@ -1,104 +1,8 @@
+++
title = "Installing"
weight = 20
title = "Nix / NixOS"
weight = 24
+++
# Docker
There is a [docker-compose](https://docs.docker.com/compose/) setup
available in the `/docker` folder. This setup is also taking care of
all the necessary [prerequisites](@/docs/install/prereq.md) and
creates a container to watch a directory for incoming files. It's only
3 steps:
1. Clone the github repository
```bash
$ git clone https://github.com/eikek/docspell
```
If you don't have git or don't want to clone the whole repo, use these steps instead:
``` bash
mkdir -p docspell/docker
cd docspell/docker
wget https://raw.githubusercontent.com/eikek/docspell/master/docker/docker-compose.yml
wget https://raw.githubusercontent.com/eikek/docspell/master/docker/docspell.conf
wget https://raw.githubusercontent.com/eikek/docspell/master/docker/.env
```
2. Change into the `docker` directory:
```bash
$ cd docspell/docker
```
3. Run `docker-compose up`:
```bash
$ export DOCSPELL_HEADER_VALUE="my-secret-123"
$ docker-compose up
```
The environment variable defines a secret that is shared between
some containers. You can define whatever you like. Please see the
[consumedir.sh](@/docs/tools/consumedir.md#docker) docs for
additional info.
4. Goto `http://localhost:7880`, signup and login. When signing up,
you can choose the same name for collective and user. Then login
with this name and the password.
5. (Optional) Create a folder `./docs/<collective-name>` (the name you
chose for the collective at registration) and place files in there
for importing them.
The directory contains a file `docspell.conf` that you can
[modify](@/docs/configure/_index.md) as needed.
# Download, Unpack, Run
You can install via zip or deb archives. Please see the
[prerequisites](@/docs/install/prereq.md) first.
## Using zip files
You need to download the two files:
- [docspell-restserver-{{version()}}.zip](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-restserver-{{version()}}.zip)
- [docspell-joex-{{version()}}.zip](https://github.com/eikek/docspell/releases/download/v{{version()}}/docspell-joex-{{version()}}.zip)
1. Unzip both files:
``` bash
$ unzip docspell-*.zip
```
2. Open two terminal windows and navigate to the the directory
containing the zip files.
3. Start both components executing:
``` bash
$ ./docspell-restserver*/bin/docspell-restserver
```
in one terminal and
``` bash
$ ./docspell-joex*/bin/docspell-joex
```
in the other.
4. Point your browser to: <http://localhost:7880/app>
5. Register a new account, sign in and try it.
Note, that this setup doesn't include watching a directory. You can
use the [consumedir.sh](@/docs/tools/consumedir.md) tool for this or
use the docker variant below.
## Using deb files
The DEB packages can be installed on Debian, or Debian based Distros:
``` bash
$ sudo dpkg -i docspell*.deb
```
Then the start scripts are in your `$PATH`. Run `docspell-restserver`
or `docspell-joex` from a terminal window.
The packages come with a systemd unit file that will be installed to
autostart the services.
# Nix
## Install via Nix
@ -159,6 +63,17 @@ selecting the most current release. For example it translates to
is `{{version()}}`.
## Upgrading
Since [downgrading](@/docs/install/downgrading.md) is not supported,
it is recommended to backup your database before upgrading. Should
something not work as expected, restore the database backup and go
back to the previous version.
When using the provided nix setup, the `currentPkg` always points to
the latest release. Thus it is enough to run `nix-build`.
## Docspell on NixOS {#nixos}
If you are running [NixOS](https://nixos.org), there is a module

View File

@ -75,7 +75,7 @@ no fulltext search.
When installing manually (i.e. not via docker), just install solr and
create a core as described in the [solr
documentation](https://lucene.apache.org/solr/guide/8_4/installing-solr.html).
documentation](https://solr.apache.org/guide/8_4/installing-solr.html).
That will provide you with the connection url (the last part is the
core name).

View File

@ -5,28 +5,43 @@ weight = 0
To get started, here are some quick links:
- Using [docker and
docker-compose](@/docs/install/installing.md#docker). This sets up
everything: all prerequisites, both docspell components and a
container running the [consumedir.sh](@/docs/tools/consumedir.md)
- Using [docker and docker-compose](@/docs/install/docker.md). This
sets up everything: all prerequisites, both docspell components and
a container running the [consumedir.sh](@/docs/tools/consumedir.md)
script to import files that are dropped in a folder.
- [Download, Unpack and
Run](@/docs/install/installing.md#download-unpack-run). This option
is also very quick, but you need to check the
- [Download, Unpack and Run](@/docs/install/download_run.md). This
option is also very quick, but you need to check the
[prerequisites](@/docs/install/prereq.md) yourself. Database is
already setup, but you'd need to setup SOLR (when using fulltext
search) and install some programs for the joex component. This
applies to the `zip` and `deb` files. The files can be downloaded
from the [release page](https://github.com/eikek/docspell/releases/latest).
- via the [nix package manager](@/docs/install/installing.md#nix) and/or as a [NixOS
module](@/docs/install/installing.md#nixos). If you use nix/nixos, you
know what to do. The linked page contains some examples.
- [Unraid](https://www.unraid.net/): While not official, there are
user provided [notes and unraid
from the [release
page](https://github.com/eikek/docspell/releases/latest).
- via the [nix package manager](@/docs/install/nix.md) and/or as a
[NixOS module](@/docs/install/nix.md#nixos). If you use nix/nixos,
you know what to do. The linked page contains some examples.
- [Unraid](https://www.unraid.net/): There are user provided [notes
and unraid
templates](https://github.com/vakilando/unraid-docker-templates)
which can get you started. Thanks for providing these!
Every [component](@docs/intro/_index.md#components) (restserver, joex,
Every [component](@/docs/intro/_index.md#components) (restserver, joex,
consumedir) can run on different machines and multiple times. Most of
the time running all on one machine is sufficient and also for
simplicity, the docker-compose setup reflects this variant.
While there are many different ways to run docspell, at some point all
call docspell binaries. These accept one argument: a [config
file](@/docs/configure/_index.md). If this is not given, the default
is used, which gets you started on a single machine, but it is very
likely you want to change these to match your use-case/setup.
{% infobubble(mode="info", title="⚠ Please note") %}
Please have a look at the [configuration
page](@/docs/configure/_index.md) page, before making docspell
publicly available. By default, everyone can create an account. This
is great for trying out and using it in an internal network. But when
opened up to the outside, it is recommended to lock this down.
{% end %}

View File

@ -1,93 +0,0 @@
+++
title = "Running"
weight = 30
+++
# Running
Run the start script (in the corresponding `bin/` directory when using
the zip files):
```
$ ./docspell-restserver*/bin/docspell-restserver
$ ./docspell-joex*/bin/docspell-joex
```
This will startup both components using the default configuration. The
configuration should be adopted to your needs. For example, the
database connection is configured to use a H2 database in the `/tmp`
directory. Please refer to the [configuration
page](@/docs/configure/_index.md) for how to create a custom config
file. Once you have your config file, simply pass it as argument to
the command:
```
$ ./docspell-restserver*/bin/docspell-restserver /path/to/server-config.conf
$ ./docspell-joex*/bin/docspell-joex /path/to/joex-config.conf
```
After starting the rest server, you can reach the web application at
path `/app`, so using default values it would be
`http://localhost:7880/app`. There also is a redirect from `/` to
`/app`.
You should be able to create a new account and sign in. Check the
[configuration page](@/docs/configure/_index.md) to further customize
docspell.
{% infobubble(mode="info", title="⚠ Please note") %}
Please have a look at the [configuration
page](@/docs/configure/_index.md) page, before making docspell
available in the internet. By default, everyone can create an account.
This is great for trying out and using it in an internal network. But
when opened up to the outside, it is recommended to lock this down.
{% end %}
## Memory
Using the options below you can define how much memory the JVM process
is able to use. This might be necessary to adopt depending on the
usage scenario and configured text analysis features.
Please have a look at the corresponding [configuration
section](@/docs/configure/_index.md#memory-usage).
## Options
The start scripts support some options to configure the JVM. One often
used setting is the maximum heap size of the JVM. By default, java
determines it based on properties of the current machine. You can
specify it by given java startup options to the command:
```
$ ./docspell-restserver*/bin/docspell-restserver -J-Xmx1G -- /path/to/server-config.conf
```
This would limit the maximum heap to 1GB. The double slash separates
internal options and the arguments to the program. Another frequently
used option is to change the default temp directory. Usually it is
`/tmp`, but it may be desired to have a dedicated temp directory,
which can be configured:
```
$ ./docspell-restserver*/bin/docspell-restserver -J-Xmx1G -Djava.io.tmpdir=/path/to/othertemp -- /path/to/server-config.conf
```
The command:
```
$ ./docspell-restserver*/bin/docspell-restserver -h
```
gives an overview of supported options.
It is recommended to run joex with the G1GC enabled. If you use java8,
you need to add an option to use G1GC (`-XX:+UseG1GC`), for java11
this is not necessary (but doesn't hurt either). This could look like
this:
```
./docspell-joex-{{version()}}/bin/docspell-joex -J-Xmx1596M -J-XX:+UseG1GC -- /path/to/joex.conf
```

View File

@ -1,35 +0,0 @@
+++
title = "Upgrading"
weight = 35
+++
Upgrading docspell requires to download the newer binaries and
re-installing them depending on your setup.
The database is migrated on application start automatically.
Since [downgrading](@/docs/install/downgrading.md) is not supported,
it is recommended to backup your database before upgrading. Should
something not work as expected, restore the database backup and go
back to the previous version.
# Docker-Compose
When using the provided `docker-compose` setup, you only need to pull
the new images. The latest release is always tagged with `-LATEST`.
``` bash
$ docker-compose down
$ docker-compose pull
$ docker-compose up --force-recreate --build -d
```
# Zip / Deb Files
When using the zip or deb files, either install the new deb files via
your package manager or download and unpack the new zip files.
# Nix
When using the provided nix setup, the `currentPkg` always points to
the latest release. Thus it is enough to run `nix-build`.