Update documentation and changelog

Changelog.md

@@ -1,5 +1,55 @@
 # Changelog
 
+## v0.8.0
+
+*Soon*
+
+- Adds the [full-text
+  search](https://docspell.org/doc/finding#full-text-search) feature
+  (see #69). It requires a separate
+  [Solr](https://lucene.apache.org/solr) instance. Items can be
+  searched by documents contents and item/file names. It is possible
+  to use full-text search to further confine the results via the
+  search menu.
+- Fixes column types for item date and due-date for MariaDB (see #158)
+  and adds an upper limit for due-dates (which is configurable).
+- Fixes a bug when cancelling jobs where stuck jobs were only removed
+  from the queue, but their cancel routine was not called.
+- Changes cancelling process-item jobs, the item will still be created
+  and not removed.
+- Fixes a bug where items could not be deleted if there were sent
+  mails attached.
+- Fixes the openapi spec for the joex component.
+- Changes to the `consumedir.sh` script:
+  - Allow to recursively watch or traverse directories
+  - Allow it to work with the integration endpoint. This allows using
+    `consumedir.sh` for all collectives.
+- The docker setup now starts a solr container automatically and
+  configures the consumedir container to use the integration endpoint.
+  It is still necessary to define an environment variable.
+
+### Configuration Changes
+
+- Set new default for `docspell.server.max-item-page-size` to `200`.
+- New `full-text-search` section for restserver and joex.
+
+### REST Api Changes
+
+- Add `/open/fts/reIndexAll/{key}` to re-index the full-text search
+  index. The `key` must be defined in the config file, so only admins
+  can execute this.
+- Add `/sec/fts/reIndex` to allow a collective to re-index their data
+  only.
+- Add `/open/integration/checkfile/{id}/{checksum}` to check whether a
+  file is in docspell via the integration endpoint.
+- Add `/sec/item/searchIndex` to allow searching the full-text index
+  only. This route returns the results as ordered by SOLR and not
+  ordered by date.
+- The `ItemSearch` input data is extended to support the new full-text
+  search field.
+- The `ItemLight` result structure now can contain "highlighting"
+  information that is provided by the full-text search index.
+
 ## v0.7.0
 
 *June 17, 2020*

modules/microsite/docs/doc/tools/consumedir.md

@@ -116,21 +116,26 @@ url as described [here](../uploading#anonymous-upload).
 
 ## Docker
 
-The provided docker image runs this script to watch a directory for
-new files. If a new file is detected, it is pushed to docspell.
+The provided docker image runs this script to watch a single
+directory, `./docs` in current directory, for new files. If a new file
+is detected, it is pushed to docspell.
 
-For this to work, the container must know about a valid upload url.
-Therefore, you must first signup and create such an upload url, as
-described [here](doc/uploading#anonymous-upload). Get only the id
-(something like `AvR6sA8GKFm-hgYDgZfwzXa-Tqnu8yqyz6X-KzuefvEvrRf`) and
-define an environment variable `SOURCE_ID` with that value before
-running `docker-compose up` a second time.
+This utilizes the [integration
+endpoint](../uploading#integration-endpoint), which is enabled in the
+config file, to allow uploading documents for all collectives. A
+subfolder must be created for each registered collective. The docker
+containers are configured to use http-header protection for the
+integration endpoint. This requires you to provide a secret, that is
+shared between the rest-server and the `consumedir.sh` script. This
+can be done by defining an environment variable which gets picked up
+by the containers defined in `docker-compose.yml`:
 
 ```
-export SOURCE_ID="AvR6sA8GKFm-hgYDgZfwzXa-Tqnu8yqyz6X-KzuefvEvrRf"
+export DOCSPELL_HEADER_VALUE="my-secret"
 docker-compose up
 ```
 
-Now you can create a folder `./docs` and place all files in there that
-you want to import. Once dropped in this folder the `consumedir`
-container will push it to docspell.
+
+Now you can create a folder `./docs/<collective-name>` and place all
+files in there that you want to import. Once dropped in this folder
+the `consumedir` container will push it to docspell.

modules/microsite/docs/getit.md

@@ -69,6 +69,9 @@ You need to download the two files:
 4. Point your browser to: <http://localhost:7880/app>
 5. Register a new account, sign in and try it.
 
+Note, that this setup doesn't include watching a directory. You can
+use the [`consumedir.sh`](doc/tools/consumedir) tool for this or use
+the docker variant below.
 
 ## With Docker
 
@@ -84,19 +87,19 @@ available in the `/docker` folder.
    $ cd docspell/docker
    ```
 3. Run `docker-compose up`:
+
    ```bash
+   $ export DOCSPELL_HEADER_VALUE="my-secret-123"
    $ docker-compose up
    ```
-4. Goto <http://localhost:7880>, signup and login
+
+   The environment variable defines a secret that is shared between
+   some containers. You can define whatever you like. Please see the
+   [`consumedir.sh`](doc/tools/consumedir#docker) docs for additional
+   info.
+4. Goto <http://localhost:7880>, signup and login. Create a folder
+   `./docs/<collective-name>` and place files in there for importing
+   them.
 
 The directory contains a file `docspell.conf` that you can
 [modify](doc/configure) as needed.
-
-
-### Watching files in a directory
-
-This setup starts a container running the
-[`consumedir.sh`](doc/tools/consumedir) script. It is configured to
-watch one directory and upload files arriving in there to docspell.
-Please see the [`consumedir.sh`](doc/tools/consumedir#docker) docs for
-additional steps.