From 4e2236198596b8d949738c4c4b161312bfbe40a5 Mon Sep 17 00:00:00 2001 From: Eike Kettner Date: Mon, 25 May 2020 09:52:19 +0200 Subject: [PATCH] Documentation updates --- modules/microsite/docs/doc/install.md | 2 +- modules/microsite/docs/doc/tools.md | 187 +----------------- .../microsite/docs/doc/tools/browserext.md | 84 ++++++++ .../microsite/docs/doc/tools/consumedir.md | 64 ++++++ modules/microsite/docs/doc/tools/ds.md | 47 +++++ modules/microsite/docs/features.md | 8 +- modules/microsite/docs/getit.md | 23 ++- .../main/resources/microsite/data/menu.yml | 14 +- 8 files changed, 236 insertions(+), 193 deletions(-) create mode 100644 modules/microsite/docs/doc/tools/browserext.md create mode 100644 modules/microsite/docs/doc/tools/consumedir.md create mode 100644 modules/microsite/docs/doc/tools/ds.md diff --git a/modules/microsite/docs/doc/install.md b/modules/microsite/docs/doc/install.md index 9f64126f..ae84ae5b 100644 --- a/modules/microsite/docs/doc/install.md +++ b/modules/microsite/docs/doc/install.md @@ -57,7 +57,7 @@ build using JDK 8. But a later version of Java should work as well. The next tools are only required on machines running the *Joex* component. -### External Tools for Joex +### External Programs for Joex - [Ghostscript](http://pages.cs.wisc.edu/~ghost/) (the `gs` command) is used to extract/convert PDF files into images that are then fed diff --git a/modules/microsite/docs/doc/tools.md b/modules/microsite/docs/doc/tools.md index f364d031..6a05262a 100644 --- a/modules/microsite/docs/doc/tools.md +++ b/modules/microsite/docs/doc/tools.md @@ -9,183 +9,10 @@ permalink: doc/tools The `tools/` folder contains some scripts and other resources intented for integrating docspell. -## consumedir - -The `consumerdir.sh` is a bash script that works in two modes: - -- Go through all files in given directories (non recursively) and sent - each to docspell. -- Watch one or more directories for new files and upload them to - docspell. - -It can watch or go through one or more directories. Files can be -uploaded to multiple urls. - -Run the script with the `-h` option, to see a short help text. The -help text will also show the values for any given option. - -The script requires `curl` for uploading. It requires the -`inotifywait` command if directories should be watched for new -files. If the `-m` option is used, the script will skip duplicate -files. For this the `sha256sum` command is required. - -Example for watching two directories: - -``` bash -./tools/consumedir.sh --path ~/Downloads --path ~/pdfs -m -dv http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ -``` - -The script by default watches the given directories. If the `-o` -option is used, it will instead go through these directories and -upload all pdf files in there. - -Example for uploading all immediatly (the same as above only with `-o` -added): - -``` bash -./tools/consumedir.sh -o --path ~/Downloads --path ~/pdfs/ -m -dv http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ -``` - - -### Systemd - -The script can be used with systemd to run as a service. This is an -example unit file: - -``` -[Unit] -After=networking.target -Description=Docspell Consumedir - -[Service] -Environment="PATH=/set/a/path" - -ExecStart=/bin/su -s /bin/bash someuser -c "consumedir.sh --path '/a/path/' -m 'http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ'" -``` - -This unit file is just an example, it needs some fiddling. It assumes -an existing user `someuser` that is used to run this service. The url -`http://localhost:7880/api/v1/open/upload/...` is an anonymous upload -url as described [here](./uploading#anonymous-upload). - - -## ds.sh - -A bash script to quickly upload files from the command line. It reads -a configuration file containing the URLs to upload to. Then each file -given to the script will be uploaded to al URLs in the config. - -The config file is expected in -`$XDG_CONFIG_HOME/docspell/ds.conf`. `$XDG_CONFIG_HOME` defaults to -`~/.config`. - -The config file contains lines with key-value pairs, separated by a -`=` sign. Lines starting with `#` are ignored. Example: - -``` -# Config file -url.1 = http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ -url.2 = http://localhost:7880/api/v1/open/upload/item/6DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ -``` - -The key must start with `url`. The urls should be [anonymous upload -urls](./uploading#anonymous-upload). - - -### Usage - -- The `-c` option allows to specifiy a different config file. -- The `-h` option shows a help overview. -- The `-d` option deletes files after upload was successful -- The `-e` option can be used to check for file existence in docspell. - Instead of uploading, the script only checks whether the file is in - docspell or not. - -The script takes a list of files as arguments. - - -Example: - -``` bash -./ds.sh ~/Downloads/*.pdf -``` - - -## Webextension for Docspell - -Idea: Inside the browser click on a PDF and send it to docspell. It is -downloaded in the context of your current page. Then handed to an -application that pushes it to docspell. There is a browser add-on -implementing this in `tools/webextension`. This add-on only works with -firefox. - -### Install - -This is a bit complicated, since you need to install external tools -and the web extension. Both work together. - -#### Install `ds.sh` - -First copy the `ds.sh` tool somewhere in your `PATH`, maybe -`/usr/local/bin` as described above. - - -#### Install the native part - -Then install the "native" part of the web extension: - -Copy or symlink the `native.py` script into some known location. For -example: - -``` bash -ln -s ~/docspell-checkout/tools/webextension/native/native.py /usr/local/share/docspell/native.py -``` - -Then copy the `app_manifest.json` to -`$HOME/.mozilla/native-messaging-hosts/docspell.json`. For example: - -``` bash -cp ~/docspell-checkout/tools/webextension/native/app_manifest.json ~/.mozilla/native-messaging-hosts/docspell.json -``` - -See -[here](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_manifests#Manifest_location) -for details. - -And you might want to modify this json file, so the path to the -`native.py` script is correct (it must be absolute). - -If the `ds.sh` script is in your `$PATH`, then this should -work. Otherwise, edit the `native.py` script and change the path to -the tool. Or create a file `$HOME/.config/docspell/ds.cmd` whose -content is the path to the `ds.sh` script. - - -#### Install the extension - -An extension file can be build using the `make-xpi.sh` script. But -installing it in "standard" firefox won't work, because [Mozilla -requires extensions to be signed by -them](https://wiki.mozilla.org/Add-ons/Extension_Signing). This means -creating an account and going through some process…. So here are two -alternatives: - -1. Open firefox and type `about:debugging` in the addressbar. Then - click on *'Load Temporary Add-on...'* and select the - `manifest.json` file. The extension is now installed. The downside - is, that the extension will be removed once firefox is closed. -2. Use Firefox ESR, which allows to install Add-ons not signed by - Mozilla. But it has to be configured: Open firefox and type - `about:config` in the address bar. Search for key - `xpinstall.signatures.required` and set it to `false`. This is - described on the last paragraph on [this - page](https://support.mozilla.org/en-US/kb/add-on-signing-in-firefox). - -When you right click on a file link, there should be a context menu -entry *'Docspell Upload Helper'*. The add-on will download this file -using the browser and then send the file path to the `native.py` -script. This script will in turn call `ds.sh` which finally uploads it -to your configured URLs. - -Open the Add-ons page (`Ctrl`+`Shift`+`A`), the new add-on should be -there. +- [ds.sh](ds) A script to quickly upload files from the command + line. +- [Consume Directory](consumedir) A script to watch a directory + for new files and upload them to docspell. +- [Browser Extension](browserext) An extension for firefox to + upload files from your browser via *right-click -> upload to + docspell*. diff --git a/modules/microsite/docs/doc/tools/browserext.md b/modules/microsite/docs/doc/tools/browserext.md new file mode 100644 index 00000000..e213c081 --- /dev/null +++ b/modules/microsite/docs/doc/tools/browserext.md @@ -0,0 +1,84 @@ +--- +layout: docs +title: Browser Extension (Firefox) +permalink: doc/tools/browserext +--- + +# {{ page.title }} + +The idea is to click on a file in firefox and send it to docspell. It +is downloaded in the context of your current page. Then handed to an +application that pushes it to docspell. There is a browser add-on +implementing this in `tools/webextension`. This add-on only works with +firefox. + +### Install + +This is a bit complicated, since you need to install external tools +and the web extension. Both work together. + +#### Install `ds.sh` + +First copy the `ds.sh` tool somewhere in your `PATH`, maybe +`/usr/local/bin` as described above. + + +#### Install the native part + +Then install the "native" part of the web extension: + +Copy or symlink the `native.py` script into some known location. For +example: + +``` bash +ln -s ~/docspell-checkout/tools/webextension/native/native.py /usr/local/share/docspell/native.py +``` + +Then copy the `app_manifest.json` to +`$HOME/.mozilla/native-messaging-hosts/docspell.json`. For example: + +``` bash +cp ~/docspell-checkout/tools/webextension/native/app_manifest.json ~/.mozilla/native-messaging-hosts/docspell.json +``` + +See +[here](https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_manifests#Manifest_location) +for details. + +And you might want to modify this json file, so the path to the +`native.py` script is correct (it must be absolute). + +If the `ds.sh` script is in your `$PATH`, then this should +work. Otherwise, edit the `native.py` script and change the path to +the tool. Or create a file `$HOME/.config/docspell/ds.cmd` whose +content is the path to the `ds.sh` script. + + +#### Install the extension + +An extension file can be build using the `make-xpi.sh` script. But +installing it in "standard" firefox won't work, because [Mozilla +requires extensions to be signed by +them](https://wiki.mozilla.org/Add-ons/Extension_Signing). This means +creating an account and going through some process…. So here are two +alternatives: + +1. Open firefox and type `about:debugging` in the addressbar. Then + click on *'Load Temporary Add-on...'* and select the + `manifest.json` file. The extension is now installed. The downside + is, that the extension will be removed once firefox is closed. +2. Use Firefox ESR, which allows to install Add-ons not signed by + Mozilla. But it has to be configured: Open firefox and type + `about:config` in the address bar. Search for key + `xpinstall.signatures.required` and set it to `false`. This is + described on the last paragraph on [this + page](https://support.mozilla.org/en-US/kb/add-on-signing-in-firefox). + +When you right click on a file link, there should be a context menu +entry *'Docspell Upload Helper'*. The add-on will download this file +using the browser and then send the file path to the `native.py` +script. This script will in turn call `ds.sh` which finally uploads it +to your configured URLs. + +Open the Add-ons page (`Ctrl`+`Shift`+`A`), the new add-on should be +there. diff --git a/modules/microsite/docs/doc/tools/consumedir.md b/modules/microsite/docs/doc/tools/consumedir.md new file mode 100644 index 00000000..d686a7da --- /dev/null +++ b/modules/microsite/docs/doc/tools/consumedir.md @@ -0,0 +1,64 @@ +--- +layout: docs +title: Consume Directory +permalink: doc/tools/consumedir +--- + +# {{ page.title }} + +The `consumerdir.sh` is a bash script that works in two modes: + +- Go through all files in given directories (non recursively) and sent + each to docspell. +- Watch one or more directories for new files and upload them to + docspell. + +It can watch or go through one or more directories. Files can be +uploaded to multiple urls. + +Run the script with the `-h` option, to see a short help text. The +help text will also show the values for any given option. + +The script requires `curl` for uploading. It requires the +`inotifywait` command if directories should be watched for new +files. If the `-m` option is used, the script will skip duplicate +files. For this the `sha256sum` command is required. + +Example for watching two directories: + +``` bash +./tools/consumedir.sh --path ~/Downloads --path ~/pdfs -m -dv http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ +``` + +The script by default watches the given directories. If the `-o` +option is used, it will instead go through these directories and +upload all files in there. + +Example for uploading all immediatly (the same as above only with `-o` +added): + +``` bash +./tools/consumedir.sh -o --path ~/Downloads --path ~/pdfs/ -m -dv http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ +``` + + +## Systemd + +The script can be used with systemd to run as a service. This is an +example unit file: + +``` +[Unit] +After=networking.target +Description=Docspell Consumedir + +[Service] +Environment="PATH=/set/a/path" + +ExecStart=/bin/su -s /bin/bash someuser -c "consumedir.sh --path '/a/path/' -m 'http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ'" +``` + +This unit file is just an example, it needs some fiddling. It assumes +an existing user `someuser` that is used to run this service. The url +`http://localhost:7880/api/v1/open/upload/...` is an anonymous upload +url as described [here](../uploading#anonymous-upload). diff --git a/modules/microsite/docs/doc/tools/ds.md b/modules/microsite/docs/doc/tools/ds.md new file mode 100644 index 00000000..0cefc1ed --- /dev/null +++ b/modules/microsite/docs/doc/tools/ds.md @@ -0,0 +1,47 @@ +--- +layout: docs +title: Upload CLI +permalink: doc/tools/ds +--- + +# {{ page.title }} + +A bash script is provided to quickly upload files from the command +line. It reads a configuration file containing the URLs to upload to. +Then each file given to the script will be uploaded to al URLs in the +config. + +The config file is expected in +`$XDG_CONFIG_HOME/docspell/ds.conf`. `$XDG_CONFIG_HOME` defaults to +`~/.config`. + +The config file contains lines with key-value pairs, separated by a +`=` sign. Lines starting with `#` are ignored. Example: + +``` +# Config file +url.1 = http://localhost:7880/api/v1/open/upload/item/5DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ +url.2 = http://localhost:7880/api/v1/open/upload/item/6DxhjkvWf9S-CkWqF3Kr892-WgoCspFWDo7-XBykwCyAUxQ +``` + +The key must start with `url`. The urls should be [anonymous upload +urls](./uploading#anonymous-upload). + + +### Usage + +- The `-c` option allows to specifiy a different config file. +- The `-h` option shows a help overview. +- The `-d` option deletes files after upload was successful +- The `-e` option can be used to check for file existence in docspell. + Instead of uploading, the script only checks whether the file is in + docspell or not. + +The script takes a list of files as arguments. + + +Example: + +``` bash +./ds.sh ~/Downloads/*.pdf +``` diff --git a/modules/microsite/docs/features.md b/modules/microsite/docs/features.md index 62fc76be..6402f144 100644 --- a/modules/microsite/docs/features.md +++ b/modules/microsite/docs/features.md @@ -42,9 +42,11 @@ permalink: features - [eml](https://en.wikipedia.org/wiki/Email#Filename_extensions) (e-mail files in plain text MIME) - [Tooling](doc/tools): - - Watch a folder: watch folders for changes and send files to docspell - - Firefox plugin: right click on a link and send the file to docspell - - Simple CLI for uploading files + - [Watch a folder](doc/tools/consumedir): watch folders for changes + and send files to docspell + - [Simple CLI for uploading files](doc/tools/ds) + - [Firefox plugin](doc/tools/browserext): right click on a link and + send the file to docspell - License: GPLv3 diff --git a/modules/microsite/docs/getit.md b/modules/microsite/docs/getit.md index afc182fc..1e08bec1 100644 --- a/modules/microsite/docs/getit.md +++ b/modules/microsite/docs/getit.md @@ -11,8 +11,20 @@ Page](https://github.com/eikek/docspell/releases). There are `deb` packages and generic zip files. Alternatively, docspell can be installed via [nix](doc/nix) or docker (see below). +There are the following quick ways to get docspell to run on your +machine: -## Prerequisite +- [Download, Unpack, Run](#without-docker) +- [With Docker](#with-docker) +- [NixOs Module](doc/nix#docspell-as-a-service-on-nixos) + +Check the [early demo video](demo) to see the basic idea. Refer to the +[documentation](doc) for more information on how to use docspell. + + +## Without Docker + +### Prerequisite Install Java (use your package manager or look [here](https://adoptopenjdk.net/)). @@ -31,7 +43,7 @@ PDF conversion requires the following tools: - [wkhtmltopdf](https://wkhtmltopdf.org/) -## Using zip files +### Using zip files You need to download the two files: @@ -57,11 +69,8 @@ You need to download the two files: 4. Point your browser to: 5. Register a new account, sign in and try it. -Check the [early demo video](demo) to see the basic idea. Refer to the -[documentation](doc) for more information on how to use docspell. - -## Docker +## With Docker There is a [docker-compose](https://docs.docker.com/compose/) setup available in the `/docker` folder. @@ -81,4 +90,4 @@ available in the `/docker` folder. 4. Goto , signup and login The directory contains a file `docspell.conf` that you can -[modify](configure) as needed. +[modify](doc/configure) as needed. diff --git a/modules/microsite/src/main/resources/microsite/data/menu.yml b/modules/microsite/src/main/resources/microsite/data/menu.yml index 55cfe096..56601cf8 100644 --- a/modules/microsite/src/main/resources/microsite/data/menu.yml +++ b/modules/microsite/src/main/resources/microsite/data/menu.yml @@ -54,8 +54,18 @@ options: - title: Joex url: doc/joex - - title: Tools - url: doc/tools + - title: Tools + url: doc/tools + + nested_options: + - title: Upload CLI + url: doc/tools/ds + + - title: Consume Directory + url: doc/tools/consumedir + + - title: Browser Extension (Firefox) + url: doc/tools/browserext - title: Development url: dev