19 KiB
+++ title = "CLI" description = "A command line interface to." weight = 5 +++
Docspell CLI
Introduction
The docspell client, short dsc, is a tool to use docspell through the command line. It is also aims to be useful for your own scripts and programs.
It is supposed to replace most of the shell scripts from the tools/
directory.
It is a work in progress; eventually most of the api will be covered.
Usage
Download the binary for your architecture from the release
page and rename it
to dsc. Then run dsc help to see an overview of all commands. The
help of each command is available via dsc help [command] or dsc [command] --help.
There are docker images at dockerhub, but it's usually easier to just download the binary. They should work on most systems without additional setups.
Below are some quick infos to get started, please see the project page for more info.
Configuration
A configuration file can be used to have some predefined settings, for example the docspell url, the admin secret etc. They can be overriden by specifying them as options.
The config looks like this:
docspell_url = "http://localhost:7880"
default_format = "Tabular"
admin_secret = "admin123"
default_account = "demo"
pdf_viewer = ["zathura", "{}"]
#pass_entry = "my/entry"
For linuxes, the default location is ~/.config/dsc/config.toml. You
can give a config file explicitly via an option or the environment
variable DSC_CONFIG.
If you use the pass password manager, you can add your password entry to the config file as well.
Output format
The "output format" defines how the information is printed on screen.
The default output format is Tabular which prints a simple table.
This table can also be formatted as CSV using csv as output format.
These two modes are intended for humans and they may not present all
information available.
Alternatively, there is json and lisp as output format. These are
intended for machine consumption. They always contain all information.
If you look for some detail, use for example json to get all data in
a structured form. On the shell you can use the awesome tool
jq to get exactly what you want.
Login
Many tasks require to be logged in. This can be done via the login
subcommand. You can specify account and password or fallback to the
values in the config file.
Once logged in, the session token will be saved to the filesystem
(next to the config file) and is used for subsequent commands. It is
renewed if expiry is near. If you don't issue any commands for a while
you need to login again.
Demo
Use Cases / Examples
These are some examples. Each command has a good help explaining all
the options. Run dsc [subcommand] --help to see it.
Uploads files
The upload subcommand can upload files to docspell. This is the
replacement for the ds.sh shell script.
You can specify a list of files that are all being uploaded. This command doesn't require to be logged in, it can also upload via a source id or via the integration endpoint.
A source id can be given in the config file, then there are no additional options required. The simplest form is this:
❯ dsc upload *.pdf
File already in Docspell: article-velo.pdf
Adding to request: test-ocr.pdf
Sending request …
┌─────────┬──────────────────┐
│ success │ message │
├─────────┼──────────────────┤
│ true │ Files submitted. │
└─────────┴──────────────────┘
By default, duplicate files are detected and not uploaded. This
uploads all files in one single request. By default, each file results
in one item. Using --single-item all files can be put into one item.
It is possible to specify certain metadata, like tags or a folder, that is then attached to the resulting item.
Upload by traversing a directory
The above use case was about uploading files. Using the upload
subcommand with the --traverse option, you can traverse directories
and upload all files in them. In this mode, each file will be uploaded
in a separate request, so the --single-item option cannot be used.
There are options to exclude/include files based on a glob pattern.
❯ dsc upload --traverse .
File already in Docspell: article-velo.pdf
File already in Docspell: demo/dirc/scan.21-03-12.15-50-54.pdf
File already in Docspell: demo/dirc/test-stamp.pdf
File already in Docspell: demo/letter-de.pdf
Uploading eike/keywords.pdf
File already in Docspell: eike/large-file.pdf
Uploading eike/letter-en.pdf
File already in Docspell: test-ocr.pdf
┌─────────┬────────────┐
│ success │ message │
├─────────┼────────────┤
│ true │ Uploaded 2 │
└─────────┴────────────┘
The --poll option allows to periodically traverse and upload
directories.
Watch a directory
The watch subcommand can be used to watch one or more directories
and upload files when they arrive. It uses the upload command under
the hood and therefore most options are also available here. You can
upload via a source url, the integration endpoint or a valid session
(requires to login).
It detects file creations and skips a rename within a watched folder.
The flag -r or --recursive is required to recursively watch a
directory.
❯ dsc watch -r .
Watching directory (Recursive): .
Press Ctrl-C to quit.
------------------------------------------------------------------------------
Got: /home/eike/workspace/projects/dsc/local/files/./demo/letter-de.pdf
Adding to request: /home/eike/workspace/projects/dsc/local/files/./demo/letter-de.pdf
Sending request …
Server: Files submitted.
The --matches option allows to define a pattern for files to include.
If watching a directory is not possible due to system constraints
(e.g. when using NFS or SAMBA shares), a less efficient option is to
use the upload subcommand with --poll option which periodically
traverses a directory.
When using the integration endpoint, it requires to specify -i and
potentially a secret if the endpoint is protected with a secret.
Download files
The download command allows to download files that match a given
query. It is possible to download them all flat into some directory or
directly into a zip file. For example, download all files that are
tagged with todo into a zip file:
❯ dsc download --zip 'tag:todo'
Zipping 2 attachments into docspell-files.zip
Downloading DOC-20191223-155707.jpg.pdf …
Downloading DOC-20200803-174448.jpg.pdf …
It downloads the converted PDF files by default, which can be changed via some options.
❯ dsc download --zip --original 'tag:todo'
Zipping original files of 2 attachments into docspell-files.zip
Downloading DOC-20191223-155707.jpg …
Downloading DOC-20200803-174448.jpg …
Export data
The export command allows to download all items with metadata and
their files. It downloads all items by default, but a query is also
supported.
In contrast to the download command, this is intended for getting
everything out of docspell in some independent format. Files are
downloaded (only original files) and the items metadata is also
stored. So you don't loose the tags and correspondents that are
carefully maintained with each item.
It expects one directory where it will create a specfific directory structure as follows:
exports/
├── by_date
│ ├── 2019-07
│ ├── 2020-08
| ├── BV2po65mAFU-…-bqUiwjz8f2W -> ../../items/BV/BV2po65mAFU-…-bqUiwjz8f2W
| └── FTUnhZ3AE1H-…-RQ9KhtRi486 -> ../../items/FT/FTUnhZ3AE1H-…-RQ9KhtRi486
│ ├── …
│ └── 2021-07
├── by_folder
│ ├── Finance
| │ ├── 455h3cQNdna-…-t6dF7NjAuDm -> ../../items/45/455h3cQNdna-…-t6dF7NjAuDm
| │ ├── 5yQ95tQ4khY-…-S9KrxcbRkZR -> ../../items/5y/5yQ95tQ4khY-…-S9KrxcbRkZR
| │ ├── Bank
| │ │ ├── 7xoiE4XdwgD-…-Eb2S3BCSd38 -> ../../items/7x/7xoiE4XdwgD-…-Eb2S3BCSd38
| │ │ ├── 93npVoA73Cx-…-BnxYNsf4Qvi -> ../../items/93/93npVoA73Cx-…-BnxYNsf4Qvi
├── by_tag
│ ├── Contract
│ ├── Important
│ ├── Invoice
| │ ├── 455h3cQNdna-…-t6dF7NjAuDm -> ../../items/45/455h3cQNdna-…-t6dF7NjAuDm
| │ ├── 5yQ95tQ4khY-…-S9KrxcbRkZR -> ../../items/5y/5yQ95tQ4khY-…-S9KrxcbRkZR
| │ ├── 7xoiE4XdwgD-…-Eb2S3BCSd38 -> ../../items/7x/7xoiE4XdwgD-…-Eb2S3BCSd38
| │ └── 93npVoA73Cx-…-BnxYNsf4Qvi -> ../../items/93/93npVoA73Cx-…-BnxYNsf4Qvi
│ ├── …
│ └── Todo
└── items
├── 45
| └── 455h3cQNdna-w8oTEw9wtpE-G7bCJbVpZPw-t6dF7NjAuDm
| ├── files
| │ └── DOC-20200803-174448.jpg
| └── metadata.json
├── …
└── Hb
All your items are stored below the items directory. It contains
subdirectories that are created from the first two characters of the
item id. Inside this folder, a folder with the complete item id is
created and there all the data to the item is stored: the metadata in
metadata.json and all files below files/.
The options --date-links, --folder-links, and --tag-links create
the other two folders: by_tag, by_folder and by_date. In there
you'll find symlinks into the items folder that are organized by
some metadata, namely tags, folder, and the item's date.
With the --link-naming option you can control the filename used for
the links created in the by_* folders. The default is the items' id,
which is not very self-speaking and thus might not fit your needs. Using
--link-naming name you can configure it to use the item's name instead.
If you use path separators in your folder names, such as the "/" in
Finance/Institute, you can configure export to unwrap your folder names
into recursive paths (as shown in the example output above) using the
--folder-delimiter "/" option. Otherwise, the name would be sanitized
resulting in Finance-Institute.
Example run (producing output shown above):
❯ dsc export --all --date-links --folder-links --folder-delimiter "/" --tag-links --target exports
Exported item: test3.zip
Exported item: README.md
Exported item: LICENSE.txt
Exported item: TestRMarkdown.pdf
Exported item: DOC-20191223-155729.jpg
Exported item: DOC-20191223-155707.jpg
Exported item: DOC-20200808-154204.jpg
Exported item: DOC-20200807-115654.jpg
Exported item: DOC-20200803-174448.jpg
Exported item: DOC-20200803-174448.jpg
Exported item: DOC-20200803-174448.jpg
Exported item: DOC-20200803-174448.jpg
Exported item: DOC-20200804-132305.jpg
Exported item: DOC-20191223-155707.jpg
Exported item: keyweb.eml
Exported 15 items.
With output format json or lisp each item is printed instead in
full detail.
Admin commands
These are a set of commands that simply call a route at the server to submit a maintenance task or to reset the password of some user. These commands require the admin secret either in the config file or as an argument.
Reset user password
❯ dsc admin reset-password --account demo
┌─────────┬──────────────┬──────────────────┐
│ success │ new password │ message │
├─────────┼──────────────┼──────────────────┤
│ true │ 2q2UeCVvMYg │ Password updated │
└─────────┴──────────────┴──────────────────┘
Recreate fulltext index
❯ dsc admin --admin-secret admin123 recreate-index
┌─────────┬─────────────────────────────────────┐
│ success │ message │
├─────────┼─────────────────────────────────────┤
│ true │ Full-text index will be re-created. │
└─────────┴─────────────────────────────────────┘
Convert all files to PDF
❯ dsc admin --admin-secret admin123 convert-all-pdf
┌─────────┬─────────────────────────────────┐
│ success │ message │
├─────────┼─────────────────────────────────┤
│ true │ Convert all PDFs task submitted │
└─────────┴─────────────────────────────────┘
This may be necessary if you disabled pdf conversion before and are enabling it now.
Regenerate preview images
❯ dsc admin --admin-secret admin123 convert-all-pdf
┌─────────┬───────────────────────────────────────┐
│ success │ message │
├─────────┼───────────────────────────────────────┤
│ true │ Generate all previews task submitted. │
└─────────┴───────────────────────────────────────┘
This submits tasks to (re)generate preview images of all files. This
is necessary if you changed the preview.dpi setting in joex'
config.
Search for items
The search command takes a query and
prints the results.
❯ dsc search 'corr:*'
┌──────────┬────────────────────────────┬───────────┬────────────┬─────┬───────────────────────────┬───────────────┬────────┬─────────────┬────────────┬───────┐
│ id │ name │ state │ date │ due │ correspondent │ concerning │ folder │ tags │ fields │ files │
├──────────┼────────────────────────────┼───────────┼────────────┼─────┼───────────────────────────┼───────────────┼────────┼─────────────┼────────────┼───────┤
│ HVK7JuCF │ test-ocr.pdf │ created │ 2021-07-18 │ │ Pancake Company │ │ │ Certificate │ │ 1 │
│ 3odNawKE │ letter-en.pdf │ confirmed │ 2021-07-18 │ │ Pancake Company │ │ │ invoice │ │ 1 │
│ 3MA5NdhS │ large-file.pdf │ confirmed │ 2021-07-18 │ │ Axa │ │ │ Certificate │ │ 1 │
│ HDumXkRm │ keywords.pdf │ confirmed │ 2021-07-18 │ │ Pancake Company │ │ │ invoice │ │ 1 │
│ 733gM656 │ test-stamp.pdf │ created │ 2021-07-18 │ │ Pancake Company │ │ │ Contract │ │ 1 │
│ 8LiciitB │ scan.21-03-12.15-50-54.pdf │ confirmed │ 2021-07-18 │ │ Supermarket │ │ │ Receipt │ CHF 89.44 │ 1 │
│ 8nFt2z7T │ article-velo.pdf │ confirmed │ 2021-07-18 │ │ Supermarket/Rudolf Müller │ Rudolf Müller │ │ invoice │ CHF 123.11 │ 1 │
│ kfugGdXU │ letter-de.pdf │ created │ 2021-07-18 │ │ Axa │ Rudolf Müller │ │ invoice │ │ 1 │
└──────────┴────────────────────────────┴───────────┴────────────┴─────┴───────────────────────────┴───────────────┴────────┴─────────────┴────────────┴───────┘
The same can be formatted as json and, for example, only print the ids:
❯ dsc -f json search 'corr:*' | jq '.groups[].items[].id'
"HVK7JuCFt4W-qxkcwq1cWCV-dvpGo4DpZzU-Q16Xoujojas"
"3odNawKE1Ek-YJrWfPzekAq-47cjt14sexd-GK35JAEAanx"
"3MA5NdhSrbx-3JkjEpqHiyU-XyVNb15tioh-SUVjMLi1aoV"
"HDumXkRmDea-dNryjtRjk3V-ysdJmJNQGQS-UFb4DWNZJ3F"
"733gM656S4T-d4HmEgdAV6Z-9zuHAd3biKM-mBwNriZpqMB"
"8LiciitBVTi-DTmgiEUdqAJ-xXPckMvFHMc-JSiJMYvLaWh"
"8nFt2z7T9go-1qaCTTgodub-592n6gpmdNR-VRcyYAyT7qj"
"kfugGdXUGUc-mReaUnJxyUL-R44Lf7yH6RK-2JbZ1bv7dw"
Docker
The provided docker-compose setup runs this script to watch a single
directory, ./docs in current directory, for new files. If a new file
is detected, it is pushed to docspell.
This utilizes the integration
endpoint, which is enabled
in the config file, to allow uploading documents for all collectives.
A subfolder must be created for each registered collective. The docker
containers are configured to use http-header protection for the
integration endpoint. This requires you to provide a secret, that is
shared between the rest-server and the dsc tool. This can be done by
defining an environment variable which gets picked up by the
containers defined in docker-compose.yml:
docker-compose up -d
Now you can create a folder ./docs/<collective-name> and place all
files in there that you want to import. Once dropped in this folder
the consumedir container will push it to docspell.