Commit Graph

296 Commits

Author SHA1 Message Date
eikek
309a52393a Initial impl for totp 2021-08-31 16:45:39 +02:00
eikek
cf88f5c2de Allow to specify ordering when retrieving meta data
The query now searches in more fields. For example, when getting a
list of tags, the query is applied to the tag name *and* category.
When listing persons, the query now also looks in the associated
organization name.

This has been used to make some headers in the meta data tables
clickable to sort the list accordingly.

Refs: #965, #538
2021-08-27 23:04:54 +02:00
eikek
993a391c13 Add the attachment-only option to a source
The upload request can now contain a boolean for importing only
attachments when e-mails are uploaded. This option is now also added
to a source url.

Refs: #983
2021-08-23 14:19:11 +02:00
eikek
751fa3da5a Add attachments-only filter to uploads
When uploading a file which is an e-mail, this option allows to skip
the mail body when the file is being processed.
2021-08-21 13:49:12 +02:00
Scala Steward
e4fecefaea
Reformat with scalafmt 3.0.0 2021-08-19 08:50:30 +02:00
eikek
85085ec173 Implement item merge 2021-08-16 12:32:54 +02:00
eikek
f4a2b86ea8 Use a minimum age of items to remove
In order to keep deleted items for a while, the periodic task can now
use a duration to only remove items with a certain age. This can be
used to ensure that a deleted item stays at least X days before it
will be removed from the database.

Refs: #347
2021-08-15 12:32:50 +02:00
eikek
31d885ed79 Refactor user tasks to support collective and user scopes
Before, there were periodic tasks run per collective and not user by
making sure that submitter + group are the same value. This is now
encoded in `UserTaskScope` so it is now obvious and errors can be
reduced when using this.
2021-08-14 22:07:56 +02:00
eikek
27fd7a5867 Make sure the empty-trash task is started for all collectives 2021-08-14 20:40:04 +02:00
eikek
4901276c66 Change "empty trash" settings for a collective and submit the job 2021-08-14 19:33:15 +02:00
eikek
f999662905 Add routes to restore deleted items 2021-08-14 16:48:03 +02:00
eikek
edb344314f Use an enum instead of a boolean to differentiate search
It's not very likely to have more modes of search besides normal and
trashed, but got surprised in that way quite often and it's nicer this
way anyways.
2021-08-14 15:11:48 +02:00
eikek
a7b74bd5ae Allow to search in soft-deleted items
A new query/request parameter can be used to apply a search to only
soft-deleted items.

The query expression `Trashed` has been introduced which selects only
items with state `Deleted`. This is another option an analog to
`ValidItemStates` (both cannot be used together as they would select
no items). This new query node is not added to the parser, because
users may not use it in their own queries - it must be part of the
"fixed" query so the application can control in which subset to search
(it would otherwise be possible to select any items).
2021-08-14 14:53:05 +02:00
eikek
cb777e30c0 Delete items by introducing a deleted state
When deleting items via the http api, they are not deleted anymore but
a new status "Deleted" is set. The collective insights contains now a
count separately for deleted items.
2021-08-14 14:18:03 +02:00
eikek
08ce6b5256 Fix path to viewerjs for pdf view 2021-08-13 00:58:13 +02:00
eikek
1901fe1a8c Adopt deprecated APIs from fs2; use fs2.Path 2021-08-07 17:51:56 +02:00
eikek
9578dd2b2b Add route to remove tags for a single item 2021-07-25 23:27:06 +02:00
eikek
fcef52856a Allow tag ids or tag names when replacing tags 2021-07-25 21:26:22 +02:00
eikek
916217df4f Make convert-all-pdfs an admin endpoint 2021-07-25 01:25:24 +02:00
eikek
f7eed33545 Return a 404 if a source was not found when checking a file 2021-07-08 21:17:48 +02:00
eikek
1120434cd9 Replace generating preview images with an admin endpoint
It doesn't make much sense to have this per collective, because this
is triggered by an admin after changing the server config file. So it
is now implemented as an admin endpoint that affects all files.
2021-07-04 21:37:34 +02:00
eikek
8e5c88fd32 Add copyright header to source files 2021-07-04 10:57:53 +02:00
eikek
7dcb61ef56 Use old ip checks and fix dev-ui-build script
These checks could be improved to not use javas InetAddress. But for
now it should have the same behaviour as before.
2021-06-26 14:43:16 +02:00
eikek
bd791b4593 Upgrade code base to CE3 2021-06-22 22:53:34 +02:00
eikek
bdc7822f50 Add documentation about docker setup 2021-05-31 22:19:49 +02:00
Eike Kettner
25788a0b23 Add routes for storing/retrieving client settings 2021-05-27 21:34:05 +02:00
Eike Kettner
a1a93e5ca6 Fixes searching items with fulltext
When using fulltext only search, then only the index must be searched.
This wasn't working anymore, because the routes added a query to
always select valid items (those not being processed). But this lead
to the downstream code to always consult the database, too. Since the
routes are using a "simple-search" interface, this is now adding the
valid-state condition if applicable. There are still more low-level
interfaces that can be used when searching should be done differently.

Closes: #823
2021-05-23 14:14:25 +02:00
Stefan Scheidewig
558197e415 Fixed the imports 2021-04-15 20:49:34 +02:00
Stefan Scheidewig
fa34312020 Implemented endpoint to delete multiple attachments 2021-04-15 18:05:01 +02:00
Stefan Scheidewig
93f772351a Made multi id read logic sharable 2021-04-15 18:03:58 +02:00
Eike Kettner
3e0914ece7 Correctly count tag categories
If multiple tags of the same category are applied to the same item,
just summing tag counts will produce the wrong results as now items
are counted multiple times.
2021-04-11 14:34:44 +02:00
Scala Steward
144ea852bf Update fs2-core, fs2-io to 2.5.4 2021-03-31 21:10:42 +02:00
Eike Kettner
c36073b852 Allow to give human readable summary to user tasks 2021-03-27 22:13:13 +01:00
Eike Kettner
03459124ca Don't return items still in processing 2021-03-27 22:13:13 +01:00
Eike Kettner
cc38b850a6 Remove deprecated search routes and some refactoring 2021-03-27 22:13:13 +01:00
Eike Kettner
274e433d9d Allow to change use enum for org/equipment 2021-03-10 23:55:18 +01:00
Eike Kettner
0229a867af Add a use colum to metadata entities 2021-03-10 23:55:18 +01:00
Eike Kettner
1a188afbd7 Re-add ui-version to keep browser resource cache in service-worker 2021-03-10 23:16:01 +01:00
Eike Kettner
64e3cb04f3 Remove fomantic ui and ui-header handling 2021-03-09 22:45:29 +01:00
Eike Kettner
7b1ec97c97 Fix sort when using fulltext only 2021-03-08 00:47:15 +01:00
Eike Kettner
63d146c2de Resolve fulltext search queries the same way as before
For now, fulltext search is only possible when being the only term or
inside the root AND expression.
2021-03-07 09:40:47 +01:00
Eike Kettner
d4006461f6 Serving scalajs artifacts and provide errors to js 2021-03-03 00:45:10 +01:00
Eike Kettner
dadab0d308 Implement search by query in endpoints 2021-03-01 15:31:02 +01:00
Eike Kettner
e079ec1987 Provide custom error structure for parse failures 2021-03-01 00:51:01 +01:00
Eike Kettner
9013d9264e Add more convenient date parsers and some basic macros 2021-03-01 00:51:01 +01:00
Eike Kettner
af73b59ec2 Parser improvements
- default expressions into a and node
- fix parsing string lists that end in whitespace
- fix package names of internal classes
2021-03-01 00:51:01 +01:00
Eike Kettner
a80d73d5d2 Optimize imports 2021-03-01 00:51:01 +01:00
Eike Kettner
e9ed998e3a Basic poc to search via custom query 2021-03-01 00:51:01 +01:00
Eike Kettner
186014a1c6 Refactor search to separate between a base query and user query
The `findBase` is adding only strictly required conditions. Everything
else comes from the user.
2021-03-01 00:51:01 +01:00
Eike Kettner
e6d9ce2c37 Remove obsolete type capabilities
These are now detected by the new scala compiler and lead to compile
errors.
2021-03-01 00:16:30 +01:00
mergify[bot]
9156e56d65
Merge pull request #651 from eikek/language-source
Add language to a source
2021-02-19 01:30:24 +00:00
Eike Kettner
7ef3185659 Add language to a source
Allows to define upload urls for different languages.
2021-02-18 23:34:42 +01:00
Eike Kettner
5c6f57cc6a Trim values of some input data
Some values don't make sense when padded with whitespace, like
organization name, source ids etc. These are trimmed now when
received.
2021-02-18 22:07:32 +01:00
Eike Kettner
5181283b1b Add a short-name to organizations 2021-02-17 22:55:35 +01:00
Eike Kettner
20ccdda609 Add a notes field to equipments 2021-02-17 22:39:07 +01:00
Eike Kettner
48eee00c0b Allow person to be correspondent, concerning or both 2021-02-16 22:49:55 +01:00
Eike Kettner
f29353adb3 Use different worker cache name per ui
Avoid to have users clear their browser caches
2021-02-14 11:29:36 +01:00
Eike Kettner
f6e5bda2d8 Change default ui to be the new ui
The old ui is still available, using a header: `Docspell-Ui: 1`. Using
a browser addon a "ui switch" can be easily achieved.
2021-02-14 01:46:13 +01:00
Eike Kettner
dd935454c9 First version of new ui based on tailwind
This drops fomantic-ui as css toolkit and introduces tailwindcss. With
tailwind there are no predefined components, but it's very easy to
create those. So customizing the look&feel is much simpler, most of
the time no additional css is needed.

This requires a complete rewrite of the markup + styles. Luckily all
logic can be kept as is. The now old ui is not removed, it is still
available by using a request header `Docspell-Ui` with a value of `1`
for the old ui and `2` for the new ui.

Another addition is "dev mode", where docspell serves assets with a
no-cache header, to disable browser caching. This makes developing a
lot easier.
2021-02-14 01:46:13 +01:00
Eike Kettner
96612e0e59 Refactor scan mailbox form and add flag for post-processing
Mails are filtered once by using an imap search and then by some globs
to filter files and subjects. Imap can search by subject via a
string-contains, but not via globs or patterns (afaik). The subject
filter is applied to all downloaded mail headers. Now for post
processing (moving to some target folder or deleting), it can be
chosen to post-process all "seen" mails or only those that matched all
filters.
2021-01-24 01:46:31 +01:00
mergify[bot]
8dd1672c8c
Merge pull request #583 from eikek/fix-baseurl-setting
Render baseurl without trailing slash
2021-01-21 23:44:14 +00:00
Eike Kettner
0ec620fcf0 Render baseurl without trailing slash
The webapp expects it like this currently, because the url is only a
string.
2021-01-21 21:42:08 +01:00
Eike Kettner
a6f29153c4 Control what tag categories to use for auto-tagging 2021-01-19 01:20:13 +01:00
Eike Kettner
249f9e6e2a Extend guessing tags to all tag categories 2021-01-18 21:51:45 +01:00
Eike Kettner
b08e88cd69 Add (inofficial) routes to get system information 2021-01-05 20:54:53 +01:00
Eike Kettner
668abf2140 Add a reset-password admin route 2021-01-04 20:59:31 +01:00
Eike Kettner
2a172ce720 Remove fulltext recreate-key config value
It's now in the admin routes, protected by the
`admin-endpoint.secret`.
2021-01-04 15:18:02 +01:00
Eike Kettner
306f064ad9 Create a place for admin routes
And move re-creation of fulltext index in this place.
2021-01-04 15:18:02 +01:00
Eike Kettner
0cfd8974d3 Add a flag to imap settings to enable/disable oauth2 scheme 2021-01-04 11:03:04 +01:00
Eike Kettner
6346bf6a34 Add summary for fulltext searches 2020-12-17 00:11:33 +01:00
Eike Kettner
8fba637ebe Add folder counts to search summary 2020-12-16 01:14:27 +01:00
Eike Kettner
80e23d1c84 Add a route to get search summary 2020-12-15 23:37:02 +01:00
Eike Kettner
80406cabc2 Refactoring some code into separate files 2020-12-15 21:03:47 +01:00
Eike Kettner
adee496b77 Convert source record 2020-12-15 21:03:46 +01:00
Eike Kettner
b338f18e98 Remove redundant fileCount from search result 2020-12-15 21:03:46 +01:00
Eike Kettner
5882405f30 Search index if search object only contains this field 2020-12-05 02:59:57 +01:00
Eike Kettner
0337391b20 Change remember period to 30 days 2020-12-04 20:30:25 +01:00
Eike Kettner
a0642905db Use remember-me cookie if present 2020-12-04 17:59:25 +01:00
Eike Kettner
c10c1fad72 Prepare remember-me authentication variant 2020-12-04 17:59:25 +01:00
Eike Kettner
fc2668feee Allow to connect a person to an organization 2020-12-01 23:39:45 +01:00
Eike Kettner
0ee8ff66d5 Allow to search by source name 2020-11-30 14:07:45 +01:00
Eike Kettner
7052bc6b8e Add cc and bcc to item mail 2020-11-28 01:36:59 +01:00
Eike Kettner
5fe532001b Allow to specify document lanugage with the request 2020-11-23 20:49:01 +01:00
Eike Kettner
7b7f1e4d6d Return custom field values with search results 2020-11-23 10:23:25 +01:00
Eike Kettner
066c856981 Allow to search for custom field values 2020-11-22 21:41:09 +01:00
Eike Kettner
1aefff37aa Return custom field values with item details 2020-11-22 21:41:09 +01:00
Eike Kettner
8d35d100d6 Change custom fields for multiple items 2020-11-22 21:41:09 +01:00
Eike Kettner
93295d63a5 Change custom field values for a single item 2020-11-22 21:41:09 +01:00
Eike Kettner
62313ab03a Add and change custom fields 2020-11-22 21:41:09 +01:00
Eike Kettner
248ad04dd0 Prepare custom fields 2020-11-22 21:41:09 +01:00
Eike Kettner
5034e12bec Add a subject filter to scan-mailbox args 2020-11-13 23:15:20 +01:00
Eike Kettner
04ba14f802 Amend source form with tags and file-filter
Allow to define tags and a file filter per source.
2020-11-12 22:37:28 +01:00
Eike Kettner
4fd6e02ec0 Improve glob and filter archive entries 2020-11-11 21:01:23 +01:00
Eike Kettner
55a6f7aaf6 Add more properties to upload meta data 2020-11-11 21:01:23 +01:00
Eike Kettner
29455d638c Add startup task to find page counts of existing files 2020-11-09 20:35:35 +01:00
Eike Kettner
8c08bf233d Amend search results with attachment info
This uses again another query per item to retrieve some information
about each attachment already in the search results.
2020-11-09 14:24:28 +01:00
Eike Kettner
8c8788bc69 Provide fallback image for previews 2020-11-09 09:00:03 +01:00
Eike Kettner
f4e50c5229 Provide endpoints to submit tasks to re-generate previews
The scaling factor can be given in the config file. When this changes,
images can be regenerated via POSTing to certain endpoints. It is
possible to regenerate just one attachment preview or all within a
collective.
2020-11-09 09:00:02 +01:00
Eike Kettner
757ad31165 Add a route to get the item preview
This is the first available preview of an attachment wrt position. If
all attachments have a preview image, the preview of the first
attachment is returned.
2020-11-08 15:12:56 +01:00
Eike Kettner
8cc89fd3b7 Move handling binary responses to a shared space 2020-11-08 15:12:56 +01:00