Commit Graph

240 Commits

Author SHA1 Message Date
c79bdad7e4 Rename methods 2022-01-10 15:09:49 +01:00
699cf091e6 Allow bookmarks in periodic query notification 2022-01-10 14:25:20 +01:00
3a642ee01e Fix sending notification mails from background tasks
When looking up the mail settings, a valid user id must be given.
2021-12-22 17:57:26 +01:00
0b606e6b05 Use logfmt for log lines and remove ansi color codes 2021-12-19 22:29:56 +01:00
c21b2cdd29 Update scalafmt to 3.0.8 2021-12-11 22:46:55 +01:00
4ffc8d1f14 Add support for more generic notification
This is a start to have different kinds of notifications. It is
possible to be notified via e-mail, matrix or gotify. It also extends
the current "periodic query" for due items by allowing notification
over different channels. A "generic periodic query" variant is added
as well.
2021-12-11 18:57:32 +01:00
501c6f2988 Updating stanford corenlp to 4.3.2; adding more languages
There are models for Spanish, that have been added now. Also the
Hungarian language has been added to the list of supported
languages (for tesseract mainly, no nlp models)
2021-11-20 14:31:39 +01:00
20fc9955ba Fix reprocessing to lookup the last used language 2021-11-20 01:34:16 +01:00
0651db9901 Make publish async, replace joexclient in periodic job scheduler 2021-11-16 22:55:37 +01:00
7a8527f821 Indicate number of running jobs in tob nav
Issue: #1069
2021-11-12 22:01:30 +01:00
cf933b60a7 Encode ws messages as JSON 2021-11-11 23:54:37 +01:00
7b8afe8371 Make internal endpoints available to nodes only 2021-11-08 00:13:02 +01:00
61c01ad79b Use job group instead of parsing the task arguments 2021-11-08 00:07:44 +01:00
3e58d97f72 Add websockets and notify frontend when an item is processed 2021-11-06 21:32:30 +01:00
f38d520a1d Hooking the new pubsub impl into the application 2021-11-06 01:44:06 +01:00
668cd7d974 Refactor config validation 2021-10-25 11:27:06 +02:00
118d23c3a2 Add list of env variables to documentation
Issue: #1121
2021-10-25 00:23:20 +02:00
4e5924d796 Refactor config loading and add config from environment
Issue: #1121
2021-10-24 23:02:39 +02:00
48d2bec9c4 Update http4s to 0.23.6 2021-10-24 18:24:03 +02:00
aa8f3b82fc Use passwords when reading PDFs 2021-09-30 11:48:59 +02:00
3c93b63c8a Add option to decrypt PDFs during conversion
Refs: #1074
2021-09-29 23:04:26 +02:00
9013f2de5b Update scalafmt settings 2021-09-22 17:23:24 +02:00
20a829cf7a Refactoring for migrating to binny library 2021-09-22 14:18:43 +02:00
9785db0683 Change license header of all files 2021-09-21 22:35:38 +02:00
193b81bf7d Fix version check
Refs: #1068
2021-09-21 22:07:19 +02:00
751fa3da5a Add attachments-only filter to uploads
When uploading a file which is an e-mail, this option allows to skip
the mail body when the file is being processed.
2021-08-21 13:49:12 +02:00
5d33b3841a Add a task to check for updates periodically
It must be enabled and configured by the admin.

Refs: #990
2021-08-20 00:25:27 +02:00
90421599ea Fix storing empty-trash task
It was wrongly stored using RPeriodicTask directly, but the higher
level `UserTask` must be used instead, because this ensures a
correctly scoped periodic task using the `updateOneTask` method. Since
this is a system task, it can be given a fixed ID which makes it now
safe even if stored using RPeriodicTask directly.

The bug resulted in multiple empty-trash tasks to be inserted (on each
restart).

Refs: #347
2021-08-20 00:25:25 +02:00
e4fecefaea Reformat with scalafmt 3.0.0 2021-08-19 08:50:30 +02:00
14e4a8f792 Fixup for deleting items
First, when checking for existence of a file, deleted items are not
conisdered.

The working with fulltext search has been changed: deleted items are
removed from fulltext index and are re-added when they are restored.
The fulltext index currently doesn't hold the item state and it would
mean much more work to introduce it into the index (or, worse, to
reprocess the results from the index). Thus, deleted items can only be
searched using database queries. It is probably a very rare use case
when fulltext search should be applied to deleted items. They have
been deleted for a reason and the most likely case is that they are
simply removed.

Refs: #347
2021-08-15 16:00:30 +02:00
f4a2b86ea8 Use a minimum age of items to remove
In order to keep deleted items for a while, the periodic task can now
use a duration to only remove items with a certain age. This can be
used to ensure that a deleted item stays at least X days before it
will be removed from the database.

Refs: #347
2021-08-15 12:32:50 +02:00
31d885ed79 Refactor user tasks to support collective and user scopes
Before, there were periodic tasks run per collective and not user by
making sure that submitter + group are the same value. This is now
encoded in `UserTaskScope` so it is now obvious and errors can be
reduced when using this.
2021-08-14 22:07:56 +02:00
27fd7a5867 Make sure the empty-trash task is started for all collectives 2021-08-14 20:40:04 +02:00
50706c3d6d Add a task implementation to delete items 2021-08-14 19:33:18 +02:00
1901fe1a8c Adopt deprecated APIs from fs2; use fs2.Path 2021-08-07 17:51:56 +02:00
1c0d87527b Log error when setting folder doesn't work 2021-07-17 15:10:00 +02:00
8e5c88fd32 Add copyright header to source files 2021-07-04 10:57:53 +02:00
bd791b4593 Upgrade code base to CE3 2021-06-22 22:53:34 +02:00
ac7d00c28f Refactor re-index task 2021-06-07 21:17:29 +02:00
3ee0846e19 Remove fts_migration table
It is now stored it SOLR instead.
2021-06-07 17:53:47 +02:00
5205ee0623 Store solr migration state in a solr document 2021-06-07 17:53:37 +02:00
bdc7822f50 Add documentation about docker setup 2021-05-31 22:19:49 +02:00
e1bbc2edf5 Apply autoformat 2021-04-10 16:31:58 +02:00
144ea852bf Update fs2-core, fs2-io to 2.5.4 2021-03-31 21:10:42 +02:00
c36073b852 Allow to give human readable summary to user tasks 2021-03-27 22:13:13 +01:00
cc38b850a6 Remove deprecated search routes and some refactoring 2021-03-27 22:13:13 +01:00
f8bd42e5bd Redo pdf conversion and text extraction on reprocess
When processing a new file conversion and text extraction is skipped
if detected to be already done. This prevents running expensive tasks
again after restarting/retrying. When explicitely reprocessing a file,
these tasks should run again and replace the existing results.
2021-03-12 00:45:28 +01:00
a7ee0aa08b Add a flag to processing task to distinguish re-/processing 2021-03-12 00:45:23 +01:00
058c31e1f6 Reprocessing now sets metadata to an item if not in state confirmed
When reprocessing an item, the metadat of all *files* are replaced.
This change now also sets some metadat to an item, but only if the
item is not in state "confirmed". Confirmed items are not touched, but
the metadata of the files is updated.
2021-03-12 00:16:19 +01:00
0229a867af Add a use colum to metadata entities 2021-03-10 23:55:18 +01:00