Commit Graph

137 Commits

Author SHA1 Message Date
Eike Kettner
b1a1a2b837 Add archives to collective insights 2020-03-19 22:43:18 +01:00
Eike Kettner
439aaee27b Search archives when looking for files via checksum 2020-03-19 22:42:48 +01:00
Eike Kettner
4ed7a137f7 Add support for archive files
Each attachment is now first extracted into potentially multiple ones,
if it is recognized as an archive. This is the first step in
processing. The original archive file is also stored and the resulting
attachments are associated to their original archive.

First support is implemented for zip files.
2020-03-19 22:42:27 +01:00
Eike Kettner
00ca6b5697 Improve text analysis
- Search for consecutive labels

- Sort list of candidates by a weight

- Search for organizations using person labels
2020-03-17 22:34:50 +01:00
Eike Kettner
718e44a21c Add cleanup jobs task 2020-03-09 20:24:00 +01:00
Eike Kettner
854a596da3 Integrate periodic tasks
The first use case for periodic task is the cleanup of expired
invitation keys. This is part of a house-keeping periodic task.
2020-03-08 22:49:49 +01:00
Eike Kettner
616c333fa5 Implement storage routines for periodic scheduler 2020-03-08 13:56:23 +01:00
Eike Kettner
1e598bd902 Sketch a scheduler for running periodic tasks
Periodic tasks are special in that they are usually kept around and
started based on a schedule. A new component checks periodic tasks and
submits them in the queue once they are due.

In order to avoid duplicate periodic jobs, the tracker of a job is
used to store the periodic job id. Each time a periodic task is due,
it is first checked if there is a job running (or queued) for this
task.
2020-03-08 12:55:03 +01:00
Eike Kettner
42c59179b8 Fix search by checksum to include source files 2020-03-02 20:56:32 +01:00
Eike Kettner
2f87065b2e sbt scalafmtAll 2020-02-25 20:55:00 +01:00
Eike Kettner
cc16b0c024 Fix query to also work with mariadb 2020-02-24 13:34:54 +01:00
Eike Kettner
661cc3e65f Fix deleting attachments (again) 2020-02-23 20:18:13 +01:00
Eike Kettner
d937e0501a Add source files to collective insights 2020-02-23 20:17:53 +01:00
Eike Kettner
957073fe62 Return info about original files in item detail
This adds data to the current rest api.
2020-02-23 14:25:32 +01:00
Eike Kettner
74a037887d Fix deleting items and attachments to also remove the binary files 2020-02-22 00:54:55 +01:00
Eike Kettner
72fd3b1a25 Implement downloading original file 2020-02-20 22:33:57 +01:00
Eike Kettner
97305d27ff Integrate support for more files into processing and upload
The restriction that only pdf files can be uploaded is removed. All
files can now be uploaded. The processing may not process all. It is
still possible to restrict file uploads by types via a configuration.
2020-02-19 23:27:00 +01:00
Eike Kettner
ba3865ef5e Starting to support more file types
First, files are be converted to PDF for archiving. It is also easier
to create a preview. This is done via the `ConvertPdf` processing
task (which is not yet implemented).

Text extraction then tries first with the original file. If that
fails, OCR is done on the (potentially) converted pdf file.

To not loose information of the original file, it is saved using the
table `attachment_source`. If the original file is already a pdf, or
the conversion did not succeed, the `attachment` and
`attachment_source` record point to the same file.
2020-02-10 12:42:45 +01:00
Eike Kettner
5c37efeaba Apply scalafmt to all files 2020-02-09 01:54:26 +01:00
Eike Kettner
9b66604b96 Include item notes in search 2020-02-08 13:39:06 +01:00
Eike Kettner
6d0c140e8e Add mariadb database migration 2020-01-12 01:17:49 +01:00
Eike Kettner
d535130c9e Provide email proposals from address book 2020-01-12 01:04:42 +01:00
Eike Kettner
2ecfb679d9 Add routes to retrieve sent mails 2020-01-11 12:58:04 +01:00
Eike Kettner
b795a22992 Send mails for items 2020-01-10 00:45:29 +01:00
Eike Kettner
2d69d39dd1 Connect multiple items to a mail 2020-01-09 18:20:59 +01:00
Eike Kettner
7a3289c41d Prepare sending mail 2020-01-08 22:44:34 +01:00
Eike Kettner
32050a9faf Finish mail settings 2020-01-07 00:20:28 +01:00
Eike Kettner
f235f3a030 Starting with mail functionality 2020-01-05 23:23:28 +01:00
Eike Kettner
2e3454c7a1 Starting with mail settings 2020-01-05 15:31:32 +01:00
Eike Kettner
8814de3c38 Allow simple search when listing meta data 2020-01-02 20:21:49 +01:00
Eike Kettner
eb6c483ef0 Add route to check for files by their checksum
Adopt scripts in `tools/` to check for existing files using these
routes.
2019-12-31 23:45:02 +01:00
Eike Kettner
d05e919eb4 Update doobie, use legacy java.time conversions 2019-12-31 13:55:09 +01:00
Eike Kettner
fc3e22e399 Apply scalafmt to all files 2019-12-30 21:44:13 +01:00
Eike Kettner
a9e70401de Update dependencies 2019-12-28 12:38:11 +01:00
Eike Kettner
2ad1586d00 Set stricter compile options and fix cookie data 2019-09-28 22:17:45 +02:00
Eike Kettner
831cd8b655 Initial version.
Features:

- Upload PDF files let them analyze

- Manage meta data and items

- See processing in webapp
2019-09-21 22:02:36 +02:00
Eike Kettner
6154e6a387 Initial application stub 2019-09-21 14:54:03 +02:00