TheAnachronism/docspell
1
0
Fork 0
mirror of https://github.com/TheAnachronism/docspell.git synced 2024-11-13 02:31:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
NaN Commits 2 Branches 1 Tag 89 MiB
383614f908
Commit Graph

336 Commits

Author SHA1 Message Date
Eike Kettner
229baa0719 Fix redirecting / to /app 
Before all paths not otherwise handled were redirected, but it should
only be the root path.
 2020-05-23 09:58:34 +02:00
Eike Kettner
4227278bb0 Update documentation 2020-05-23 00:10:07 +02:00
Eike Kettner
69c46f9f1f Fix date-picker initialization in search menu 2020-05-21 23:27:08 +02:00
Eike Kettner
920fcf28dd Change webapp to support multiple scan-mailbox tasks 2020-05-21 22:04:45 +02:00
Eike Kettner
9f9dd6c0fb Change routes for scan-mailbox task to allow multiple tasks per user 2020-05-21 22:04:45 +02:00
Eike Kettner
743aa9d754 Hide correct list element in item card 2020-05-21 01:02:46 +02:00
Eike Kettner
4724e691bc Update user info in scan-mailbox form 2020-05-21 00:09:47 +02:00
Eike Kettner
09b6b1bb05 Improve user-task texts 2020-05-20 22:58:19 +02:00
Eike Kettner
d9782582d8 Use max-mails setting with higher priority 
The `mail-chunk-size` is set to its configured value or `max-mails`
whichever is lower.
 2020-05-20 22:44:29 +02:00
Eike Kettner
c0259dba7e Allow to enable debug flag for javamail 2020-05-20 22:15:25 +02:00
Eike Kettner
2858d6b853 Notify job executors at the end of the task 2020-05-20 19:44:45 +02:00
Eike Kettner
31a1abf395 Add server limits to importing mails task 2020-05-20 17:52:38 +02:00
Eike Kettner
f2d67dc816 Initial impl of import from mailbox user task 2020-05-20 17:52:38 +02:00
Eike Kettner
451a09dda0 Allow to skip joex notification on uploads 2020-05-20 17:52:38 +02:00
Eike Kettner
852455c610 Add upload operation to task arguments 2020-05-20 17:52:38 +02:00
Eike Kettner
6e8582ea80 Implement scan-mailbox form and routes 2020-05-20 17:52:38 +02:00
Eike Kettner
0d6677f90b Add stub form scan-mailbox form 2020-05-20 17:52:38 +02:00
Eike Kettner
a4be63fd77 Add stub for scan-mailbox task 2020-05-20 17:52:38 +02:00
Eike Kettner
ebe65c7624 Change menu title in user settings 2020-05-20 17:52:38 +02:00
Eike Kettner
5d5311913c Add ScanMailboxArgs 2020-05-20 17:52:38 +02:00
Eike Kettner
c9de74fd91 Add imap settings 2020-05-18 08:46:04 +02:00
Eike Kettner
cece5b6193 Fix mobile support 
Next to the missing css classs, the viewport definition was missing in
the html.
 2020-05-17 23:21:42 +02:00
Eike Kettner
f5c23ceb3d Change edit-notes icon 2020-05-17 22:41:25 +02:00
Eike Kettner
dd2e66095a Update screenshots 2020-05-17 22:23:11 +02:00
Eike Kettner
ee4d4e7ae5 Improve responsiveness of detail view 2020-05-17 21:58:19 +02:00
Eike Kettner
72a88f21cb Use cards for viewing items 2020-05-17 21:42:16 +02:00
Eike Kettner
98f202b797 More ui tweaks to item list 2020-05-17 18:16:04 +02:00
Eike Kettner
62111e5f93 Allow to hide the search menu 
This is more convenient on small screens
 2020-05-17 18:15:44 +02:00
Eike Kettner
f45b40342c Improve document list view 
Replace the html table with something that has more vertical space,
but scales better horizontally. On most screens at least 6-8 entries
should be visible at once.
 2020-05-17 18:15:22 +02:00
Eike Kettner
d65c1e0d36 Use date from e-mails to set item date 2020-05-17 11:58:51 +02:00
Eike Kettner
5e6ce1737c Change recognizing dates with short years 
Short years are now added to the current centure (2000) such that date
strings like 12/26/11 result in 12/26/2011 and not 12/26/1911.
 2020-05-17 11:58:51 +02:00
Eike Kettner
3e10e2175a Sort by weights better and save them 2020-05-17 11:58:51 +02:00
Scala Steward
5d6658770e Update emil-common, emil-doobie, ... to 0.6.0 2020-05-17 11:55:53 +02:00
Eike Kettner
6747a86fea Simplify jsoup sanitizer to reuse from emil 2020-05-14 23:56:08 +02:00
Eike Kettner
0a5501dcb0 Fix findFileByChecksum 2020-05-10 21:03:12 +02:00
Eike Kettner
9c882e1be9 Fix package name 2020-05-10 21:03:12 +02:00
Eike Kettner
bd5066740d Joex depends on backend module 
The job executor depends on backend module, since it may control the
application via user tasks. The `ONode` can now be moved from the
store module into the backend module.
 2020-05-10 21:03:12 +02:00
Eike Kettner
c41cdeefec Update scalafmt to 2.5.1 + scalafmtAll 2020-05-04 23:53:57 +02:00
Eike Kettner
9ec75cf85e Set next version to 0.6.0-SNAPSHOT 2020-04-30 22:39:02 +02:00
Eike Kettner
e1117bcc92 Set version to 0.5.0 2020-04-30 22:04:29 +02:00
Eike Kettner
96c5e99f19 Fix scaladoc tag 
There is no scaladoc tag @implNote.
 2020-04-30 22:04:29 +02:00
Eike Kettner
a9869e927e Add new feature to list 2020-04-30 21:41:19 +02:00
Eike Kettner
1d1df58cb9 Put edit notes icon in item action bar 2020-04-30 21:23:56 +02:00
Eike Kettner
0a1b3fcf95 Set list-id header for notification mails 2020-04-30 21:23:56 +02:00
Eike Kettner
75a66ecb86 Update http4s to 0.21.4 2020-04-29 01:05:13 +02:00
Eike Kettner
7fa25c3711 Move notes toggle to top 2020-04-27 21:25:49 +02:00
Eike Kettner
a939839041 Delete single attachments 2020-04-26 23:11:49 +02:00
Eike Kettner
75c32fa2bf Remove wrong header line in upload page 2020-04-24 23:25:04 +02:00
Eike Kettner
fa10fe3fae Update scala to 2.13.2 2020-04-24 22:24:31 +02:00
Eike Kettner
315ea63f44 Improve notify mail template 2020-04-23 23:17:34 +02:00
Eike Kettner
af4e5f161d Add documentation about notify-due-items 2020-04-23 22:26:06 +02:00
Eike Kettner
06d2c9ecba Add info text to int field 2020-04-23 21:46:10 +02:00
Eike Kettner
84e0ebf1a2 Add a flag for restricting overdue items 2020-04-23 21:37:03 +02:00
Eike Kettner
b2ca314da9 Check code formatting with travis ci 2020-04-23 20:25:21 +02:00
Eike Kettner
d52efdfcf0 Improve mail template 2020-04-22 23:41:09 +02:00
Eike Kettner
ffc1cdee51 Sort due items by their earliest due date 2020-04-22 22:21:28 +02:00
Eike Kettner
e1f9ae2629 Include links to items into mail template 2020-04-22 21:53:25 +02:00
Eike Kettner
bbfd694b45 Allow to start a user task once 2020-04-22 21:08:45 +02:00
Eike Kettner
af5b62c057 Remove form message on updates 2020-04-22 21:08:45 +02:00
Eike Kettner
2723d6b43b Implement notify-due-items task 2020-04-22 21:08:45 +02:00
Eike Kettner
e7b81c701f Better handle no connections 2020-04-22 21:08:45 +02:00
Eike Kettner
2042824ab4 Fix initial loading notify form 2020-04-22 21:08:45 +02:00
Eike Kettner
93182c040e Save notify-due-items user task 2020-04-22 21:08:45 +02:00
Eike Kettner
3a90d874a5 Improve form 2020-04-22 21:08:45 +02:00
Eike Kettner
5a2e28415a Streamline form and input field 2020-04-22 21:08:45 +02:00
Eike Kettner
c8683743d0 Provide help within the form 2020-04-22 21:08:45 +02:00
Eike Kettner
5bfa7b05a0 Add better input for calendar events 2020-04-22 21:08:45 +02:00
Eike Kettner
3524904faf Add routes to check calendar events 2020-04-22 21:08:45 +02:00
Eike Kettner
9b30542974 Copy/paste form together 2020-04-22 21:08:45 +02:00
Eike Kettner
ad772c0c25 Server-side stub impl for notify-due-items 2020-04-22 21:08:45 +02:00
Eike Kettner
e97e0db45c Prepare notification form 2020-04-22 21:08:45 +02:00
Eike Kettner
362e1a5e14 Fix compile errors in test code 2020-04-07 23:00:25 +02:00
Eike Kettner
1206105f0b Fix several bugs with handling e-mail files 
- When converting from html->pdf, the wkhtmltopdf program exits with
  errors if the document contains invalid links. The content is now
  cleaned before handed to wkhtmltopdf.
- Update emil library which fixes a bug when reading mails without
  explicit transfer encoding (8bit)
- Add a info header to converted mails
 2020-04-07 22:38:25 +02:00
Eike Kettner
63161b5bdf Add docker setup to quickstart 2020-03-31 22:56:51 +02:00
Eike Kettner
efc73c1060 Set version to 0.5.0-SNAPSHOT 2020-03-28 23:52:15 +01:00
Eike Kettner
c77ead3921 Set version to 0.4.0 2020-03-28 21:44:14 +01:00
Eike Kettner
e331808ecf Update microsite 2020-03-28 21:44:14 +01:00
Eike Kettner
6a1297fc95 Add a limit for text analysis 2020-03-27 22:54:49 +01:00
Eike Kettner
14a25fe23e Fix serializing mediatype parameters 2020-03-27 21:50:06 +01:00
Eike Kettner
aed5dfaff6 Fix mimetype extractors 2020-03-27 21:49:55 +01:00
Eike Kettner
75405dbcba Update documentation 2020-03-27 20:16:18 +01:00
Eike Kettner
16edf84752 Setup new site 2020-03-27 00:35:15 +01:00
Eike Kettner
9656ba62f4 scalafmtAll 2020-03-26 18:26:00 +01:00
Eike Kettner
09ea724c13 Store message-id of eml files 2020-03-25 22:00:51 +01:00
Eike Kettner
43efb4e6ba Use doobie support from emil project 2020-03-24 23:40:29 +01:00
Eike Kettner
e305b46708 Extract tnef attachments and fix incomplete html 
The wkhtmltopdf requires the content encoding set correctly in the
document.
 2020-03-24 23:40:29 +01:00
Eike Kettner
0b80572664 Fix encodings for mails with non-utf8 html parts 2020-03-24 23:40:29 +01:00
Eike Kettner
cf7ccd572c Improve handling encodings 
Html and text files are not fixed to be UTF-8. The encoding is now
detected, which may not work for all files. Default/fallback will be
utf-8.

There is still a problem with mails that contain html parts not in
utf8 encoding. The mail text is always returned as a string and the
original encoding is lost. Then the html is stored using utf-8 bytes,
but wkhtmltopdf reads it using latin1. It seems that the `--encoding`
setting doesn't override encoding provided by the document.
 2020-03-23 22:51:28 +01:00
Eike Kettner
b265421a46 Allow to use the browser's pdf viewer 
The viewerjs library has some limitations. Sometimes PDFs are quite
blurry and some content is displayed scrambled. Switching to the
browsers build-in PDF viewer (for chromium and firefox) fixes this. So
while on mobile the viewerjs is the only working viewer, for desktop
use it might be desireable to use the browsers builtin viewer instead.
 2020-03-22 22:03:43 +01:00
Eike Kettner
75ead33652 Provide a download link to the original archive file 2020-03-22 21:48:49 +01:00
Eike Kettner
7e6eec9533 Include archive infos in item detail 2020-03-22 21:35:50 +01:00
Eike Kettner
cbc95b11e6 Add routes to retrive the archive of an attachment 2020-03-22 21:21:49 +01:00
Eike Kettner
9a99c852a8 Fix typo in search menu 2020-03-22 21:08:01 +01:00
Eike Kettner
3703dce9a6 Update fs2 to 2.3.0 2020-03-20 22:47:09 +01:00
Eike Kettner
cba466ed47 Set item due date candidate 
After processing, set the due date of an item to the first candidate.
The earliest due date is considered best match.
 2020-03-20 22:39:09 +01:00
Eike Kettner
74a6cf1dd1 Remove unused migration directory 2020-03-19 22:43:41 +01:00
Eike Kettner
b1a1a2b837 Add archives to collective insights 2020-03-19 22:43:18 +01:00
Eike Kettner
d78bd4142c Update documentation 2020-03-19 22:42:58 +01:00
Eike Kettner
439aaee27b Search archives when looking for files via checksum 2020-03-19 22:42:48 +01:00
Eike Kettner
6b1156182c Add support for eml (rfc822 email) files 2020-03-19 22:42:40 +01:00
Eike Kettner
4ed7a137f7 Add support for archive files 
Each attachment is now first extracted into potentially multiple ones,
if it is recognized as an archive. This is the first step in
processing. The original archive file is also stored and the resulting
attachments are associated to their original archive.

First support is implemented for zip files.
 2020-03-19 22:42:27 +01:00
Eike Kettner
10f3d5b7ed Fix bug to select other attachments 2020-03-17 22:37:43 +01:00
Eike Kettner
f0449dd2ce Properly initialize thread pools 2020-03-17 22:37:12 +01:00
Eike Kettner
00ca6b5697 Improve text analysis 
- Search for consecutive labels

- Sort list of candidates by a weight

- Search for organizations using person labels
 2020-03-17 22:34:50 +01:00
Eike Kettner
718e44a21c Add cleanup jobs task 2020-03-09 20:24:00 +01:00
Eike Kettner
854a596da3 Integrate periodic tasks 
The first use case for periodic task is the cleanup of expired
invitation keys. This is part of a house-keeping periodic task.
 2020-03-08 22:49:49 +01:00
Eike Kettner
616c333fa5 Implement storage routines for periodic scheduler 2020-03-08 13:56:23 +01:00
Eike Kettner
1e598bd902 Sketch a scheduler for running periodic tasks 
Periodic tasks are special in that they are usually kept around and
started based on a schedule. A new component checks periodic tasks and
submits them in the queue once they are due.

In order to avoid duplicate periodic jobs, the tracker of a job is
used to store the periodic job id. Each time a periodic task is due,
it is first checked if there is a job running (or queued) for this
task.
 2020-03-08 12:55:03 +01:00
Eike Kettner
9b28858d06 Create a simple client for joex in its api module 
This client can be used within the backend app and later in other
modules. The `OJoex` object is replaced with a better implementation
where the http client is initialized once on app start.
 2020-03-03 23:07:49 +01:00
Eike Kettner
42c59179b8 Fix search by checksum to include source files 2020-03-02 20:56:32 +01:00
Eike Kettner
867b59ac10 Fix link in doc menu 2020-03-01 14:08:21 +01:00
Eike Kettner
d8bbcb1409 Fix front-page links for microsite 
The links work while testing locally with jekyll. Must be checked at
the published site.
 2020-03-01 09:45:38 +01:00
Eike Kettner
b7f2c051f4 Set next version to 0.4.0-SNAPSHOT 2020-02-28 21:19:01 +01:00
Eike Kettner
aa3b9258c4 Set version to 0.3.0 2020-02-28 20:52:39 +01:00
Eike Kettner
3f53779ae4 Change documentation side menu and front 2020-02-28 20:52:39 +01:00
Eike Kettner
ad8d64eded Fix microsite and add changelog 2020-02-27 23:59:03 +01:00
Eike Kettner
1bb464b9ed Extend tools/ds.sh to check for file existence 2020-02-27 20:03:46 +01:00
Eike Kettner
902fd63125 Fix initializing concerned equipment 2020-02-26 20:43:16 +01:00
Eike Kettner
2f87065b2e sbt scalafmtAll 2020-02-25 20:55:00 +01:00
Eike Kettner
c8d090ae28 Remove small notes form field in favor for the new one 2020-02-24 22:34:32 +01:00
Eike Kettner
381de1e198 Show project version in the documentation 2020-02-24 20:59:15 +01:00
Eike Kettner
25c3f2b541 Add more explaining tooltips 2020-02-24 15:18:42 +01:00
Eike Kettner
478797e2a4 Add a help link to the main menu 2020-02-24 15:11:58 +01:00
Eike Kettner
36093c5d52 Add reverse proxy doc 2020-02-24 15:11:50 +01:00
Eike Kettner
cc16b0c024 Fix query to also work with mariadb 2020-02-24 13:34:54 +01:00
Eike Kettner
5f32eadaba Fix dropdown in source create view 2020-02-23 23:01:48 +01:00
Eike Kettner
661cc3e65f Fix deleting attachments (again) 2020-02-23 20:18:13 +01:00
Eike Kettner
d937e0501a Add source files to collective insights 2020-02-23 20:17:53 +01:00
Eike Kettner
be8eacdbe9 Display full attachment name in title tooltip 2020-02-23 14:36:19 +01:00
Eike Kettner
1f431c3222 Make link to original file enabled if files are different 2020-02-23 14:33:22 +01:00
Eike Kettner
957073fe62 Return info about original files in item detail 
This adds data to the current rest api.
 2020-02-23 14:25:32 +01:00
Eike Kettner
ec419c7bfd Adopt nix modules to new config 2020-02-22 12:40:56 +01:00
Eike Kettner
74a037887d Fix deleting items and attachments to also remove the binary files 2020-02-22 00:54:55 +01:00
Eike Kettner
8cfecfb3dd Update docs 2020-02-22 00:48:58 +01:00
Eike Kettner
98576a5fb5 Add link to original file 2020-02-20 22:40:27 +01:00
Eike Kettner
72fd3b1a25 Implement downloading original file 2020-02-20 22:33:57 +01:00
Eike Kettner
39809f9d05 Sketch route for retrieving original file 2020-02-20 22:12:27 +01:00
Eike Kettner
7fe8843893 Update documentation sites 2020-02-20 21:43:37 +01:00
Eike Kettner
3f316ab4d0 Update config file doc 2020-02-20 21:10:00 +01:00
Eike Kettner
fbe0c1aec5 Allow more chars for mimetype 2020-02-20 00:39:31 +01:00
Eike Kettner
97305d27ff Integrate support for more files into processing and upload 
The restriction that only pdf files can be uploaded is removed. All
files can now be uploaded. The processing may not process all. It is
still possible to restrict file uploads by types via a configuration.
 2020-02-19 23:27:00 +01:00
Eike Kettner
9b1349734e Convert some files to pdf 2020-02-19 02:03:10 +01:00
Eike Kettner
5869e2ee6e Streamline extern-conv stdin/infile 2020-02-18 12:43:47 +01:00
Eike Kettner
0dcc00836b Make logger configurable in system commands 2020-02-18 12:02:43 +01:00
Eike Kettner
bd605b8c94 Add first drafts for converting 2020-02-18 01:31:22 +01:00
Eike Kettner
c665c212a0 Early draft for running wkhtmltopdf 2020-02-17 14:02:23 +01:00
Eike Kettner
e0682464b5 Configure pdf extraction; move Logger and DataType to common 2020-02-17 14:01:36 +01:00
Eike Kettner
3d615181e0 Early draft for text extraction 2020-02-17 01:57:22 +01:00
Eike Kettner
8143a4edcc Adding extraction primitives 2020-02-16 21:37:26 +01:00
Eike Kettner
851ee7ef0f Reorganize processing code 
Use separate modules for

- text extraction
- conversion to pdf
- text analysis
 2020-02-15 21:25:25 +01:00