TheAnachronism/docspell
1
0
Fork 0
mirror of https://github.com/TheAnachronism/docspell.git synced 2024-11-13 02:31:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
NaN Commits 2 Branches 1 Tag 89 MiB
4362301212
Commit Graph

1207 Commits

Author SHA1 Message Date
Eike Kettner
af73b59ec2 Parser improvements 
- default expressions into a and node
- fix parsing string lists that end in whitespace
- fix package names of internal classes
 2021-03-01 00:51:01 +01:00
Eike Kettner
a80d73d5d2 Optimize imports 2021-03-01 00:51:01 +01:00
Eike Kettner
e9ed998e3a Basic poc to search via custom query 2021-03-01 00:51:01 +01:00
Eike Kettner
186014a1c6 Refactor search to separate between a base query and user query 
The `findBase` is adding only strictly required conditions. Everything
else comes from the user.
 2021-03-01 00:51:01 +01:00
Eike Kettner
c3cdec416c Sketching some basic tests 2021-03-01 00:50:52 +01:00
Eike Kettner
be5c7ffb88 First draft of ast and parser 2021-03-01 00:46:57 +01:00
Eike Kettner
74a79a79d9 Initial project setup 2021-03-01 00:46:57 +01:00
Eike Kettner
cfa36a5270 Fix preview png tests 
Outcome was checked manually.
 2021-03-01 00:33:57 +01:00
Eike Kettner
e6d9ce2c37 Remove obsolete type capabilities 
These are now detected by the new scala compiler and lead to compile
errors.
 2021-03-01 00:16:30 +01:00
Eike Kettner
c7d4c77e6d Allow more suggestions for date variants in English 2021-02-26 00:35:17 +01:00
Yumekui
017241bccb
Fix typo 2021-02-22 15:09:10 +00:00
Eike Kettner
cd2f623773 Set version to 0.21.0-SNAPSHOT 2021-02-19 23:32:10 +01:00
Eike Kettner
4d3a25a8c1 Set version to 0.20.0 2021-02-19 22:36:55 +01:00
Eike Kettner
07a061febc Make sure certain styles are not stripped by purge-css 2021-02-19 22:36:55 +01:00
Eike Kettner
211499a7e6 Add bottom border to metadata view 2021-02-19 11:17:35 +01:00
Eike Kettner
d819878974 Re-add the button to create new tags in detail view 2021-02-19 11:07:42 +01:00
mergify[bot]
9156e56d65
Merge pull request #651 from eikek/language-source 
Add language to a source
 2021-02-19 01:30:24 +00:00
Eike Kettner
7ef3185659 Add language to a source 
Allows to define upload urls for different languages.
 2021-02-18 23:34:42 +01:00
Eike Kettner
5c6f57cc6a Trim values of some input data 
Some values don't make sense when padded with whitespace, like
organization name, source ids etc. These are trimmed now when
received.
 2021-02-18 22:07:32 +01:00
Eike Kettner
70fa57587b Fix setting ui theme for anonymous 2021-02-18 22:02:56 +01:00
Eike Kettner
0e9d8f8794 Fix hover when folder is a drop-target 2021-02-18 00:43:15 +01:00
Eike Kettner
d7bc963450 Cleanup nodes that are not reachable anymore 2021-02-18 00:37:18 +01:00
mergify[bot]
b16166f1e9
Merge pull request #645 from eikek/equipment-description 
Equipment description
 2021-02-17 22:17:03 +00:00
Eike Kettner
5181283b1b Add a short-name to organizations 2021-02-17 22:55:35 +01:00
Eike Kettner
20ccdda609 Add a notes field to equipments 2021-02-17 22:39:07 +01:00
Eike Kettner
e07d732193 Show item position in detail view 2021-02-17 22:18:22 +01:00
Eike Kettner
a79fa6f74a Add version info back to frontend 2021-02-17 21:54:53 +01:00
Eike Kettner
e177336155 Better align tags in multi-select field 2021-02-17 00:14:06 +01:00
Eike Kettner
4ef1996137 Fix extracted content border 2021-02-17 00:14:06 +01:00
Eike Kettner
33cfd6df6f Hide Load-more button when there is no more 2021-02-17 00:14:06 +01:00
Eike Kettner
29d80adace Ui updates: place result message below buttons 2021-02-17 00:14:06 +01:00
Eike Kettner
48eee00c0b Allow person to be correspondent, concerning or both 2021-02-16 22:49:55 +01:00
Eike Kettner
d99ce76d89 Remove person suggestion if it doesn't match with organization 2021-02-16 00:29:54 +01:00
Eike Kettner
eb308cfa85 Remove duplicate candidates when creating list of suggestions 2021-02-15 23:56:09 +01:00
Eike Kettner
64bacb665a Show queue counts in side menu 2021-02-15 00:27:55 +01:00
Eike Kettner
07edaaaf5e Try fix attachment selection for firefox 
It works great in chromium, in firefox the border is a bit off.
 2021-02-15 00:11:37 +01:00
Eike Kettner
9c90d7c4c2 Fix min height of tag area in detail 
This avoids "nervous/noisy" ui flickering when changing between items
where one has no tags/fields to show.
 2021-02-15 00:10:38 +01:00
Eike Kettner
a10971607e Always show custom field dropdown in detail mode 
Even if there are no custom fields define, the control is shown now.
The user can create a new one in this view.
 2021-02-14 23:34:18 +01:00
Eike Kettner
dec6148df6 Hide field stats component if there is nothing to show 2021-02-14 23:29:05 +01:00
Eike Kettner
31cb8284a5 Fix formatting icon in top nav bar 2021-02-14 23:22:37 +01:00
Eike Kettner
1dcc3941ed Fix overflowing content in item cards 
When displaying long lines this leads to horizontal scrolling the
complete item list.
 2021-02-14 23:18:18 +01:00
Eike Kettner
f29353adb3 Use different worker cache name per ui 
Avoid to have users clear their browser caches
 2021-02-14 11:29:36 +01:00
Eike Kettner
5461321e59 Remove unused styles and prevent postcss from purging relevant ones 2021-02-14 02:20:01 +01:00
Eike Kettner
f6e5bda2d8 Change default ui to be the new ui 
The old ui is still available, using a header: `Docspell-Ui: 1`. Using
a browser addon a "ui switch" can be easily achieved.
 2021-02-14 01:46:13 +01:00
Eike Kettner
dd935454c9 First version of new ui based on tailwind 
This drops fomantic-ui as css toolkit and introduces tailwindcss. With
tailwind there are no predefined components, but it's very easy to
create those. So customizing the look&feel is much simpler, most of
the time no additional css is needed.

This requires a complete rewrite of the markup + styles. Luckily all
logic can be kept as is. The now old ui is not removed, it is still
available by using a request header `Docspell-Ui` with a value of `1`
for the old ui and `2` for the new ui.

Another addition is "dev mode", where docspell serves assets with a
no-cache header, to disable browser caching. This makes developing a
lot easier.
 2021-02-14 01:46:13 +01:00
Eike Kettner
442b76c5af Setup tailwind with docspell build 2021-02-03 21:35:15 +01:00
Eike Kettner
62f596be35 Set version to 0.20.0-SNAPSHOT 2021-01-25 10:19:00 +01:00
Eike Kettner
73b54addc5 Set version to 0.19.0 2021-01-25 09:21:16 +01:00
Eike Kettner
394aeeccb6 Introduce a sql literal and constants in query builder 
The h2 jdbc driver could not translate the union query in QCollective
when the `kind` was set via a constant value. Using literals works
here. Renamed the corresponding elements in the query builder.
 2021-01-25 00:18:24 +01:00
mergify[bot]
6cc9c159d6
Merge pull request #590 from eikek/scan-mailbox-filter 
Refactor scan mailbox form and add flag for post-processing
 2021-01-24 01:06:51 +00:00
Eike Kettner
96612e0e59 Refactor scan mailbox form and add flag for post-processing 
Mails are filtered once by using an imap search and then by some globs
to filter files and subjects. Imap can search by subject via a
string-contains, but not via globs or patterns (afaik). The subject
filter is applied to all downloaded mail headers. Now for post
processing (moving to some target folder or deleting), it can be
chosen to post-process all "seen" mails or only those that matched all
filters.
 2021-01-24 01:46:31 +01:00
Eike Kettner
1b66e2af5c Fix classifier_settings table 2021-01-23 21:30:26 +01:00
Eike Kettner
c7e850116f Make the text length limit optional 2021-01-22 23:06:50 +01:00
mergify[bot]
8dd1672c8c
Merge pull request #583 from eikek/fix-baseurl-setting 
Render baseurl without trailing slash
 2021-01-21 23:44:14 +00:00
mergify[bot]
38e0a50942
Merge pull request #582 from eikek/delete-org-fix 
Fix deleting organization
 2021-01-21 22:56:56 +00:00
Eike Kettner
0ec620fcf0 Render baseurl without trailing slash 
The webapp expects it like this currently, because the url is only a
string.
 2021-01-21 21:42:08 +01:00
Eike Kettner
f4a03e7c69 Fix deleting organization 
The foreign key in person must be resetted.
 2021-01-21 21:27:02 +01:00
Eike Kettner
4cba96f390 Always return classifier results as suggestion 
The classifier results are spliced into the suggestion list at second
place. When linking they are only used if nlp didn't find anything.
 2021-01-21 21:05:28 +01:00
Eike Kettner
9957c3267e Add constraints from config to classifier training 
For large and/or many documents, training the classifier can lead to
OOM errors. Some limits have been set by default.
 2021-01-21 17:46:39 +01:00
Eike Kettner
363cf5aef0 Quote names in sql changesets 2021-01-21 00:22:58 +01:00
Eike Kettner
38387e00a0 Fix mariadb migration 2021-01-21 00:22:53 +01:00
Eike Kettner
a6c31be22f Update documentation 2021-01-20 22:47:15 +01:00
Eike Kettner
85ddc61d9d Move date proposal setting to nlp config 2021-01-20 19:17:29 +01:00
Eike Kettner
5d366c3bd6 Make labels in classifier settings more clear 2021-01-20 01:05:59 +01:00
Eike Kettner
b12d965223 Improve logging 2021-01-20 00:40:58 +01:00
Eike Kettner
27c24c128d Store tags guessed with classifier in database 2021-01-20 00:30:40 +01:00
Eike Kettner
9d83cb7fe4 Store item based proposals in separate table 
Classifier don't work on each attachment, but on all. So the results
must not be stored at an attachment. This reverts some previous
changes to put the classifier results for item entities into its own
table.
 2021-01-19 23:48:09 +01:00
Eike Kettner
3ff9284a64 Return classifier results as suggestions 2021-01-19 23:13:51 +01:00
Eike Kettner
75573c905e Use classifier results as fallback when linking proposed metadata 2021-01-19 23:13:34 +01:00
Eike Kettner
8455d1badf Lookup results from classifier 
The model may be out of date, data may change. Then it should be
looked up to fetch the id to be compatible with next stages.
 2021-01-19 22:56:01 +01:00
Eike Kettner
1cd3441462 Run classifier for item entities (concerned, correspondent) 
Store the results separately from nlp results in attachment metadata.
 2021-01-19 22:08:29 +01:00
Eike Kettner
d124f0c1a9 Rename db changeset 
It's not just a fix, but adds new things
 2021-01-19 22:08:29 +01:00
Eike Kettner
5c487ef7a9 Refactor running classifier in text analysis 2021-01-19 21:30:02 +01:00
Eike Kettner
99dcaae66b Learn classifiers for item entities 
Learns classifiers for concerned and correspondent entities. This can
be used as an alternative to or after nlp.
 2021-01-19 20:54:47 +01:00
Eike Kettner
a6f29153c4 Control what tag categories to use for auto-tagging 2021-01-19 01:20:13 +01:00
Eike Kettner
cce8878898 Exclude tags w/o category from classifying; remove obsolete models 2021-01-18 21:51:49 +01:00
Eike Kettner
3e28ce1254 Add the sql concat function to query builder 2021-01-18 21:51:45 +01:00
Eike Kettner
249f9e6e2a Extend guessing tags to all tag categories 2021-01-18 21:51:45 +01:00
Eike Kettner
3f75af0807 Add 9 more lanugages to the list of document lanugages 2021-01-18 17:41:40 +01:00
Eike Kettner
94bb18c152 Refactor solr language fields 2021-01-18 17:41:40 +01:00
Eike Kettner
26dff18ae0 Add spanish as an example 
Adding a new language without nlp requires now only to fill out the
pieces:

- define a list of month names to support date recognition
- add it to joex' dockerfile to be available for tesseract
- update the solr migration/field definitions
- update the elm file so it shows up on the client
 2021-01-18 17:41:40 +01:00
Eike Kettner
360cad3304 Refactoring solr/fts migration 
When re-indexing everything, skip intermediate populating the index
and do this as the very last step.

Parameterize adding new fields by their language.
 2021-01-18 17:41:40 +01:00
Eike Kettner
ff121d462c Disable memory intensive tests on travis 2021-01-18 17:41:40 +01:00
Eike Kettner
f01646aeb5 Reorganize nlp pipeline and add nlp-unsupported language italian 
Improves and reorganizes how nlp pipelines are setup. Now users can
choose from many options, depending on their hardware and usage
scenario.

This is the base to use more languages without depending on what
stanford-nlp supports. Support then is involves to text extraction and
simple regex-ner processing.
 2021-01-18 17:41:40 +01:00
Eike Kettner
a70e9ab614 Store used language for processing on attachmentmeta 
Issue: #570
 2021-01-17 22:56:33 +01:00
Eike Kettner
6cf3f9be5a Fix joex version endpoint in spec 2021-01-17 22:56:33 +01:00
Eike Kettner
aa937797be Choose nlp mode in config file 2021-01-17 22:56:33 +01:00
Eike Kettner
54a09861c4 Use model cache with basic annotator 2021-01-17 22:56:33 +01:00
Eike Kettner
a77f67d73a Make pipeline cache generic to be used with BasicCRFAnnotator 2021-01-17 22:56:33 +01:00
Eike Kettner
4462ebae0f Resurrect the basic ner classifier 2021-01-17 22:56:33 +01:00
Eike Kettner
a699e87304 Separate ner from classification 2021-01-17 22:56:33 +01:00
Eike Kettner
f02f15e5bd Move blocker into constructor of text analyser 2021-01-17 22:56:33 +01:00
Eike Kettner
b2b8ad625a scalafmt 2021-01-17 20:11:58 +01:00
Eike Kettner
f0f0e6e0d4 Search for categories case-insensitive 
The string was already lowercased, but the comparison was not.

Fixes #568
 2021-01-17 20:10:24 +01:00
Eike Kettner
623a61dbb6 Introduce a lowerEq operator to the query builder 2021-01-17 20:10:00 +01:00
Eike Kettner
54bd75e99e Set version to 0.19.0-SNAPSHOT 2021-01-11 23:27:47 +01:00
Eike Kettner
0d1b55a205 Set version to 0.18.0 2021-01-11 22:39:40 +01:00
Eike Kettner
d77b5855e4 Set default pool-size to 1 2021-01-11 22:30:59 +01:00
Eike Kettner
38ae7a9027 Make source a quick link on card and detail 2021-01-11 21:37:36 +01:00
Eike Kettner
33458766fe Correcty reset search menu when clicking on custom-field quick link 2021-01-11 14:03:23 +01:00
Eike Kettner
7beda302b2 Fix and improve tag search menu 
Show also "empty tags", where the count is 0. Before only tags with a
count > 0 were displayed. When searching this is fine, but when using
drag&drop to attach tags to items, it is good to see all. They can be
hidden via a button.

The tags are now ordered by their count descending, but regarding to
the overall count – not the current view. Otherwise the tags are
reordered when clicking on them, which is confusing. Also it then
shows the "more important" (most used) tags first, even when the
result is a subset.

A fix was made related to updating the menu. When coming back from
the detail view where a tag with prior count=0 was associated, the
menu didn't show it, because it relied on a previous state, where this
tag were not included.
 2021-01-11 13:01:38 +01:00
Eike Kettner
3fccc3df39 Return all tags in search stats result 
Before only tags with a count > 0 were included. Now those that have
not attached to any item are returned as well.
 2021-01-11 12:13:13 +01:00
Eike Kettner
75986c461f Fix ner date label boundary reporting 2021-01-10 09:10:39 +01:00
Eike Kettner
fb05e997ab Provide multiple date suggestions for English 
Issue: #561
 2021-01-10 09:02:26 +01:00
Eike Kettner
bddafa7d28 Fix looping over already seen mails when they are skipped 
When skipping mails due to a filter, it must still enter the
post-handling step. Otherwise it will be seen again on next run.

Issue: #551
 2021-01-09 15:07:18 +01:00
Eike Kettner
d712f8303d Make glob matching case-insensitive by default 2021-01-09 13:23:15 +01:00
Eike Kettner
cbca4d234f Fix scrolling to card 
That was broken due to the independent scroll in commit #bcb1b8.
 2021-01-09 02:00:01 +01:00
Eike Kettner
cef1c38cc4 Restrict height of job output 2021-01-09 01:49:55 +01:00
Eike Kettner
0abd7dea10 Fix scrolling to top in detail view 2021-01-09 01:16:59 +01:00
Eike Kettner
c0d7aba5d5 Improve selecting attachments of an item 
Use a list of small thumbnails instead of just names.

Closes: #396
 2021-01-09 01:16:59 +01:00
Eike Kettner
9bc2084499 Allow to click on custom fields in detail view 
Closes: #514
 2021-01-09 01:16:59 +01:00
Eike Kettner
48d182667d Harmonize login and register page 2021-01-09 01:16:59 +01:00
Eike Kettner
752c8f9be2 Show new-invite as normal page 2021-01-09 01:16:59 +01:00
Eike Kettner
bcb1b87fc0 Enable independent scrolling of search menu and list 
Fixes some other minor css issues.

Closes: #541
 2021-01-09 01:16:59 +01:00
Eike Kettner
3c12e3678f Allow to search for * in custom date fields 
This requires to pass the raw input through to the caller.

Closes: #550
 2021-01-09 01:16:59 +01:00
Bo Jeanes
14dacaa837
Fix typo 2021-01-09 10:41:53 +11:00
Eike Kettner
716252721c Fix cache clearing 
It must be cancelled when obtaining a pipeline.
 2021-01-07 23:31:01 +01:00
Eike Kettner
a670bbb6c2 Make idle interval when clearing nlp cache configurable 2021-01-06 23:03:00 +01:00
Eike Kettner
73a9572835 Poc for clearing stanford pipeline after some idle time 2021-01-05 23:56:20 +01:00
Eike Kettner
b08e88cd69 Add (inofficial) routes to get system information 2021-01-05 20:54:53 +01:00
Eike Kettner
30df887934 Sort custom field options in dropdown 2021-01-05 18:04:54 +01:00
Eike Kettner
668abf2140 Add a reset-password admin route 2021-01-04 20:59:31 +01:00
Eike Kettner
2a172ce720 Remove fulltext recreate-key config value 
It's now in the admin routes, protected by the
`admin-endpoint.secret`.
 2021-01-04 15:18:02 +01:00
Eike Kettner
306f064ad9 Create a place for admin routes 
And move re-creation of fulltext index in this place.
 2021-01-04 15:18:02 +01:00
Eike Kettner
0cfd8974d3 Add a flag to imap settings to enable/disable oauth2 scheme 2021-01-04 11:03:04 +01:00
mergify[bot]
84d160aa38
Merge pull request #534 from eikek/duplicate-check 
Fixing find-by-checksum with exclusions
 2021-01-03 11:55:28 +00:00
Eike Kettner
95fd386c14 Fixing find-by-checksum with exclusions 
The NOT-IN query must check for null separately, as everything with
null evaluates to false in sql resulting in not finding existing
duplicates.
 2021-01-03 12:29:03 +01:00
Eike Kettner
40b093ae72 Fix selecting imap connection in scan mailbox settings 
Issue: #524
 2021-01-03 10:31:36 +01:00
Eike Kettner
611e480eb4 Use more prominent log line to indicate start of processing 
Issue: #530
 2021-01-02 21:47:54 +01:00
Eike Kettner
97dfcece97 Fix duplicate check on restarts 
Issue: #530
 2021-01-02 21:18:05 +01:00
Eike Kettner
2b8ef3c2f7 Allow to hide search stats in user settings 2021-01-02 10:20:18 +01:00
Eike Kettner
ae57881f2d Sort custom fields in views 2021-01-02 10:09:05 +01:00
Eike Kettner
70983f115c Fix custom field summary table for mobile view 2021-01-02 09:55:12 +01:00
Eike Kettner
a9ed0364d2 Fix linking guessed tags 
Since tag names are lower-cased the search must happen lower-cased, too.
 2021-01-02 01:30:31 +01:00
Eike Kettner
36858da624 Fix search condition for empty items set 2020-12-17 23:07:04 +01:00
Eike Kettner
4ec133b0b9 Remove unused imports 2020-12-17 23:06:58 +01:00
Eike Kettner
69f57d1eb1 Replace empty hrefs with a href # 2020-12-17 21:15:33 +01:00
Eike Kettner
c9d4e8ec46 Fix selecting items in multi-edit mode 
For some to me unknown reason, changing the dom slightly (removing
hidden elements), resulted in a different event dispatching. The cards
while being attached to an event would reload the page as if the event
is propagated. This happned by commit #8d7b3c7d in Home/View.elm.
Adding the hidden nodes back into the dom, "fixed" it.

This change now gives a better fix in assuring that every anchor has
either a sensible `href` or an event and a `href #`.
 2020-12-17 21:02:23 +01:00
Eike Kettner
6346bf6a34 Add summary for fulltext searches 2020-12-17 00:11:33 +01:00
Eike Kettner
8d7b3c7d74 Show custom field summary above results 2020-12-16 22:06:34 +01:00
Eike Kettner
b66738b4c3 Add folder count to search menu 2020-12-16 19:20:30 +01:00
Eike Kettner
8fba637ebe Add folder counts to search summary 2020-12-16 01:14:27 +01:00
Eike Kettner
a995ea8729 Update tag counts in search menu 2020-12-16 00:56:12 +01:00
Eike Kettner
80e23d1c84 Add a route to get search summary 2020-12-15 23:37:02 +01:00
Eike Kettner
77627534bc Improve on basic search summary 2020-12-15 23:37:02 +01:00
Eike Kettner
f3855628d5 Extend query builder with more functions 2020-12-15 23:34:12 +01:00
Eike Kettner
4ca6dfccae Get basic search summary 2020-12-15 23:10:13 +01:00
Eike Kettner
56d6d2e2ac Allow changing more parts of a select 2020-12-15 22:12:44 +01:00
Eike Kettner
f1c4b4adb0 Extract find-item query condition 2020-12-15 21:03:47 +01:00
Eike Kettner
2dff686fa0 Introduce unit condition 2020-12-15 21:03:47 +01:00
Eike Kettner
80406cabc2 Refactoring some code into separate files 2020-12-15 21:03:47 +01:00
Eike Kettner
278b1c22c9 Remove old code 2020-12-15 21:03:46 +01:00
Eike Kettner
2cecd01837 Convert rest of QItem 2020-12-15 21:03:46 +01:00
Eike Kettner
d1606d6f16 Remove old commented code 2020-12-15 21:03:46 +01:00
Eike Kettner
266fec9eb5 Convert find items query 2020-12-15 21:03:46 +01:00
Eike Kettner
5e2c5d2a50 Extends query builder 2020-12-15 21:03:46 +01:00
Eike Kettner
35c62049f5 Start converting QItem 2020-12-15 21:03:46 +01:00
Eike Kettner
a355767fdb Convert all query libs besides QItem 2020-12-15 21:03:46 +01:00
Eike Kettner
fd6d09587d Convert more records 2020-12-15 21:03:46 +01:00
Eike Kettner
613696539f Minor refactorings 2020-12-15 21:03:46 +01:00
Eike Kettner
d6f28d3eca Convert folder 2020-12-15 21:03:46 +01:00
Eike Kettner
87eb8c7f55 Convert more records 2020-12-15 21:03:46 +01:00
Eike Kettner
e3f6892abd Convert job record 2020-12-15 21:03:46 +01:00
Eike Kettner
1aa1f4367e Convert periodic tasks 2020-12-15 21:03:46 +01:00
Eike Kettner
3cef932ccd Convert more records 2020-12-15 21:03:46 +01:00
Eike Kettner
fe4815c737 Convert RSentMail 2020-12-15 21:03:46 +01:00
Eike Kettner
5cbf0d5602 Convert more records 2020-12-15 21:03:46 +01:00
Eike Kettner
10b49fccf8 Converting user and userimap records 2020-12-15 21:03:46 +01:00
Eike Kettner
c5c7f7ed3b Convert equipment record 2020-12-15 21:03:46 +01:00
Eike Kettner
adee496b77 Convert source record 2020-12-15 21:03:46 +01:00
Eike Kettner
2dbb1db2fd Initial outline for a simple query builder 2020-12-15 21:03:46 +01:00
Eike Kettner
b338f18e98 Remove redundant fileCount from search result 2020-12-15 21:03:46 +01:00
Eike Kettner
21cd778c37 Merge branch 'fix-0.17' into master 2020-12-15 20:45:39 +01:00
Eike Kettner
14428c0ead Set version to 0.17.1 2020-12-15 20:25:59 +01:00
Eike Kettner
a987ede7c9 Avoid server roundtrip when switching pages 
When changing pages in webui a server request should not happen. This
makes the webapp snappy again.

Issue: #508
 2020-12-15 20:22:12 +01:00
Eike Kettner
ae0a543101 Set next version to 0.18.0-SNAPSHOT 2020-12-14 14:57:40 +01:00
Eike Kettner
398f969207 Set version to 0.17.0 2020-12-14 14:34:27 +01:00
Eike Kettner
27d087b14c Fix foreign key constraints 2020-12-14 14:34:22 +01:00
Eike Kettner
7e42426edc Fix page routing with remember me 2020-12-11 01:16:57 +01:00
Eike Kettner
c00714ab61 Don't redirect to login for open pages 2020-12-08 21:12:09 +01:00
Eike Kettner
f55566388a Fix endless reload of login page 2020-12-06 00:58:15 +01:00
Eike Kettner
66bd4f53cb Fix person-org relation info 2020-12-05 22:44:19 +01:00
Eike Kettner
2aed7ba142 Simplify search bar and search menu 
The functionality of the search bar is now in the search menu, too.
The search menu shows one input field for "textual search", which is
either the fulltext search (if enabled) or a basic search in various
names.
 2020-12-05 22:38:27 +01:00
Eike Kettner
5882405f30 Search index if search object only contains this field 2020-12-05 02:59:57 +01:00
Eike Kettner
0b6f965fcb Fix rememberme for missing local storage 2020-12-04 22:57:21 +01:00
Eike Kettner
0337391b20 Change remember period to 30 days 2020-12-04 20:30:25 +01:00
Eike Kettner
f5ae389eea Cleanup remember-me tokens periodically 2020-12-04 17:59:25 +01:00
Eike Kettner
a0642905db Use remember-me cookie if present 2020-12-04 17:59:25 +01:00
Eike Kettner
c10c1fad72 Prepare remember-me authentication variant 2020-12-04 17:59:25 +01:00
Eike Kettner
290989f67f Reorder correspondent person suggestion based on org relationship 2020-12-01 23:39:45 +01:00
Eike Kettner
d4470ab5fd Restrict person dropdown options to the associated organization 2020-12-01 23:39:45 +01:00
Eike Kettner
a6dd71af9f Set an organization to a person in webapp 2020-12-01 23:39:45 +01:00
Eike Kettner
2e28c8e57b Harmonize table views 2020-12-01 23:39:45 +01:00
Eike Kettner
fc2668feee Allow to connect a person to an organization 2020-12-01 23:39:45 +01:00
Eike Kettner
346d23928d Allow to search in source names in webui 2020-11-30 14:26:10 +01:00
Eike Kettner
0ee8ff66d5 Allow to search by source name 2020-11-30 14:07:45 +01:00
Eike Kettner
bdc49aae98 Let user change template for card title and subtitle 2020-11-29 23:36:25 +01:00
Eike Kettner
81a136d915 Use a template for rendering title and subtitle of the item card 
Introduces `ItemTemplate` to conveniently create strings given an
item.
 2020-11-29 23:36:20 +01:00
Eike Kettner
45e4035e07 Set version to 0.17.0-SNAPSHOT 2020-11-28 18:13:50 +01:00
Eike Kettner
4682bf4a0e Set version to 0.16.0 2020-11-28 17:55:40 +01:00
mergify[bot]
7ee69f7a57
Merge pull request #481 from eikek/reload-fix 
Reload fix
 2020-11-28 01:23:26 +00:00
Eike Kettner
75055ba34b Fix help link menu item 2020-11-28 01:48:58 +01:00
Eike Kettner
7052bc6b8e Add cc and bcc to item mail 2020-11-28 01:36:59 +01:00
Eike Kettner
d62c4a5a72 Fix updating search view 
For example, when content-search was activated the view was not
correctly updated after deleting or drag-and-drop.
 2020-11-28 00:51:16 +01:00
Tammo van Lessen
e9347176bd
Fixes an off-by-one classic to also accept dates in January 2020-11-28 00:43:35 +01:00
Eike Kettner
32c9113bd4 Remove unused code 2020-11-28 00:00:10 +01:00
Eike Kettner
1091a4fbdc Show concerning/correspond person exclusively in dropdowns 
Now a person that is not marked as "concerning only" is treated as
"correspondent only". The options of all dropdown fields are now
restricted to those subsets.

Issue: #332
 2020-11-27 23:13:37 +01:00
Eike Kettner
3fabe0a582 Update to Scala 2.13.4 2020-11-27 20:26:24 +01:00
Eike Kettner
7cc290135a Show currently logged in user 
Closes: #329
 2020-11-26 23:53:55 +01:00
Eike Kettner
c87163052b Allow to choose from existing tag categories 
Also fixes a ui problem with a too long dropdown menu in the small
modal popup when adding tags or custom fields in item detail view.

Issue: #331
 2020-11-26 23:53:50 +01:00
Eike Kettner
0919eec3c2 Improve field query and fix mariadb's pickiness with parens 
If no query is given, don't search with `like '%'`. MariaDB doesn't
want parens around columns in the GROUP BY clause.
 2020-11-25 21:08:49 +01:00
Eike Kettner
52c6659f9f Add missing schema migrations for custom fields 2020-11-25 21:08:49 +01:00
Eike Kettner
41b26fbbd6 Don't hide field dropdown when no fields are defined 2020-11-24 23:38:22 +01:00
Eike Kettner
a18ac17f0c Search with wildcards for custom fields 2020-11-24 21:44:27 +01:00
Eike Kettner
c389c8fe66 Change field-type into field-format in ui 2020-11-24 21:44:27 +01:00
mergify[bot]
436022ceee
Merge pull request #466 from eikek/wildcard-number-search 
Allow to search with wildcard in a number custom field
 2020-11-24 00:32:02 +00:00
Eike Kettner
c1c92f0932 Add language to upload page 2020-11-23 21:12:45 +01:00
Eike Kettner
2384af1347 Fix centering dropzone title 2020-11-23 21:12:33 +01:00
Eike Kettner
7fee52f6f2 Add language to scan-mailbox-form 2020-11-23 21:00:19 +01:00
Eike Kettner
5fe532001b Allow to specify document lanugage with the request 2020-11-23 20:49:01 +01:00
Eike Kettner
29a5419072 Allow to search with wildcard in a number custom field 2020-11-23 13:44:49 +01:00
Eike Kettner
1ef035f061 Fix hiding tags/fields when corresponding setting exists 2020-11-23 11:20:59 +01:00
Eike Kettner
9bea0298ad Allow to query custom field values with wildcards 2020-11-23 10:59:13 +01:00
Eike Kettner
473985c80e Filter empty custom field values in webui 2020-11-23 10:47:45 +01:00
Eike Kettner
7712e02d2d Don't allow empty custom field values 2020-11-23 10:38:59 +01:00
Eike Kettner
f8c6d183ed Don't scroll in preview image 2020-11-23 10:30:24 +01:00
Eike Kettner
6d22bac720 Display custom field values on item card 2020-11-23 10:23:37 +01:00
Eike Kettner
7b7f1e4d6d Return custom field values with search results 2020-11-23 10:23:25 +01:00
Eike Kettner
44459aecd0 Fix initialising date fields 2020-11-23 09:43:12 +01:00
Eike Kettner
cdcc8210fe Throttle customfield update requests 2020-11-23 09:27:05 +01:00
Eike Kettner
7026852123 Indicate saving custom field values 2020-11-23 00:16:50 +01:00
Eike Kettner
bcdb2fc0fe Show custom field values in item detail header 2020-11-22 23:14:51 +01:00
Eike Kettner
a2e0c23644 Amend search form for custom fields 2020-11-22 21:41:09 +01:00
Eike Kettner
23b343649c Fix and enhance custom-multi-input field 2020-11-22 21:41:09 +01:00
Eike Kettner
066c856981 Allow to search for custom field values 2020-11-22 21:41:09 +01:00
Eike Kettner
c5ab663091 Allow a comma and a point for money values 2020-11-22 21:41:09 +01:00
Eike Kettner
bb19e02c66 Ui improvements 
- don't show custom fields in edit menu if there are none. This
reduces load of ui elements. The first custom field must be created in
manage-data page.

- Add more validation to the money type
 2020-11-22 21:41:09 +01:00
Eike Kettner
ff30ed5558 Add custom fields to multi-edit form 2020-11-22 21:41:09 +01:00
Eike Kettner
76647d132f Show custom field values in detail view 2020-11-22 21:41:09 +01:00
Eike Kettner
1aefff37aa Return custom field values with item details 2020-11-22 21:41:09 +01:00
Eike Kettner
1ee36cef8f Add fields when clicking in the dropdown 
Remove the additional button
 2020-11-22 21:41:09 +01:00
Eike Kettner
cc6db61a3a Allow to create fields when editing items 2020-11-22 21:41:09 +01:00
Eike Kettner
af1cca7d83 Fix condition for deleting custom field value 2020-11-22 21:41:09 +01:00
Eike Kettner
4059ef31c1 Allow providing values for custom fields in item detail 2020-11-22 21:41:09 +01:00
Eike Kettner
e90f65f941 Manage custom fields in webui 2020-11-22 21:41:09 +01:00
Eike Kettner
8d35d100d6 Change custom fields for multiple items 2020-11-22 21:41:09 +01:00
Eike Kettner
93295d63a5 Change custom field values for a single item 2020-11-22 21:41:09 +01:00
Eike Kettner
62313ab03a Add and change custom fields 2020-11-22 21:41:09 +01:00
Eike Kettner
248ad04dd0 Prepare custom fields 2020-11-22 21:41:09 +01:00
Eike Kettner
417581845b Fix openapi datastructures 2020-11-16 09:53:44 +01:00
Eike Kettner
1d6cdac92d Set version to 0.16.0-SNAPSHOT 2020-11-15 13:11:39 +01:00
Eike Kettner
79510529c2 Set version to 0.15.0 2020-11-15 12:51:25 +01:00
Eike Kettner
f8634a5098 Use js pdf renderer for file view when settings tell so 2020-11-15 01:13:13 +01:00
Eike Kettner
063c9895dd Fix table layout and notification form 
The styling is broken if th elements are outside a tr. The
notification form had the same problem as the scan-mailbox form
reported in #382
 2020-11-15 00:09:29 +01:00
Eike Kettner
46dbdd2625 Fix css when hovering a blue card 2020-11-14 13:42:30 +01:00
Eike Kettner
b98edba9e5 Click on tag in item detail to switch to search 2020-11-14 13:16:35 +01:00
Eike Kettner
fe8c122968 Switch to search by clicking on correspondent/concerned in detail 2020-11-14 12:57:38 +01:00
Eike Kettner
cf578a88d3 Make item detail header a link list 2020-11-14 12:11:04 +01:00
Eike Kettner
e2545de347 Make item-detail and item-light consistent in field naming 
This changes the public structure for an "item-light" as returned in
search results. It makes unifying things easier if both names are the
same.
 2020-11-14 11:37:50 +01:00
Eike Kettner
23391ef41c Refactor link-target in a separate module 2020-11-14 11:32:19 +01:00
Eike Kettner
4b700fb249 Indicate on a card that multi-select mode is active 2020-11-14 11:03:22 +01:00
Eike Kettner
ee6b54006c Fix reset search for content-only and provide it on top bar 2020-11-14 10:41:20 +01:00
Eike Kettner
795cd8bc80 Set search when clicking on parts of the item card 
When user clicks on correspondent etc. on an item card, the search is
set to this value.
 2020-11-14 01:45:10 +01:00
Eike Kettner
faf97222ea Make resetting search form more efficient 
Instead of reloading the whole menu, only empty the fields.
 2020-11-14 01:45:10 +01:00
Eike Kettner
b1d0ae0cc8 Expose clicks on item cards 2020-11-14 01:45:10 +01:00
Eike Kettner
ec5b822e09 Add subject-filter to scan-mailbox-form 2020-11-13 23:15:20 +01:00
Eike Kettner
5034e12bec Add a subject filter to scan-mailbox args 2020-11-13 23:15:20 +01:00
Eike Kettner
2f9e0a8214 Use more prominent indication when an item is selected 
Downside to this approach is that the item card is not accessible when
selected.
 2020-11-13 01:54:19 +01:00
Eike Kettner
8950cc4dc9 Improve item card 2020-11-13 01:18:45 +01:00
mergify[bot]
e5ce1fd45f
Merge pull request #437 from eikek/upload-improvements 
Upload improvements
 2020-11-12 22:58:08 +00:00
Eike Kettner
75c1c5b7f4 Add tags and file-filter settings to scan-mailbox form 2020-11-12 22:37:37 +01:00
Eike Kettner
04ba14f802 Amend source form with tags and file-filter 
Allow to define tags and a file filter per source.
 2020-11-12 22:37:28 +01:00
Eike Kettner
4fd6e02ec0 Improve glob and filter archive entries 2020-11-11 21:01:23 +01:00
Eike Kettner
27eb5d70de Apply given tags in processing step 
Issue: #346
 2020-11-11 21:01:23 +01:00
Eike Kettner
55a6f7aaf6 Add more properties to upload meta data 2020-11-11 21:01:23 +01:00
Eike Kettner
a21a97f7d5 Add a simple glob data type 2020-11-10 22:44:08 +01:00
Eike Kettner
746e04c624 Improve logging when creating preview images 2020-11-10 22:25:46 +01:00
Eike Kettner
ed26fe226b Split item card it several functions 2020-11-10 22:25:25 +01:00
Eike Kettner
976aa75710 Move card size definition in css and fix height bug 
For very tall images (sometimes shopping receipts), the height must be
restricted, too.
 2020-11-09 21:36:17 +01:00
Eike Kettner
10305bc82d Minor improvements 2020-11-09 21:16:53 +01:00
Eike Kettner
29455d638c Add startup task to find page counts of existing files 2020-11-09 20:35:35 +01:00
Eike Kettner
de00b46e5d Move searchbar to the right 2020-11-09 17:12:06 +01:00
Eike Kettner
89646ef3f6 Hide number of pages, if item is only one file with one page 2020-11-09 17:06:44 +01:00
Eike Kettner
7a14b05ea7 Enhance item card displaying current file and number of pages 2020-11-09 17:04:56 +01:00
Eike Kettner
848c245db6 Change the card link to only use the main content 
The card is no longer a link itself. The main target is moved to be
the content (the area containing the title and tags). This is in
preparation of upcoming changes: if the whole card is a link, it
cannot contain other links, due to a restriction by html. Later a card
may have more links to provide.
 2020-11-09 14:36:58 +01:00
Eike Kettner
67e8994aec Use attachment preview urls 
This changes the preview urls to use the concrete attachment ids. This
way browsers have it easier to switch the preview image when the
attachment position is changed.
 2020-11-09 14:29:52 +01:00
Eike Kettner
8c08bf233d Amend search results with attachment info 
This uses again another query per item to retrieve some information
about each attachment already in the search results.
 2020-11-09 14:24:28 +01:00
Eike Kettner
a77f34b7ba Add a processing step to retrieve page counts 2020-11-09 11:08:24 +01:00
Eike Kettner
5906c705c9 Allow the user to choose between 3 preview sizes 2020-11-09 09:40:21 +01:00
Eike Kettner
8c8788bc69 Provide fallback image for previews 2020-11-09 09:00:03 +01:00
Eike Kettner
d4bbb936b6 Count preview image sizes in insight data 2020-11-09 09:00:03 +01:00
Eike Kettner
30682fbecc Document the re-generate all previews endpoint 2020-11-09 09:00:03 +01:00
Eike Kettner
f4e50c5229 Provide endpoints to submit tasks to re-generate previews 
The scaling factor can be given in the config file. When this changes,
images can be regenerated via POSTing to certain endpoints. It is
possible to regenerate just one attachment preview or all within a
collective.
 2020-11-09 09:00:02 +01:00
Eike Kettner
6037b54959 Don't fail processing if generating preview fails 2020-11-09 00:05:11 +01:00
Eike Kettner
cf6e63785d Fix potential index-out-of-bounds error in classifier 
The stanford library expects a non-empty text.
 2020-11-09 00:04:51 +01:00
Eike Kettner
709848244c Create tasks to generate all previews 
There is a task to generate preview images per attachment. It can
either add them (if not present yet) or overwrite them (e.g. some
config has changed).

There is a task that selects all attachments without previews and
submits a task to create it. This is submitted on start automatically
to generate previews for all existing attachments.
 2020-11-08 23:46:02 +01:00
Eike Kettner
eede194352 Fix deleting preview files 2020-11-08 21:27:55 +01:00
Eike Kettner
2c96590aad First ui view of preview images for items 
Users can choose to not show them via ui settings
 2020-11-08 15:12:56 +01:00
Eike Kettner
7ba6baf6f0 Make preview image smaller 2020-11-08 15:12:56 +01:00
Eike Kettner
757ad31165 Add a route to get the item preview 
This is the first available preview of an attachment wrt position. If
all attachments have a preview image, the preview of the first
attachment is returned.
 2020-11-08 15:12:56 +01:00
Eike Kettner
8cc89fd3b7 Move handling binary responses to a shared space 2020-11-08 15:12:56 +01:00
Eike Kettner
d376ef3ef1 Add simple route to get the preview image 2020-11-08 13:33:39 +01:00
Eike Kettner
6db5c39d78 Fix converted filename 
Mark it by default with a string from the config file.

Issue: 397
 2020-11-08 09:45:03 +01:00
Eike Kettner
ef7cb4e779 Create a preview image of all files during processing 2020-11-08 01:25:59 +01:00
Eike Kettner
0841a33ae3 Add a table to hold the preview files 2020-11-08 01:25:38 +01:00
Eike Kettner
350a271b22 Add simple pdf page preview function 2020-11-08 01:25:14 +01:00
Eike Kettner
59dfae6a49 Introduce fomantic-ui, replacing semantic-ui 
Replaced semantic-ui with the drop-in replacement fomantic-ui [0]
which is a maintained fork. The fomantic-ui used here is a custom
build [1] of the less-version _without_ google-fonts (css-only). The
javascript part of fomantic-ui is not used, and also jquery could be
dropped now.

[0] https://fomantic-ui.com
[1] https://github.com/eikek/fomantic-slim-default

Issue: #349
 2020-11-05 22:44:10 +01:00
Eike Kettner
47e780d35f Set version to 0.15.0-SNAPSHOT 2020-11-01 21:00:19 +01:00
Eike Kettner
1a1104c387 Set version to 0.14.0 2020-11-01 20:25:07 +01:00
Eike Kettner
a965605a9e Improve tag multi-edit 2020-10-31 14:42:17 +01:00
Eike Kettner
f4c79c72ae Allow to remove tags from multiple items 2020-10-31 14:42:17 +01:00
Eike Kettner
6a16d8afc7 Allow to confirm/unconfirm multiple items 2020-10-29 23:15:38 +01:00
Eike Kettner
ab50792879 Change icon for hiding/showing search menu 2020-10-29 23:02:08 +01:00
Eike Kettner
0461cfefe7 Fix sql error for mariadb <10.4 
MariaDB below 10.4 doesn't support parentheses around selects for
`intersect` and `union`.

https://mariadb.com/kb/en/intersect/#parentheses

Fixes #404
 2020-10-28 22:54:51 +01:00
Eike Kettner
509823404e Try improve search bar/menu switch 
Adressing some parts of #395
 2020-10-28 21:55:42 +01:00
mergify[bot]
96d7eb5c97
Merge pull request #401 from eikek/fix-replace-cards 
Preserve context highlighting from full-text search
 2020-10-27 23:57:12 +00:00
Eike Kettner
c6b8355902 Preserve context highlighting from full-text search 
When replacing changed cards the fulltext highlighting should be
preserved from the original item.
 2020-10-27 22:51:11 +01:00
Eike Kettner
183bcf5bec Add ui-settings flag for side menus 2020-10-27 22:35:52 +01:00
Eike Kettner
83b41b027f Change default position for notes to bottom 2020-10-27 20:43:58 +01:00
mergify[bot]
2ad2f3be57
Merge pull request #392 from eikek/fix-process-timeout 
Fix process timeout
 2020-10-26 23:48:27 +00:00
Eike Kettner
ab1139523a Let the convert-all task retry when pdf conversion fails 2020-10-26 23:39:26 +01:00
Eike Kettner
b59696a9d3 Make sure to only remove/retry items in premature states 2020-10-26 23:39:26 +01:00
Eike Kettner
0114bb4d72 Use source name from config file for integration endpoint uploads 
Fixes: #389
 2020-10-26 22:37:30 +01:00
Eike Kettner
dd89e05cc2 Convert exceptions when converting to pdf into an error result 
The file processing tries pdf conversion once and keeps going if it
fails. Some errors (e.g. timeouts) are raised via an exception.

Issue: #387
 2020-10-26 19:51:02 +01:00
Eike Kettner
0f84200118 Indicate saving when changing name for multiple items 2020-10-26 15:32:58 +01:00
Eike Kettner
0eabb7d589 Change tooltip of new select-items button 2020-10-26 14:46:04 +01:00
Eike Kettner
998aad5627 Delete multiple items 2020-10-26 14:46:04 +01:00
Eike Kettner
9193d7ca51 Send multiple items to reprocessing 2020-10-26 14:03:56 +01:00
Eike Kettner
26e89bf84e Edit org/person/equipment of multiple items 2020-10-26 13:35:47 +01:00
Eike Kettner
2e6026b817 Edit dates of multiple items 2020-10-26 13:16:03 +01:00
Eike Kettner
7ade7dd70b Display direction icon when editing multiple items 2020-10-26 12:49:30 +01:00
Eike Kettner
d4043634ac Edit direction of multiple items 2020-10-26 12:48:15 +01:00
Eike Kettner
42c989a6cd Edit folder of multiple items 2020-10-26 12:39:44 +01:00
Eike Kettner
17472fa4ca Edit name of multiple items 2020-10-26 12:17:55 +01:00
Eike Kettner
7ad37c8d26 Editing tags for multiple items 2020-10-26 11:54:04 +01:00
Eike Kettner
5735a47199 Replace changed item cards 2020-10-25 23:08:40 +01:00
Eike Kettner
458fa7edd9 Allow to search in a defined item subset 2020-10-25 22:19:55 +01:00
Eike Kettner
5f2f20c8d7 Extend http api for multi-edit 2020-10-25 22:19:55 +01:00
Eike Kettner
55cfc4c908 Allow to select multiple items for deletion and edit 2020-10-25 22:19:55 +01:00
Eike Kettner
6ea9193b5e Only scroll to the last visited item when initializing the page 
When searching again on that page, scrolling/highlighting should not
happen. It now happens whenever coming to this page (not only when
going back from detail view).

Fixes: #373
 2020-10-22 23:33:57 +02:00
Eike Kettner
3e752487e4 Remove unused argument 2020-10-22 23:33:46 +02:00
Eike Kettner
fc5f7c7dca Use previously updated ui settings 
Fixes: #368
 2020-10-22 00:08:03 +02:00
Eike Kettner
3e2d272746 Add unique constraint for equipment names 
Fixes #370
 2020-10-21 22:42:19 +02:00
Eike Kettner
3c0b86cb19 Fix regex patterns used for NER 
Patterns are split on whitespace by the nlp library and then compiled,
so each "word" must be a valid regex.

Fixes: #356
 2020-10-21 00:55:14 +02:00
Eike Kettner
6639a39375 Set version to 0.14.0-SNAPSHOT 2020-10-19 15:07:01 +02:00
Eike Kettner
7a9daa9f4b Set version to 0.13.0 2020-10-19 14:16:16 +02:00
Eike Kettner
e68ee67aec Fix route in openapi spec for uploading 2020-10-19 13:45:14 +02:00
Eike Kettner
d66ae4fdc2 Display error message properly on modal edit 2020-10-19 00:30:55 +02:00
Eike Kettner
3771587e55 Find duplicate tags without category 2020-10-19 00:30:41 +02:00
Eike Kettner
6a3386ce66 Fix sql comparison with optional values 2020-10-19 00:29:41 +02:00