Commit Graph

1075 Commits

Author SHA1 Message Date
Eike Kettner
1762e7afac Externalize strings for home page 2021-04-05 21:31:45 +02:00
Eike Kettner
9c1beb2240 Externalize strings in usersettings page 2021-04-04 22:07:37 +02:00
Eike Kettner
594818499e Externalize strings in manage-data page 2021-04-04 22:07:37 +02:00
Eike Kettner
8f1380fcf5 Externalize strings in UiSettings page 2021-04-04 22:07:37 +02:00
Eike Kettner
e76d574ea3 Externalize strings for document language 2021-04-02 23:30:51 +02:00
Eike Kettner
dac95e81e6 Remove unused code for search dropdown 2021-04-02 23:16:38 +02:00
Eike Kettner
9a23ba9afb Refactor FixedDropdown 2021-04-02 23:11:54 +02:00
Eike Kettner
8d15d97857 Refactor Dropdown 2021-04-02 22:20:55 +02:00
Eike Kettner
b9c98c6578 Externalize strings in queue page 2021-04-02 11:28:21 +02:00
Eike Kettner
968c8e02b7 Externalize strings in upload page 2021-04-02 11:28:21 +02:00
Eike Kettner
12cad398ac Externalize strings in new-invite page 2021-04-02 11:28:21 +02:00
Eike Kettner
db6198673b Externalize strings in register page 2021-04-02 11:28:21 +02:00
Eike Kettner
ec237a2eaa Externalized strings 2021-04-02 11:28:21 +02:00
Eike Kettner
2b81c72d96 Provide translations for main menu 2021-04-02 11:28:21 +02:00
Eike Kettner
322a3e837c Prepare for selecting languages
UI language is stored in user settings for authenticated users;
otherwise is only stored in the current model (not persisted).
2021-04-02 11:28:21 +02:00
Eike Kettner
9fcab84267 Add icon to fixed dropdown 2021-04-02 11:28:21 +02:00
Eike Kettner
589e136883 Add flag icons 2021-04-02 11:28:21 +02:00
Scala Steward
144ea852bf Update fs2-core, fs2-io to 2.5.4 2021-03-31 21:10:42 +02:00
Eike Kettner
c36073b852 Allow to give human readable summary to user tasks 2021-03-27 22:13:13 +01:00
Eike Kettner
63331b6399 Remove unused code 2021-03-27 22:13:13 +01:00
Eike Kettner
ff0b70abfe Fix scrollbar position 2021-03-27 22:13:13 +01:00
Eike Kettner
03459124ca Don't return items still in processing 2021-03-27 22:13:13 +01:00
Eike Kettner
cc38b850a6 Remove deprecated search routes and some refactoring 2021-03-27 22:13:13 +01:00
Eike Kettner
7d513dce0b Set version to 0.22.0-SNAPSHOT 2021-03-13 18:09:29 +01:00
Eike Kettner
39d6f5789a Set version to 0.21.0 2021-03-13 16:57:33 +01:00
Eike Kettner
8c6ad8fc4e This test only doesn't work on my ci 2021-03-13 16:57:08 +01:00
Eike Kettner
177488817d Fix h2 migration
Using java source code obviously requires `javac` during migration.
2021-03-13 16:38:48 +01:00
Eike Kettner
2e443bc9b9 Fix mariadb migration 2021-03-13 15:52:38 +01:00
Eike Kettner
eb56c605cb Add scrollbars 2021-03-13 14:11:37 +01:00
Eike Kettner
df6b6aba2c Fix reading uri from a string
The read routine did not report all errors in the return type but some
raised exceptions.
2021-03-12 22:31:19 +01:00
Eike Kettner
f8bd42e5bd Redo pdf conversion and text extraction on reprocess
When processing a new file conversion and text extraction is skipped
if detected to be already done. This prevents running expensive tasks
again after restarting/retrying. When explicitely reprocessing a file,
these tasks should run again and replace the existing results.
2021-03-12 00:45:28 +01:00
Eike Kettner
a7ee0aa08b Add a flag to processing task to distinguish re-/processing 2021-03-12 00:45:23 +01:00
Eike Kettner
76f5ab6c68 Allow to reprocess single and multiple items in the ui 2021-03-12 00:16:19 +01:00
Eike Kettner
058c31e1f6 Reprocessing now sets metadata to an item if not in state confirmed
When reprocessing an item, the metadat of all *files* are replaced.
This change now also sets some metadat to an item, but only if the
item is not in state "confirmed". Confirmed items are not touched, but
the metadata of the files is updated.
2021-03-12 00:16:19 +01:00
Eike Kettner
ebaa31898e Add missing solr migration for new language field 2021-03-12 00:16:00 +01:00
Eike Kettner
274e433d9d Allow to change use enum for org/equipment 2021-03-10 23:55:18 +01:00
Eike Kettner
0229a867af Add a use colum to metadata entities 2021-03-10 23:55:18 +01:00
Eike Kettner
1a188afbd7 Re-add ui-version to keep browser resource cache in service-worker 2021-03-10 23:16:01 +01:00
Eike Kettner
a4a84abae5 Show errors from failed register request
Also include a `@` in the valid chars for "idents". This allows to use
an e-mail address as username.
2021-03-10 22:14:55 +01:00
Eike Kettner
6a63694a3e Convert unit tests to munit 2021-03-10 19:48:56 +01:00
Eike Kettner
64e3cb04f3 Remove fomantic ui and ui-header handling 2021-03-09 22:45:29 +01:00
Eike Kettner
b95338e744 Remove old ui code in frontend 2021-03-09 20:40:49 +01:00
Eike Kettner
9991ad5fcc Add latvian language 2021-03-09 00:23:17 +01:00
Eike Kettner
77a87782b7 Refactoring parser
- put all used strings in one place to have it easier to track
- don't use `$` for shortcuts, it's a detail not interesting to a
  user; now names must not clash (which is a good idea anyways)
- Added two more shortcuts `conc` and `corr`
2021-03-08 22:51:14 +01:00
Eike Kettner
e681ffa96f Extend query builder allowing more conditions
Before only a column or a dbfunction could be used in a condition. It
is now allowed for all `SelectExpr`.
2021-03-08 22:51:08 +01:00
Eike Kettner
b514b85f39 Improve parser error messages a bit 2021-03-08 10:26:39 +01:00
Eike Kettner
30c901ddf1 Add more ways to query for attachments
- find items with a specified attachment count
- find items by attachment id
2021-03-08 09:49:38 +01:00
Eike Kettner
2b2f913e85 Add checksum query expr 2021-03-08 01:53:21 +01:00
Eike Kettner
7b1ec97c97 Fix sort when using fulltext only 2021-03-08 00:47:15 +01:00
Eike Kettner
c6032ff279 Check query in client 2021-03-08 00:46:31 +01:00
Eike Kettner
31e2e99c36 Add a $year shortcut for selecting items within some year 2021-03-07 13:34:35 +01:00
Eike Kettner
7638dc5111 Flatten nested and/or queries 2021-03-07 12:36:51 +01:00
Eike Kettner
63d146c2de Resolve fulltext search queries the same way as before
For now, fulltext search is only possible when being the only term or
inside the root AND expression.
2021-03-07 09:40:47 +01:00
Eike Kettner
1c834cbb77 Correctly compare numeric field values 2021-03-03 22:54:55 +01:00
Eike Kettner
d4006461f6 Serving scalajs artifacts and provide errors to js 2021-03-03 00:45:10 +01:00
Eike Kettner
71985244f1 Use a better representation for macros 2021-03-03 00:44:49 +01:00
Eike Kettner
a48504debb Specificly search for field id vs name 2021-03-02 21:09:31 +01:00
Eike Kettner
b4b5acde13 Enable power search for power users via ui settings
A different search bar is presented if enabled in ui settings that
allows to search via the new query language.
2021-03-01 22:45:17 +01:00
Eike Kettner
168f5a1a98 Fix like search for custom fields 2021-03-01 20:56:23 +01:00
Eike Kettner
489581d90b Fix parsing nested expressions
Since whitespace is used as a separator, it cannot be consumed by
and/or parens.
2021-03-01 20:56:23 +01:00
Eike Kettner
889e4f4fb0 SearchMenu uses query string instead of json form 2021-03-01 20:56:23 +01:00
Eike Kettner
f8307f77c6 Search by field id or name 2021-03-01 20:56:23 +01:00
Eike Kettner
18992ee374 Deprecate search endpoints 2021-03-01 15:31:02 +01:00
Eike Kettner
dadab0d308 Implement search by query in endpoints 2021-03-01 15:31:02 +01:00
Eike Kettner
698ff58aa3 Provide a more convenient interface to search 2021-03-01 11:50:07 +01:00
Eike Kettner
e079ec1987 Provide custom error structure for parse failures 2021-03-01 00:51:01 +01:00
Eike Kettner
d737da768e Move to munit in query module 2021-03-01 00:51:01 +01:00
Eike Kettner
9013d9264e Add more convenient date parsers and some basic macros 2021-03-01 00:51:01 +01:00
Eike Kettner
af73b59ec2 Parser improvements
- default expressions into a and node
- fix parsing string lists that end in whitespace
- fix package names of internal classes
2021-03-01 00:51:01 +01:00
Eike Kettner
a80d73d5d2 Optimize imports 2021-03-01 00:51:01 +01:00
Eike Kettner
e9ed998e3a Basic poc to search via custom query 2021-03-01 00:51:01 +01:00
Eike Kettner
186014a1c6 Refactor search to separate between a base query and user query
The `findBase` is adding only strictly required conditions. Everything
else comes from the user.
2021-03-01 00:51:01 +01:00
Eike Kettner
c3cdec416c Sketching some basic tests 2021-03-01 00:50:52 +01:00
Eike Kettner
be5c7ffb88 First draft of ast and parser 2021-03-01 00:46:57 +01:00
Eike Kettner
74a79a79d9 Initial project setup 2021-03-01 00:46:57 +01:00
Eike Kettner
cfa36a5270 Fix preview png tests
Outcome was checked manually.
2021-03-01 00:33:57 +01:00
Eike Kettner
e6d9ce2c37 Remove obsolete type capabilities
These are now detected by the new scala compiler and lead to compile
errors.
2021-03-01 00:16:30 +01:00
Eike Kettner
c7d4c77e6d Allow more suggestions for date variants in English 2021-02-26 00:35:17 +01:00
Yumekui
017241bccb
Fix typo 2021-02-22 15:09:10 +00:00
Eike Kettner
cd2f623773 Set version to 0.21.0-SNAPSHOT 2021-02-19 23:32:10 +01:00
Eike Kettner
4d3a25a8c1 Set version to 0.20.0 2021-02-19 22:36:55 +01:00
Eike Kettner
07a061febc Make sure certain styles are not stripped by purge-css 2021-02-19 22:36:55 +01:00
Eike Kettner
211499a7e6 Add bottom border to metadata view 2021-02-19 11:17:35 +01:00
Eike Kettner
d819878974 Re-add the button to create new tags in detail view 2021-02-19 11:07:42 +01:00
mergify[bot]
9156e56d65
Merge pull request #651 from eikek/language-source
Add language to a source
2021-02-19 01:30:24 +00:00
Eike Kettner
7ef3185659 Add language to a source
Allows to define upload urls for different languages.
2021-02-18 23:34:42 +01:00
Eike Kettner
5c6f57cc6a Trim values of some input data
Some values don't make sense when padded with whitespace, like
organization name, source ids etc. These are trimmed now when
received.
2021-02-18 22:07:32 +01:00
Eike Kettner
70fa57587b Fix setting ui theme for anonymous 2021-02-18 22:02:56 +01:00
Eike Kettner
0e9d8f8794 Fix hover when folder is a drop-target 2021-02-18 00:43:15 +01:00
Eike Kettner
d7bc963450 Cleanup nodes that are not reachable anymore 2021-02-18 00:37:18 +01:00
mergify[bot]
b16166f1e9
Merge pull request #645 from eikek/equipment-description
Equipment description
2021-02-17 22:17:03 +00:00
Eike Kettner
5181283b1b Add a short-name to organizations 2021-02-17 22:55:35 +01:00
Eike Kettner
20ccdda609 Add a notes field to equipments 2021-02-17 22:39:07 +01:00
Eike Kettner
e07d732193 Show item position in detail view 2021-02-17 22:18:22 +01:00
Eike Kettner
a79fa6f74a Add version info back to frontend 2021-02-17 21:54:53 +01:00
Eike Kettner
e177336155 Better align tags in multi-select field 2021-02-17 00:14:06 +01:00
Eike Kettner
4ef1996137 Fix extracted content border 2021-02-17 00:14:06 +01:00
Eike Kettner
33cfd6df6f Hide Load-more button when there is no more 2021-02-17 00:14:06 +01:00
Eike Kettner
29d80adace Ui updates: place result message below buttons 2021-02-17 00:14:06 +01:00
Eike Kettner
48eee00c0b Allow person to be correspondent, concerning or both 2021-02-16 22:49:55 +01:00
Eike Kettner
d99ce76d89 Remove person suggestion if it doesn't match with organization 2021-02-16 00:29:54 +01:00
Eike Kettner
eb308cfa85 Remove duplicate candidates when creating list of suggestions 2021-02-15 23:56:09 +01:00
Eike Kettner
64bacb665a Show queue counts in side menu 2021-02-15 00:27:55 +01:00
Eike Kettner
07edaaaf5e Try fix attachment selection for firefox
It works great in chromium, in firefox the border is a bit off.
2021-02-15 00:11:37 +01:00
Eike Kettner
9c90d7c4c2 Fix min height of tag area in detail
This avoids "nervous/noisy" ui flickering when changing between items
where one has no tags/fields to show.
2021-02-15 00:10:38 +01:00
Eike Kettner
a10971607e Always show custom field dropdown in detail mode
Even if there are no custom fields define, the control is shown now.
The user can create a new one in this view.
2021-02-14 23:34:18 +01:00
Eike Kettner
dec6148df6 Hide field stats component if there is nothing to show 2021-02-14 23:29:05 +01:00
Eike Kettner
31cb8284a5 Fix formatting icon in top nav bar 2021-02-14 23:22:37 +01:00
Eike Kettner
1dcc3941ed Fix overflowing content in item cards
When displaying long lines this leads to horizontal scrolling the
complete item list.
2021-02-14 23:18:18 +01:00
Eike Kettner
f29353adb3 Use different worker cache name per ui
Avoid to have users clear their browser caches
2021-02-14 11:29:36 +01:00
Eike Kettner
5461321e59 Remove unused styles and prevent postcss from purging relevant ones 2021-02-14 02:20:01 +01:00
Eike Kettner
f6e5bda2d8 Change default ui to be the new ui
The old ui is still available, using a header: `Docspell-Ui: 1`. Using
a browser addon a "ui switch" can be easily achieved.
2021-02-14 01:46:13 +01:00
Eike Kettner
dd935454c9 First version of new ui based on tailwind
This drops fomantic-ui as css toolkit and introduces tailwindcss. With
tailwind there are no predefined components, but it's very easy to
create those. So customizing the look&feel is much simpler, most of
the time no additional css is needed.

This requires a complete rewrite of the markup + styles. Luckily all
logic can be kept as is. The now old ui is not removed, it is still
available by using a request header `Docspell-Ui` with a value of `1`
for the old ui and `2` for the new ui.

Another addition is "dev mode", where docspell serves assets with a
no-cache header, to disable browser caching. This makes developing a
lot easier.
2021-02-14 01:46:13 +01:00
Eike Kettner
442b76c5af Setup tailwind with docspell build 2021-02-03 21:35:15 +01:00
Eike Kettner
62f596be35 Set version to 0.20.0-SNAPSHOT 2021-01-25 10:19:00 +01:00
Eike Kettner
73b54addc5 Set version to 0.19.0 2021-01-25 09:21:16 +01:00
Eike Kettner
394aeeccb6 Introduce a sql literal and constants in query builder
The h2 jdbc driver could not translate the union query in QCollective
when the `kind` was set via a constant value. Using literals works
here. Renamed the corresponding elements in the query builder.
2021-01-25 00:18:24 +01:00
mergify[bot]
6cc9c159d6
Merge pull request #590 from eikek/scan-mailbox-filter
Refactor scan mailbox form and add flag for post-processing
2021-01-24 01:06:51 +00:00
Eike Kettner
96612e0e59 Refactor scan mailbox form and add flag for post-processing
Mails are filtered once by using an imap search and then by some globs
to filter files and subjects. Imap can search by subject via a
string-contains, but not via globs or patterns (afaik). The subject
filter is applied to all downloaded mail headers. Now for post
processing (moving to some target folder or deleting), it can be
chosen to post-process all "seen" mails or only those that matched all
filters.
2021-01-24 01:46:31 +01:00
Eike Kettner
1b66e2af5c Fix classifier_settings table 2021-01-23 21:30:26 +01:00
Eike Kettner
c7e850116f Make the text length limit optional 2021-01-22 23:06:50 +01:00
mergify[bot]
8dd1672c8c
Merge pull request #583 from eikek/fix-baseurl-setting
Render baseurl without trailing slash
2021-01-21 23:44:14 +00:00
mergify[bot]
38e0a50942
Merge pull request #582 from eikek/delete-org-fix
Fix deleting organization
2021-01-21 22:56:56 +00:00
Eike Kettner
0ec620fcf0 Render baseurl without trailing slash
The webapp expects it like this currently, because the url is only a
string.
2021-01-21 21:42:08 +01:00
Eike Kettner
f4a03e7c69 Fix deleting organization
The foreign key in person must be resetted.
2021-01-21 21:27:02 +01:00
Eike Kettner
4cba96f390 Always return classifier results as suggestion
The classifier results are spliced into the suggestion list at second
place. When linking they are only used if nlp didn't find anything.
2021-01-21 21:05:28 +01:00
Eike Kettner
9957c3267e Add constraints from config to classifier training
For large and/or many documents, training the classifier can lead to
OOM errors. Some limits have been set by default.
2021-01-21 17:46:39 +01:00
Eike Kettner
363cf5aef0 Quote names in sql changesets 2021-01-21 00:22:58 +01:00
Eike Kettner
38387e00a0 Fix mariadb migration 2021-01-21 00:22:53 +01:00
Eike Kettner
a6c31be22f Update documentation 2021-01-20 22:47:15 +01:00
Eike Kettner
85ddc61d9d Move date proposal setting to nlp config 2021-01-20 19:17:29 +01:00
Eike Kettner
5d366c3bd6 Make labels in classifier settings more clear 2021-01-20 01:05:59 +01:00
Eike Kettner
b12d965223 Improve logging 2021-01-20 00:40:58 +01:00
Eike Kettner
27c24c128d Store tags guessed with classifier in database 2021-01-20 00:30:40 +01:00
Eike Kettner
9d83cb7fe4 Store item based proposals in separate table
Classifier don't work on each attachment, but on all. So the results
must not be stored at an attachment. This reverts some previous
changes to put the classifier results for item entities into its own
table.
2021-01-19 23:48:09 +01:00
Eike Kettner
3ff9284a64 Return classifier results as suggestions 2021-01-19 23:13:51 +01:00
Eike Kettner
75573c905e Use classifier results as fallback when linking proposed metadata 2021-01-19 23:13:34 +01:00
Eike Kettner
8455d1badf Lookup results from classifier
The model may be out of date, data may change. Then it should be
looked up to fetch the id to be compatible with next stages.
2021-01-19 22:56:01 +01:00
Eike Kettner
1cd3441462 Run classifier for item entities (concerned, correspondent)
Store the results separately from nlp results in attachment metadata.
2021-01-19 22:08:29 +01:00
Eike Kettner
d124f0c1a9 Rename db changeset
It's not just a fix, but adds new things
2021-01-19 22:08:29 +01:00
Eike Kettner
5c487ef7a9 Refactor running classifier in text analysis 2021-01-19 21:30:02 +01:00
Eike Kettner
99dcaae66b Learn classifiers for item entities
Learns classifiers for concerned and correspondent entities. This can
be used as an alternative to or after nlp.
2021-01-19 20:54:47 +01:00
Eike Kettner
a6f29153c4 Control what tag categories to use for auto-tagging 2021-01-19 01:20:13 +01:00
Eike Kettner
cce8878898 Exclude tags w/o category from classifying; remove obsolete models 2021-01-18 21:51:49 +01:00
Eike Kettner
3e28ce1254 Add the sql concat function to query builder 2021-01-18 21:51:45 +01:00
Eike Kettner
249f9e6e2a Extend guessing tags to all tag categories 2021-01-18 21:51:45 +01:00
Eike Kettner
3f75af0807 Add 9 more lanugages to the list of document lanugages 2021-01-18 17:41:40 +01:00
Eike Kettner
94bb18c152 Refactor solr language fields 2021-01-18 17:41:40 +01:00
Eike Kettner
26dff18ae0 Add spanish as an example
Adding a new language without nlp requires now only to fill out the
pieces:

- define a list of month names to support date recognition
- add it to joex' dockerfile to be available for tesseract
- update the solr migration/field definitions
- update the elm file so it shows up on the client
2021-01-18 17:41:40 +01:00
Eike Kettner
360cad3304 Refactoring solr/fts migration
When re-indexing everything, skip intermediate populating the index
and do this as the very last step.

Parameterize adding new fields by their language.
2021-01-18 17:41:40 +01:00