Eike Kettner
5d366c3bd6
Make labels in classifier settings more clear
2021-01-20 01:05:59 +01:00
Eike Kettner
b12d965223
Improve logging
2021-01-20 00:40:58 +01:00
Eike Kettner
27c24c128d
Store tags guessed with classifier in database
2021-01-20 00:30:40 +01:00
Eike Kettner
9d83cb7fe4
Store item based proposals in separate table
...
Classifier don't work on each attachment, but on all. So the results
must not be stored at an attachment. This reverts some previous
changes to put the classifier results for item entities into its own
table.
2021-01-19 23:48:09 +01:00
Eike Kettner
3ff9284a64
Return classifier results as suggestions
2021-01-19 23:13:51 +01:00
Eike Kettner
75573c905e
Use classifier results as fallback when linking proposed metadata
2021-01-19 23:13:34 +01:00
Eike Kettner
8455d1badf
Lookup results from classifier
...
The model may be out of date, data may change. Then it should be
looked up to fetch the id to be compatible with next stages.
2021-01-19 22:56:01 +01:00
Eike Kettner
1cd3441462
Run classifier for item entities (concerned, correspondent)
...
Store the results separately from nlp results in attachment metadata.
2021-01-19 22:08:29 +01:00
Eike Kettner
d124f0c1a9
Rename db changeset
...
It's not just a fix, but adds new things
2021-01-19 22:08:29 +01:00
Eike Kettner
5c487ef7a9
Refactor running classifier in text analysis
2021-01-19 21:30:02 +01:00
Eike Kettner
99dcaae66b
Learn classifiers for item entities
...
Learns classifiers for concerned and correspondent entities. This can
be used as an alternative to or after nlp.
2021-01-19 20:54:47 +01:00
Eike Kettner
a6f29153c4
Control what tag categories to use for auto-tagging
2021-01-19 01:20:13 +01:00
Eike Kettner
cce8878898
Exclude tags w/o category from classifying; remove obsolete models
2021-01-18 21:51:49 +01:00
Eike Kettner
3e28ce1254
Add the sql concat function to query builder
2021-01-18 21:51:45 +01:00
Eike Kettner
249f9e6e2a
Extend guessing tags to all tag categories
2021-01-18 21:51:45 +01:00
Eike Kettner
3f75af0807
Add 9 more lanugages to the list of document lanugages
2021-01-18 17:41:40 +01:00
Eike Kettner
94bb18c152
Refactor solr language fields
2021-01-18 17:41:40 +01:00
Eike Kettner
26dff18ae0
Add spanish as an example
...
Adding a new language without nlp requires now only to fill out the
pieces:
- define a list of month names to support date recognition
- add it to joex' dockerfile to be available for tesseract
- update the solr migration/field definitions
- update the elm file so it shows up on the client
2021-01-18 17:41:40 +01:00
Eike Kettner
360cad3304
Refactoring solr/fts migration
...
When re-indexing everything, skip intermediate populating the index
and do this as the very last step.
Parameterize adding new fields by their language.
2021-01-18 17:41:40 +01:00
Eike Kettner
ff121d462c
Disable memory intensive tests on travis
2021-01-18 17:41:40 +01:00
Eike Kettner
f01646aeb5
Reorganize nlp pipeline and add nlp-unsupported language italian
...
Improves and reorganizes how nlp pipelines are setup. Now users can
choose from many options, depending on their hardware and usage
scenario.
This is the base to use more languages without depending on what
stanford-nlp supports. Support then is involves to text extraction and
simple regex-ner processing.
2021-01-18 17:41:40 +01:00
Eike Kettner
a70e9ab614
Store used language for processing on attachmentmeta
...
Issue: #570
2021-01-17 22:56:33 +01:00
Eike Kettner
6cf3f9be5a
Fix joex version endpoint in spec
2021-01-17 22:56:33 +01:00
Eike Kettner
aa937797be
Choose nlp mode in config file
2021-01-17 22:56:33 +01:00
Eike Kettner
54a09861c4
Use model cache with basic annotator
2021-01-17 22:56:33 +01:00
Eike Kettner
a77f67d73a
Make pipeline cache generic to be used with BasicCRFAnnotator
2021-01-17 22:56:33 +01:00
Eike Kettner
4462ebae0f
Resurrect the basic ner classifier
2021-01-17 22:56:33 +01:00
Eike Kettner
a699e87304
Separate ner from classification
2021-01-17 22:56:33 +01:00
Eike Kettner
f02f15e5bd
Move blocker into constructor of text analyser
2021-01-17 22:56:33 +01:00
Eike Kettner
b2b8ad625a
scalafmt
2021-01-17 20:11:58 +01:00
Eike Kettner
f0f0e6e0d4
Search for categories case-insensitive
...
The string was already lowercased, but the comparison was not.
Fixes #568
2021-01-17 20:10:24 +01:00
Eike Kettner
623a61dbb6
Introduce a lowerEq operator to the query builder
2021-01-17 20:10:00 +01:00
Eike Kettner
54bd75e99e
Set version to 0.19.0-SNAPSHOT
2021-01-11 23:27:47 +01:00
Eike Kettner
0d1b55a205
Set version to 0.18.0
2021-01-11 22:39:40 +01:00
Eike Kettner
d77b5855e4
Set default pool-size to 1
2021-01-11 22:30:59 +01:00
Eike Kettner
38ae7a9027
Make source a quick link on card and detail
2021-01-11 21:37:36 +01:00
Eike Kettner
33458766fe
Correcty reset search menu when clicking on custom-field quick link
2021-01-11 14:03:23 +01:00
Eike Kettner
7beda302b2
Fix and improve tag search menu
...
Show also "empty tags", where the count is 0. Before only tags with a
count > 0 were displayed. When searching this is fine, but when using
drag&drop to attach tags to items, it is good to see all. They can be
hidden via a button.
The tags are now ordered by their count descending, but regarding to
the overall count – not the current view. Otherwise the tags are
reordered when clicking on them, which is confusing. Also it then
shows the "more important" (most used) tags first, even when the
result is a subset.
A fix was made related to updating the menu. When coming back from
the detail view where a tag with prior count=0 was associated, the
menu didn't show it, because it relied on a previous state, where this
tag were not included.
2021-01-11 13:01:38 +01:00
Eike Kettner
3fccc3df39
Return all tags in search stats result
...
Before only tags with a count > 0 were included. Now those that have
not attached to any item are returned as well.
2021-01-11 12:13:13 +01:00
Eike Kettner
75986c461f
Fix ner date label boundary reporting
2021-01-10 09:10:39 +01:00
Eike Kettner
fb05e997ab
Provide multiple date suggestions for English
...
Issue: #561
2021-01-10 09:02:26 +01:00
Eike Kettner
bddafa7d28
Fix looping over already seen mails when they are skipped
...
When skipping mails due to a filter, it must still enter the
post-handling step. Otherwise it will be seen again on next run.
Issue: #551
2021-01-09 15:07:18 +01:00
Eike Kettner
d712f8303d
Make glob matching case-insensitive by default
2021-01-09 13:23:15 +01:00
Eike Kettner
cbca4d234f
Fix scrolling to card
...
That was broken due to the independent scroll in commit #bcb1b8.
2021-01-09 02:00:01 +01:00
Eike Kettner
cef1c38cc4
Restrict height of job output
2021-01-09 01:49:55 +01:00
Eike Kettner
0abd7dea10
Fix scrolling to top in detail view
2021-01-09 01:16:59 +01:00
Eike Kettner
c0d7aba5d5
Improve selecting attachments of an item
...
Use a list of small thumbnails instead of just names.
Closes : #396
2021-01-09 01:16:59 +01:00
Eike Kettner
9bc2084499
Allow to click on custom fields in detail view
...
Closes : #514
2021-01-09 01:16:59 +01:00
Eike Kettner
48d182667d
Harmonize login and register page
2021-01-09 01:16:59 +01:00
Eike Kettner
752c8f9be2
Show new-invite as normal page
2021-01-09 01:16:59 +01:00
Eike Kettner
bcb1b87fc0
Enable independent scrolling of search menu and list
...
Fixes some other minor css issues.
Closes : #541
2021-01-09 01:16:59 +01:00
Eike Kettner
3c12e3678f
Allow to search for *
in custom date fields
...
This requires to pass the raw input through to the caller.
Closes : #550
2021-01-09 01:16:59 +01:00
Bo Jeanes
14dacaa837
Fix typo
2021-01-09 10:41:53 +11:00
Eike Kettner
716252721c
Fix cache clearing
...
It must be cancelled when obtaining a pipeline.
2021-01-07 23:31:01 +01:00
Eike Kettner
a670bbb6c2
Make idle interval when clearing nlp cache configurable
2021-01-06 23:03:00 +01:00
Eike Kettner
73a9572835
Poc for clearing stanford pipeline after some idle time
2021-01-05 23:56:20 +01:00
Eike Kettner
b08e88cd69
Add (inofficial) routes to get system information
2021-01-05 20:54:53 +01:00
Eike Kettner
30df887934
Sort custom field options in dropdown
2021-01-05 18:04:54 +01:00
Eike Kettner
668abf2140
Add a reset-password admin route
2021-01-04 20:59:31 +01:00
Eike Kettner
2a172ce720
Remove fulltext recreate-key config value
...
It's now in the admin routes, protected by the
`admin-endpoint.secret`.
2021-01-04 15:18:02 +01:00
Eike Kettner
306f064ad9
Create a place for admin routes
...
And move re-creation of fulltext index in this place.
2021-01-04 15:18:02 +01:00
Eike Kettner
0cfd8974d3
Add a flag to imap settings to enable/disable oauth2 scheme
2021-01-04 11:03:04 +01:00
mergify[bot]
84d160aa38
Merge pull request #534 from eikek/duplicate-check
...
Fixing find-by-checksum with exclusions
2021-01-03 11:55:28 +00:00
Eike Kettner
95fd386c14
Fixing find-by-checksum with exclusions
...
The NOT-IN query must check for null separately, as everything with
null evaluates to false in sql resulting in not finding existing
duplicates.
2021-01-03 12:29:03 +01:00
Eike Kettner
40b093ae72
Fix selecting imap connection in scan mailbox settings
...
Issue: #524
2021-01-03 10:31:36 +01:00
Eike Kettner
611e480eb4
Use more prominent log line to indicate start of processing
...
Issue: #530
2021-01-02 21:47:54 +01:00
Eike Kettner
97dfcece97
Fix duplicate check on restarts
...
Issue: #530
2021-01-02 21:18:05 +01:00
Eike Kettner
2b8ef3c2f7
Allow to hide search stats in user settings
2021-01-02 10:20:18 +01:00
Eike Kettner
ae57881f2d
Sort custom fields in views
2021-01-02 10:09:05 +01:00
Eike Kettner
70983f115c
Fix custom field summary table for mobile view
2021-01-02 09:55:12 +01:00
Eike Kettner
a9ed0364d2
Fix linking guessed tags
...
Since tag names are lower-cased the search must happen lower-cased, too.
2021-01-02 01:30:31 +01:00
Eike Kettner
36858da624
Fix search condition for empty items set
2020-12-17 23:07:04 +01:00
Eike Kettner
4ec133b0b9
Remove unused imports
2020-12-17 23:06:58 +01:00
Eike Kettner
69f57d1eb1
Replace empty hrefs with a href #
2020-12-17 21:15:33 +01:00
Eike Kettner
c9d4e8ec46
Fix selecting items in multi-edit mode
...
For some to me unknown reason, changing the dom slightly (removing
hidden elements), resulted in a different event dispatching. The cards
while being attached to an event would reload the page as if the event
is propagated. This happned by commit #8d7b3c7d in Home/View.elm.
Adding the hidden nodes back into the dom, "fixed" it.
This change now gives a better fix in assuring that every anchor has
either a sensible `href` or an event and a `href #`.
2020-12-17 21:02:23 +01:00
Eike Kettner
6346bf6a34
Add summary for fulltext searches
2020-12-17 00:11:33 +01:00
Eike Kettner
8d7b3c7d74
Show custom field summary above results
2020-12-16 22:06:34 +01:00
Eike Kettner
b66738b4c3
Add folder count to search menu
2020-12-16 19:20:30 +01:00
Eike Kettner
8fba637ebe
Add folder counts to search summary
2020-12-16 01:14:27 +01:00
Eike Kettner
a995ea8729
Update tag counts in search menu
2020-12-16 00:56:12 +01:00
Eike Kettner
80e23d1c84
Add a route to get search summary
2020-12-15 23:37:02 +01:00
Eike Kettner
77627534bc
Improve on basic search summary
2020-12-15 23:37:02 +01:00
Eike Kettner
f3855628d5
Extend query builder with more functions
2020-12-15 23:34:12 +01:00
Eike Kettner
4ca6dfccae
Get basic search summary
2020-12-15 23:10:13 +01:00
Eike Kettner
56d6d2e2ac
Allow changing more parts of a select
2020-12-15 22:12:44 +01:00
Eike Kettner
f1c4b4adb0
Extract find-item query condition
2020-12-15 21:03:47 +01:00
Eike Kettner
2dff686fa0
Introduce unit condition
2020-12-15 21:03:47 +01:00
Eike Kettner
80406cabc2
Refactoring some code into separate files
2020-12-15 21:03:47 +01:00
Eike Kettner
278b1c22c9
Remove old code
2020-12-15 21:03:46 +01:00
Eike Kettner
2cecd01837
Convert rest of QItem
2020-12-15 21:03:46 +01:00
Eike Kettner
d1606d6f16
Remove old commented code
2020-12-15 21:03:46 +01:00
Eike Kettner
266fec9eb5
Convert find items query
2020-12-15 21:03:46 +01:00
Eike Kettner
5e2c5d2a50
Extends query builder
2020-12-15 21:03:46 +01:00
Eike Kettner
35c62049f5
Start converting QItem
2020-12-15 21:03:46 +01:00
Eike Kettner
a355767fdb
Convert all query libs besides QItem
2020-12-15 21:03:46 +01:00
Eike Kettner
fd6d09587d
Convert more records
2020-12-15 21:03:46 +01:00
Eike Kettner
613696539f
Minor refactorings
2020-12-15 21:03:46 +01:00
Eike Kettner
d6f28d3eca
Convert folder
2020-12-15 21:03:46 +01:00
Eike Kettner
87eb8c7f55
Convert more records
2020-12-15 21:03:46 +01:00
Eike Kettner
e3f6892abd
Convert job record
2020-12-15 21:03:46 +01:00
Eike Kettner
1aa1f4367e
Convert periodic tasks
2020-12-15 21:03:46 +01:00
Eike Kettner
3cef932ccd
Convert more records
2020-12-15 21:03:46 +01:00
Eike Kettner
fe4815c737
Convert RSentMail
2020-12-15 21:03:46 +01:00
Eike Kettner
5cbf0d5602
Convert more records
2020-12-15 21:03:46 +01:00
Eike Kettner
10b49fccf8
Converting user and userimap records
2020-12-15 21:03:46 +01:00
Eike Kettner
c5c7f7ed3b
Convert equipment record
2020-12-15 21:03:46 +01:00
Eike Kettner
adee496b77
Convert source record
2020-12-15 21:03:46 +01:00
Eike Kettner
2dbb1db2fd
Initial outline for a simple query builder
2020-12-15 21:03:46 +01:00
Eike Kettner
b338f18e98
Remove redundant fileCount from search result
2020-12-15 21:03:46 +01:00
Eike Kettner
21cd778c37
Merge branch 'fix-0.17' into master
2020-12-15 20:45:39 +01:00
Eike Kettner
14428c0ead
Set version to 0.17.1
2020-12-15 20:25:59 +01:00
Eike Kettner
a987ede7c9
Avoid server roundtrip when switching pages
...
When changing pages in webui a server request should not happen. This
makes the webapp snappy again.
Issue: #508
2020-12-15 20:22:12 +01:00
Eike Kettner
ae0a543101
Set next version to 0.18.0-SNAPSHOT
2020-12-14 14:57:40 +01:00
Eike Kettner
398f969207
Set version to 0.17.0
2020-12-14 14:34:27 +01:00
Eike Kettner
27d087b14c
Fix foreign key constraints
2020-12-14 14:34:22 +01:00
Eike Kettner
7e42426edc
Fix page routing with remember me
2020-12-11 01:16:57 +01:00
Eike Kettner
c00714ab61
Don't redirect to login for open pages
2020-12-08 21:12:09 +01:00
Eike Kettner
f55566388a
Fix endless reload of login page
2020-12-06 00:58:15 +01:00
Eike Kettner
66bd4f53cb
Fix person-org relation info
2020-12-05 22:44:19 +01:00
Eike Kettner
2aed7ba142
Simplify search bar and search menu
...
The functionality of the search bar is now in the search menu, too.
The search menu shows one input field for "textual search", which is
either the fulltext search (if enabled) or a basic search in various
names.
2020-12-05 22:38:27 +01:00
Eike Kettner
5882405f30
Search index if search object only contains this field
2020-12-05 02:59:57 +01:00
Eike Kettner
0b6f965fcb
Fix rememberme for missing local storage
2020-12-04 22:57:21 +01:00
Eike Kettner
0337391b20
Change remember period to 30 days
2020-12-04 20:30:25 +01:00
Eike Kettner
f5ae389eea
Cleanup remember-me tokens periodically
2020-12-04 17:59:25 +01:00
Eike Kettner
a0642905db
Use remember-me cookie if present
2020-12-04 17:59:25 +01:00
Eike Kettner
c10c1fad72
Prepare remember-me authentication variant
2020-12-04 17:59:25 +01:00
Eike Kettner
290989f67f
Reorder correspondent person suggestion based on org relationship
2020-12-01 23:39:45 +01:00
Eike Kettner
d4470ab5fd
Restrict person dropdown options to the associated organization
2020-12-01 23:39:45 +01:00
Eike Kettner
a6dd71af9f
Set an organization to a person in webapp
2020-12-01 23:39:45 +01:00
Eike Kettner
2e28c8e57b
Harmonize table views
2020-12-01 23:39:45 +01:00
Eike Kettner
fc2668feee
Allow to connect a person to an organization
2020-12-01 23:39:45 +01:00
Eike Kettner
346d23928d
Allow to search in source names in webui
2020-11-30 14:26:10 +01:00
Eike Kettner
0ee8ff66d5
Allow to search by source name
2020-11-30 14:07:45 +01:00
Eike Kettner
bdc49aae98
Let user change template for card title and subtitle
2020-11-29 23:36:25 +01:00
Eike Kettner
81a136d915
Use a template for rendering title and subtitle of the item card
...
Introduces `ItemTemplate` to conveniently create strings given an
item.
2020-11-29 23:36:20 +01:00
Eike Kettner
45e4035e07
Set version to 0.17.0-SNAPSHOT
2020-11-28 18:13:50 +01:00
Eike Kettner
4682bf4a0e
Set version to 0.16.0
2020-11-28 17:55:40 +01:00
mergify[bot]
7ee69f7a57
Merge pull request #481 from eikek/reload-fix
...
Reload fix
2020-11-28 01:23:26 +00:00
Eike Kettner
75055ba34b
Fix help link menu item
2020-11-28 01:48:58 +01:00
Eike Kettner
7052bc6b8e
Add cc and bcc to item mail
2020-11-28 01:36:59 +01:00
Eike Kettner
d62c4a5a72
Fix updating search view
...
For example, when content-search was activated the view was not
correctly updated after deleting or drag-and-drop.
2020-11-28 00:51:16 +01:00
Tammo van Lessen
e9347176bd
Fixes an off-by-one classic to also accept dates in January
2020-11-28 00:43:35 +01:00
Eike Kettner
32c9113bd4
Remove unused code
2020-11-28 00:00:10 +01:00
Eike Kettner
1091a4fbdc
Show concerning/correspond person exclusively in dropdowns
...
Now a person that is not marked as "concerning only" is treated as
"correspondent only". The options of all dropdown fields are now
restricted to those subsets.
Issue: #332
2020-11-27 23:13:37 +01:00
Eike Kettner
3fabe0a582
Update to Scala 2.13.4
2020-11-27 20:26:24 +01:00
Eike Kettner
7cc290135a
Show currently logged in user
...
Closes : #329
2020-11-26 23:53:55 +01:00
Eike Kettner
c87163052b
Allow to choose from existing tag categories
...
Also fixes a ui problem with a too long dropdown menu in the small
modal popup when adding tags or custom fields in item detail view.
Issue: #331
2020-11-26 23:53:50 +01:00
Eike Kettner
0919eec3c2
Improve field query and fix mariadb's pickiness with parens
...
If no query is given, don't search with `like '%'`. MariaDB doesn't
want parens around columns in the GROUP BY clause.
2020-11-25 21:08:49 +01:00
Eike Kettner
52c6659f9f
Add missing schema migrations for custom fields
2020-11-25 21:08:49 +01:00
Eike Kettner
41b26fbbd6
Don't hide field dropdown when no fields are defined
2020-11-24 23:38:22 +01:00