TheAnachronism/docspell
1
0
Fork 0
mirror of https://github.com/TheAnachronism/docspell.git synced 2024-11-13 02:31:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
NaN Commits 2 Branches 1 Tag 89 MiB
df95513ba3
Commit Graph

550 Commits

Author SHA1 Message Date
Eike Kettner
af7cfa0ae1 Allow editing metadata in item-detail 2020-08-07 01:30:43 +02:00
Eike Kettner
f3ba224124 Add missing organization/person/equipment routes 2020-08-07 01:30:43 +02:00
Eike Kettner
639ab7440e Fix edit menu layout 2020-08-06 23:49:54 +02:00
Eike Kettner
a8ea391715 Render edit-modals above the menu and not the whole page 2020-08-06 23:38:55 +02:00
Eike Kettner
a6a6e334d5 Search by tag category via web ui 2020-08-06 22:23:35 +02:00
Eike Kettner
070c2b5e5f Allow to search by tag categories 
The server accepts a list of tag categories for inclusion and
exclusion. The categories in the include list imply to return items
that have at least one tag of each category. The categories in the
exclude list imply to return all items that have no tag in any of
these categories.
 2020-08-06 21:43:27 +02:00
Eike Kettner
cf3e051e83 Fix load more button 2020-08-06 00:49:15 +02:00
Eike Kettner
dfbbcdf73c Allow only one horizontal form being open 2020-08-05 23:11:21 +02:00
Eike Kettner
082f468155 Use a icon menu for the edit menu top bar 2020-08-05 22:43:04 +02:00
Eike Kettner
baa25d0f2f Allow to set item notes below or above the files 2020-08-05 22:43:04 +02:00
Eike Kettner
0453494cc6 Make notes below the files view and always visible 
It looks similiar to github's readme. If there are no notes, the form
is displayed.
 2020-08-05 22:41:08 +02:00
Eike Kettner
1662e1e2c8 Split ItemDetail file into multiple files due to its size 2020-08-05 17:57:45 +02:00
Eike Kettner
08f953dd52 Display item notes in card view if configured 
The user can set a maximum length of the item notes to display in each
card. If set to 0, it is hidden.
 2020-08-05 00:09:44 +02:00
Eike Kettner
09d74b7e80 Return item notes with search results 
In order to not make the response very large, a admin can define a
limit on how much to return.
 2020-08-05 00:09:37 +02:00
Eike Kettner
dbd27057d1 Improve source view and add qrcode for urls 
The qr-code for urls is added so that these urls are easy to copy into
a phone. Then buttons for copying them into the clipboard have been
added.
 2020-08-03 23:58:41 +02:00
Eike Kettner
ed8f16fe73 Add a qr-code for source urls 2020-08-03 18:27:13 +02:00
Eike Kettner
deacd8e9f6 Set version to 0.10.0-SNAPSHOT 2020-08-01 19:03:32 +02:00
Eike Kettner
2664b3ddb2 Set version to 0.9.0 2020-08-01 16:09:24 +02:00
Eike Kettner
45b0deeced Print solr url on start 
This is useful info to see which url has been selected, same as db
connection.
 2020-08-01 15:59:14 +02:00
Eike Kettner
1fc57fc2b2 Set default value for min-text-len to 500 
This value is used to decide whether to try OCR or not. If text is
below this value, OCR is run and both results are compared. It was set
to 10, which is just one or two words. Since the context for docspell
are documents, this value is too low.
 2020-08-01 15:46:00 +02:00
Eike Kettner
a06d20a479 Remove duplicate results from index-only search 2020-08-01 15:46:00 +02:00
Eike Kettner
b4e11a7264 Fixes a race condition when initializing the calendar-event field 
The problem was that the field executes a request to validate its
state. This was initiated at the same time for two values. Then it was
undetermined which value comes back first.
 2020-08-01 11:42:01 +02:00
mergify[bot]
f95f01759b
Merge pull request #198 from eikek/default-search 
Default search
 2020-07-31 23:28:23 +00:00
Eike Kettner
0599176ae8 Update scala to 2.13.3 2020-08-01 01:03:43 +02:00
Eike Kettner
46b784cc33 Simplify search bar and menu 
The option "contents" has been removed from the search bar. This field
is not intended to be used alone, but rather in conjunction with other
fields. Otherwise it may be really slow on large databases.

The "name" option has been removed from the search menu. This doesn't
provide anything better over the "Names" field, that search more
fields, including item names.
 2020-08-01 00:26:41 +02:00
mergify[bot]
5bf302a40e
Merge pull request #196 from eikek/website 
Website
 2020-07-31 20:51:38 +00:00
Eike Kettner
808a5a3c94 Remove old site 2020-07-31 01:28:09 +02:00
mergify[bot]
2f5036231c
Merge pull request #194 from eikek/sort-tag-list 
Sort tag list by count
 2020-07-30 22:23:42 +00:00
Eike Kettner
091ded50cb Sort tag list by count 
It was displayed in some random order. Now the most used tag is first.
 2020-07-31 00:14:25 +02:00
Eike Kettner
79eb7b4d66 Fix datepicker position for action input fields 
Either the width and appearance must be changed to match this of an
`ui action input` or the position must be fixed as done here. It is
not correctly positioned, because the `ui input` class uses a flex.
 2020-07-31 00:02:57 +02:00
Eike Kettner
f8c6f79b10 Initial website 2020-07-30 20:33:26 +02:00
Eike Kettner
cec4948710 Add pdf meta data to extracted text to add it to full-text index 2020-07-19 01:07:49 +02:00
Eike Kettner
209c068436 Use keywords in pdfs to search for existing tags 
During processing, keywords stored in PDF metadata are used to look
them up in the tag database and associate any existing tags to the
item.

See #175
 2020-07-19 00:28:04 +02:00
Eike Kettner
da68405f9b Extract meta data from pdfs using pdfbox 2020-07-18 23:04:46 +02:00
Eike Kettner
bd20165d1a Use given folder-id when adding initial fts docs 2020-07-18 23:04:01 +02:00
Eike Kettner
3d49ceaab5 Use ocrmypdf tool to create pdf/a during conversion 
- Use another external tool to convert pdf to pdf which also adds the
  extracted text as another layer into the pdf

- Although not used, the external conversion routine will now check
  for an existing text file that is named as the pdf file with extension
  `.txt`. If present it is included in the conversion result and will be
  used as the extracted text.

- text extraction for pdf files happens now on the converted file,
  because it may already contain the text from the conversion step and
  thus avoids running OCR twice.

- All errors during conversion are not fatal; processing continues
  without a converted file.
 2020-07-18 17:19:29 +02:00
Eike Kettner
99210365ce Update documentation for folders 2020-07-17 00:02:25 +02:00
Eike Kettner
94089fd0b6 Fix decoding joex responses in JoexClient 2020-07-15 20:45:07 +02:00
Eike Kettner
c697501571 Add folders sql changeset for mariadb 2020-07-14 23:22:52 +02:00
Eike Kettner
25538d6a59 Allow to set a folder when importing mailboxes 2020-07-14 23:18:39 +02:00
Eike Kettner
225877a40c Show folder in item detail view 2020-07-14 23:18:39 +02:00
Eike Kettner
ca5b7b999f Update source form to specify folder 2020-07-14 23:18:39 +02:00
Eike Kettner
5b01c93711 Add a folder-id to item processing 
This allows to define a folder when uploading files. All generated
items are associated to this folder on creation.
 2020-07-14 23:18:39 +02:00
Eike Kettner
ec7f027b4e Fix postgres changeset for folders 2020-07-12 16:15:02 +02:00
Eike Kettner
259526a088 Organize imports 2020-07-12 13:51:52 +02:00
Eike Kettner
22fa1dba13 Apply folder restriction to fulltext only search 
And update index when folder changes.
 2020-07-12 13:50:45 +02:00
Eike Kettner
aeba4ba913 Refactor full-text migrations and add folder to solr schema 2020-07-12 13:50:14 +02:00
Eike Kettner
e387b5513f Remove items in non-member folders from sql search results 2020-07-11 22:25:56 +02:00
Eike Kettner
5b95fddf3d Make item queries depend on the account-id 
Now the user is required, too, to list items.
 2020-07-11 21:54:51 +02:00
Eike Kettner
e66c501056 Extend dropdown to display additional option info 
Use this to display folder information when setting the folder on an
item.
 2020-07-11 17:56:08 +02:00
Eike Kettner
0df541f30a Allow to search by folders 2020-07-11 16:52:13 +02:00
Eike Kettner
86443e10a6 Set the folder of an item 2020-07-11 12:57:17 +02:00
Eike Kettner
5bde78083a Hide delete button when creating new folder 2020-07-11 11:54:23 +02:00
Eike Kettner
2ab0b5e222 Rename space -> folder 2020-07-11 11:54:23 +02:00
Eike Kettner
0365c1980a Show new data about spaces in web-ui 2020-07-11 01:30:29 +02:00
Eike Kettner
60a08fc786 Return member count and if current user is owner or member 2020-07-11 01:30:29 +02:00
Eike Kettner
ea4ab11195 Allow to only return owning spaces 2020-07-11 01:30:28 +02:00
Eike Kettner
6c304b4e7a Manage spaces in web-ui 2020-07-11 01:30:28 +02:00
Eike Kettner
752a94a9e2 Implement space operations 2020-07-11 01:30:28 +02:00
Eike Kettner
0e8c9b1819 Initial outline for managing spaces 2020-07-11 01:30:28 +02:00
Eike Kettner
d43e17d9fb Transport user-id to client 2020-07-11 01:30:28 +02:00
Eike Kettner
c12201c4a5 Add routes to manage spaces 2020-07-11 01:30:28 +02:00
Eike Kettner
7ec0fc2593 Add endpoints for managing spaces to openapi spec 2020-07-11 01:30:28 +02:00
Eike Kettner
13ad5e3219 Setup space entities 2020-07-11 01:30:28 +02:00
Eike Kettner
fadd21944f Set version to 0.9.0-SNAPSHOT 2020-06-29 21:04:15 +02:00
Eike Kettner
8998706598 Set version to 0.8.0 2020-06-29 20:37:52 +02:00
Eike Kettner
7b922fec94 Update documentation and fix changelog wording 2020-06-29 20:37:52 +02:00
Eike Kettner
347a029af8 Scalafix organize-imports 2020-06-28 21:20:47 +02:00
Eike Kettner
5bad157b9e Change link on home page 2020-06-28 19:34:28 +02:00
Eike Kettner
82104ff148 Update documentation and changelog 2020-06-28 14:45:04 +02:00
Eike Kettner
d3b3c6289b Prepare docker setup for fulltext search 2020-06-28 13:37:39 +02:00
Eike Kettner
8500d4d804 Extend consumedir.sh to work with integration endpoint 
Now running one consumedir script can upload files to multiple
collectives separately.
 2020-06-28 00:08:37 +02:00
Eike Kettner
41c0f70d3b Fix cancelling jobs 
A request to cancel a job was not processed correctly. The cancelling
routine of a task must run, regardless of the (non-final) state. Now
it works like this: if a job is currently running, it is interrupted
and its cancel routine is invoked. It then enters "cancelled" state.
If it is stuck, it is loaded and only its cancel routine is run. If it
is in a final state or waiting, it is removed from the queue.
 2020-06-26 23:08:27 +02:00
Eike Kettner
23477e34f9 Change columns from timestamp to datetime 
In MariaDB the timestamp has some properties that make it a not a good
fit.
 2020-06-26 17:07:00 +02:00
Eike Kettner
d79ae6233a Restrict proposals for due date 
Avoid dates too far in the future.
 2020-06-26 16:58:17 +02:00
Eike Kettner
91da3b149e Reducing default retries to 2 
Many errors cannot be recovered from by retrying. There is currently
no way to distinguish these states so it is now set to a lower value
to have not long wait times until an item arrives.
 2020-06-25 23:57:01 +02:00
mergify[bot]
50b3554c9a
Merge pull request #160 from eikek/fts 
Fts
 2020-06-25 21:19:42 +00:00
Eike Kettner
dc8f1a0387 Fix global re-index task to re-create the schema 
Otherwise new instances could not be re-indexed.
 2020-06-25 23:02:06 +02:00
Eike Kettner
4a41168bbb Allow a collective to re-index their data 
If something goes wrong, this might be necessary.
 2020-06-25 21:52:38 +02:00
Eike Kettner
2a98c2ca42 Fix openapi spec for joex 2020-06-25 08:43:02 +02:00
Eike Kettner
c81b92af6d Documentation updates 2020-06-25 01:36:26 +02:00
Eike Kettner
0ba1736bc8 Remove items/attachments from index on delete 2020-06-25 00:00:10 +02:00
Eike Kettner
64c96942a9 Fix deleting items that have sent mails 2020-06-24 23:47:58 +02:00
Eike Kettner
14213c4c27 Allow some solr query options in the config file 2020-06-24 23:37:20 +02:00
Eike Kettner
793f33b640 Update finding documentation 2020-06-24 23:37:20 +02:00
Eike Kettner
532caed84c Consistent logging of request/responses to solr 
Using a middleware. Also add missing changesets for mariadb.
 2020-06-24 21:25:46 +02:00
Eike Kettner
47697a8056 Set some logs to trace 2020-06-24 01:16:13 +02:00
Eike Kettner
7df77208fe Fix duplicate search results 2020-06-24 01:15:53 +02:00
Eike Kettner
8e0282c25f Indicate when the search-menu is not used 2020-06-24 01:15:41 +02:00
Eike Kettner
7d7460b1c9 Cleanup + hiding false errors from log 2020-06-24 00:23:22 +02:00
Eike Kettner
30937d4908 Set default max page size to 200 2020-06-24 00:04:10 +02:00
Eike Kettner
43b18db76a Don't scroll when loading more items 2020-06-24 00:03:58 +02:00
Eike Kettner
b8558d6837 Don't trigger search when fields are cleared 2020-06-24 00:03:17 +02:00
Eike Kettner
6846f2f46e Add new search-index route to web-ui 2020-06-24 00:03:17 +02:00
Eike Kettner
d5c9923a6d Add a route that only searches the full-text index 
It returns the results in the same order as received from the index to
preserve the relevance ordering.
 2020-06-24 00:03:17 +02:00
Eike Kettner
d9f0f05613 Refactor findItemsWithTags to more general useful 2020-06-23 21:27:01 +02:00
Eike Kettner
647911563e Fix paging when using full-text search 2020-06-23 01:44:52 +02:00
Eike Kettner
15c0fb4395 Merge branch 'master' into fts 2020-06-23 00:32:27 +02:00
Eike Kettner
e06a3f8fdd ScalafmtAll 2020-06-23 00:18:59 +02:00
Eike Kettner
a3e16e57de Display search highlighting in webapp 2020-06-23 00:17:29 +02:00
Eike Kettner
ffbb16db45 Transport highlighting information to the client 2020-06-23 00:17:29 +02:00
Eike Kettner
a58ffd11e1 Return attachment-name from index 2020-06-22 21:28:26 +02:00
Eike Kettner
8ffbf651b0 Add fts routes to openapi.yml 2020-06-22 21:00:34 +02:00
Eike Kettner
5e77ff0fc5 Start search by pressing enter and not on each stroke 
This reduces (unecessary) requests and eases the restless list
rendering. Throttling is still in place to avoid too many requests
when e.g. holding the enter-key.
 2020-06-22 00:46:11 +02:00
Eike Kettner
6c8abf4cd7 Fix searchmenu styling 2020-06-21 23:45:12 +02:00
Eike Kettner
af4522c0a7 Prepare ui to handle content search 2020-06-21 22:33:10 +02:00
Eike Kettner
3d82e03a8a Remove solr query from debug log 2020-06-21 22:29:45 +02:00
Eike Kettner
cfe5aa8894 Use no-op fts-client if disabled + push this flag to the webui 2020-06-21 21:06:08 +02:00
Eike Kettner
330fdcdd5b Add rest endpoints to re-create the index 2020-06-21 20:13:33 +02:00
Eike Kettner
0d8b03fc61 Add backend operations for re-creating the full-text index 2020-06-21 15:46:51 +02:00
Eike Kettner
14ea4091c4 Renaming things 2020-06-21 13:15:02 +02:00
Eike Kettner
2f6e531c45 Refactoring index migration task 2020-06-21 01:37:23 +02:00
Eike Kettner
9acea8307d Update full-text index when changing data 2020-06-21 00:33:39 +02:00
Eike Kettner
383614f908 Allow updating single fields in solr 2020-06-20 23:37:47 +02:00
Eike Kettner
7609b2b7c3 Run scalafmtAll 2020-06-20 23:03:51 +02:00
Eike Kettner
1f4ff0d4c4 Add language to schema, extend fts-client 2020-06-20 22:44:47 +02:00
Eike Kettner
3576c45d1a First basic working solr search 2020-06-20 02:18:49 +02:00
Eike Kettner
2a0bf24088 Setup solr schema and index all data using a system task 
The task runs on application start. It sets the schema using solr's
schema api and then indexes all data in the database. Each step is
memorized so that it is not executed again on subsequent starts.
 2020-06-19 21:37:22 +02:00
Eike Kettner
1f4220eccb Index exsiting data in solr 2020-06-19 00:43:35 +02:00
Eike Kettner
60c079f664 Add task to index current database state 2020-06-18 22:38:45 +02:00
Eike Kettner
146d1b0562 Make data to index more flexible and extensible 2020-06-17 23:20:46 +02:00
Eike Kettner
522daaf57e Introducing fts client into codebase 2020-06-17 23:20:46 +02:00
Eike Kettner
ee801745a7 Add fultext field to search params 2020-06-17 23:20:46 +02:00
Eike Kettner
c7f598e3b0 Initial module setup 2020-06-17 23:20:46 +02:00
Eike Kettner
492f4d304f Set version to 0.8.0-SNAPSHOT 2020-06-17 22:28:45 +02:00
Eike Kettner
33bef52229 Set version to 0.7.0 2020-06-17 21:59:13 +02:00
Eike Kettner
bc20e9a9bb Execute search when loading page 2020-06-17 21:57:50 +02:00
Eike Kettner
4028b7979e Fix mariadb timestamp columns 
MariaDB automatically inserts the current time, even when saying `SET
datecol = null`.
 2020-06-17 21:51:30 +02:00
Eike Kettner
897d91475e Update scalafmt-core to 2.6.0 2020-06-17 19:53:56 +02:00
Eike Kettner
7a3d2e4dc6 Extract OItemSearch from OItem 2020-06-15 23:13:48 +02:00
Eike Kettner
09579eef16 Add docs for how to create a smtp gateway 2020-06-14 21:35:20 +02:00
Eike Kettner
0643534994 Add edit icon for changing attachment name 2020-06-14 17:03:07 +02:00
Eike Kettner
84a26461ed Add a route to update the name of an attachment 2020-06-14 17:03:07 +02:00
Eike Kettner
e15e2c9313 Replace hardcoded number by a config value 2020-06-14 00:49:19 +02:00
Eike Kettner
479a341b13 Add documentation page for searching 2020-06-14 00:18:18 +02:00
Eike Kettner
e0c8c45061 Hide search menu by default and add search bar to top 
Adds a search bar to the top of the listing page that is the same as
the `All Names` search field in the search menu. Search menu can be
toggled as before.
 2020-06-13 22:05:16 +02:00
Eike Kettner
88234986e6 Make name field search in item name only 
Now there is an `allNames` field that searches names of multiple
things.
 2020-06-13 21:17:29 +02:00
Eike Kettner
67666595eb Make name search case insensitive 2020-06-13 21:17:15 +02:00
Eike Kettner
217fef7384 Throttle search requests 
Throttle search requests when typing. Also fix handling subscriptions
in main.
 2020-06-13 21:17:15 +02:00
Eike Kettner
5468e24b55 Add search field for 'all-names' to search menu 2020-06-13 21:03:45 +02:00
Eike Kettner
f30c8a5e4d Add new search term that searches in all meta data 
A field that searches via substring search in names of correspondents
and concerned meta data.
 2020-06-13 17:08:26 +02:00
Eike Kettner
35c9409680 Automatically add wildcards to name search 
If wildcards are already present or if the search string appears in
quotes, wildcard '*' characters are not added.
 2020-06-13 16:31:16 +02:00
Eike Kettner
6e4aeb512d Reorganize search menu items 2020-06-13 16:19:35 +02:00
Eike Kettner
617487f5b3 Add mail-debug flag to rest-server 
It has been added to the joex application, but it should be possible
to debug mail problems on both apps.
 2020-06-13 15:10:00 +02:00
Eike Kettner
1f9f09ba6f Reorder user task table columns 2020-06-13 14:48:12 +02:00
Eike Kettner
bdfc151272 Close task form after submitting and return to list view 2020-06-13 14:40:13 +02:00
Eike Kettner
2c13f9307c Adopt UI to allow multiple notification tasks 2020-06-13 14:26:38 +02:00
Eike Kettner
e51e84408b Change notify-due-item routes to allow multiple tasks per user 2020-06-13 14:26:38 +02:00
Eike Kettner
d41ddd9729 Fix bug in dropdown 
The menu for a multi-dropdown should only show entries that are not
already selected (there is no use case to select the same item more
than once).
 2020-06-12 23:12:49 +02:00
Eike Kettner
216c89c7d1 Some small documentation fixes 2020-06-12 21:52:29 +02:00