Commit Graph

106 Commits

Author SHA1 Message Date
Eike Kettner
5882405f30 Search index if search object only contains this field 2020-12-05 02:59:57 +01:00
Eike Kettner
0560867b83 Use scalafix' semanticdb version
Which is now correct after the scalafix bump
2020-11-27 23:47:51 +01:00
Eike Kettner
3fabe0a582 Update to Scala 2.13.4 2020-11-27 20:26:24 +01:00
Eike Kettner
248ad04dd0 Prepare custom fields 2020-11-22 21:41:09 +01:00
Eike Kettner
3c4f2c6841 Fix packaging tools
Due to hardcoding files, the new ones were not packaged.
2020-11-13 22:19:21 +01:00
Eike Kettner
55a6f7aaf6 Add more properties to upload meta data 2020-11-11 21:01:23 +01:00
Eike Kettner
3a26620b2b debian packages: use same system user
Default database is H2 which requires that both processes can access
the file.
2020-10-20 01:07:23 +02:00
Eike Kettner
3b8500245f Provide gz version of web assets 2020-09-03 00:22:58 +02:00
Eike Kettner
dbd27057d1 Improve source view and add qrcode for urls
The qr-code for urls is added so that these urls are easy to copy into
a phone. Then buttons for copying them into the clipboard have been
added.
2020-08-03 23:58:41 +02:00
Eike Kettner
0599176ae8 Update scala to 2.13.3 2020-08-01 01:03:43 +02:00
Eike Kettner
9af02cd90b Autoformat build.sbt 2020-07-30 20:33:26 +02:00
Eike Kettner
9bc6c60dfe Update build for new website 2020-07-30 20:33:26 +02:00
Eike Kettner
f8c6f79b10 Initial website 2020-07-30 20:33:26 +02:00
Eike Kettner
d277f99fee Add sbt alias for reformatting 2020-07-14 23:18:39 +02:00
Eike Kettner
672ed445b1 Add scalafix and organize-imports rule 2020-06-28 21:20:47 +02:00
Eike Kettner
41c0f70d3b Fix cancelling jobs
A request to cancel a job was not processed correctly. The cancelling
routine of a task must run, regardless of the (non-final) state. Now
it works like this: if a job is currently running, it is interrupted
and its cancel routine is invoked. It then enters "cancelled" state.
If it is stuck, it is loaded and only its cancel routine is run. If it
is in a final state or waiting, it is removed from the queue.
2020-06-26 23:08:27 +02:00
Eike Kettner
330fdcdd5b Add rest endpoints to re-create the index 2020-06-21 20:13:33 +02:00
Eike Kettner
1f4220eccb Index exsiting data in solr 2020-06-19 00:43:35 +02:00
Eike Kettner
c7f598e3b0 Initial module setup 2020-06-17 23:20:46 +02:00
Eike Kettner
09579eef16 Add docs for how to create a smtp gateway 2020-06-14 21:35:20 +02:00
Eike Kettner
bd5066740d Joex depends on backend module
The job executor depends on backend module, since it may control the
application via user tasks. The `ONode` can now be moved from the
store module into the backend module.
2020-05-10 21:03:12 +02:00
Eike Kettner
0a1b3fcf95 Set list-id header for notification mails 2020-04-30 21:23:56 +02:00
Eike Kettner
fa10fe3fae Update scala to 2.13.2 2020-04-24 22:24:31 +02:00
Eike Kettner
2723d6b43b Implement notify-due-items task 2020-04-22 21:08:45 +02:00
Eike Kettner
e97e0db45c Prepare notification form 2020-04-22 21:08:45 +02:00
Eike Kettner
1206105f0b Fix several bugs with handling e-mail files
- When converting from html->pdf, the wkhtmltopdf program exits with
  errors if the document contains invalid links. The content is now
  cleaned before handed to wkhtmltopdf.
- Update emil library which fixes a bug when reading mails without
  explicit transfer encoding (8bit)
- Add a info header to converted mails
2020-04-07 22:38:25 +02:00
Eike Kettner
e331808ecf Update microsite 2020-03-28 21:44:14 +01:00
Eike Kettner
16edf84752 Setup new site 2020-03-27 00:35:15 +01:00
Eike Kettner
e305b46708 Extract tnef attachments and fix incomplete html
The wkhtmltopdf requires the content encoding set correctly in the
document.
2020-03-24 23:40:29 +01:00
Eike Kettner
cf7ccd572c Improve handling encodings
Html and text files are not fixed to be UTF-8. The encoding is now
detected, which may not work for all files. Default/fallback will be
utf-8.

There is still a problem with mails that contain html parts not in
utf8 encoding. The mail text is always returned as a string and the
original encoding is lost. Then the html is stored using utf-8 bytes,
but wkhtmltopdf reads it using latin1. It seems that the `--encoding`
setting doesn't override encoding provided by the document.
2020-03-23 22:51:28 +01:00
Eike Kettner
00ca6b5697 Improve text analysis
- Search for consecutive labels

- Sort list of candidates by a weight

- Search for organizations using person labels
2020-03-17 22:34:50 +01:00
Eike Kettner
854a596da3 Integrate periodic tasks
The first use case for periodic task is the cleanup of expired
invitation keys. This is part of a house-keeping periodic task.
2020-03-08 22:49:49 +01:00
Eike Kettner
1e598bd902 Sketch a scheduler for running periodic tasks
Periodic tasks are special in that they are usually kept around and
started based on a schedule. A new component checks periodic tasks and
submits them in the queue once they are due.

In order to avoid duplicate periodic jobs, the tracker of a job is
used to store the periodic job id. Each time a periodic task is due,
it is first checked if there is a job running (or queued) for this
task.
2020-03-08 12:55:03 +01:00
Eike Kettner
9b28858d06 Create a simple client for joex in its api module
This client can be used within the backend app and later in other
modules. The `OJoex` object is replaced with a better implementation
where the http client is initialized once on app start.
2020-03-03 23:07:49 +01:00
Eike Kettner
d8bbcb1409 Fix front-page links for microsite
The links work while testing locally with jekyll. Must be checked at
the published site.
2020-03-01 09:45:38 +01:00
Eike Kettner
ad8d64eded Fix microsite and add changelog 2020-02-27 23:59:03 +01:00
Eike Kettner
8cfecfb3dd Update docs 2020-02-22 00:48:58 +01:00
Eike Kettner
9b1349734e Convert some files to pdf 2020-02-19 02:03:10 +01:00
Eike Kettner
8143a4edcc Adding extraction primitives 2020-02-16 21:37:26 +01:00
Eike Kettner
851ee7ef0f Reorganize processing code
Use separate modules for

- text extraction
- conversion to pdf
- text analysis
2020-02-15 21:25:25 +01:00
Eike Kettner
5c3d2b2e28 Rename example-files to files 2020-02-14 11:14:09 +01:00
Eike Kettner
bf9bf25502 Rename example files 2020-02-14 11:10:54 +01:00
Eike Kettner
569aae3038 Add example files into its own project
The text and convert module can use them in their tests.
2020-02-11 22:46:23 +01:00
Eike Kettner
ce22b727b1 Add new convert module and sketch its integration 2020-02-11 00:33:52 +01:00
Eike Kettner
e1826f39ac Disable revolver plugin on non-app projects
This allows to type `reStart` in the root sbt project to start both
applications.
2020-02-08 18:02:31 +01:00
Eike Kettner
61bbdab8b5 nix: add user doc and pkg fixes
- Add user doc for how to use with nix/nixos

- fix potential collisions in packages if both are installed via
  `nix-env`
2020-01-24 21:56:48 +01:00
Eike Kettner
2454f358b1 Add sbt task to create a zip for things in tools/ 2020-01-19 20:32:52 +01:00
Eike Kettner
c84a69aa9c Update documentation 2020-01-11 22:12:51 +01:00
Eike Kettner
2e3454c7a1 Starting with mail settings 2020-01-05 15:31:32 +01:00
Eike Kettner
d05e919eb4 Update doobie, use legacy java.time conversions 2019-12-31 13:55:09 +01:00
Eike Kettner
57e274e2b0 Upgrade microsite 2019-12-30 02:33:46 +01:00
Eike Kettner
546f1a6ee3 Add scalafmt.conf and elm compile options 2019-12-29 20:52:43 +01:00
Eike Kettner
2ad1586d00 Set stricter compile options and fix cookie data 2019-09-28 22:17:45 +02:00
Eike Kettner
9a6f3eb786 Update dependencies 2019-09-21 22:02:37 +02:00
Eike Kettner
831cd8b655 Initial version.
Features:

- Upload PDF files let them analyze

- Manage meta data and items

- See processing in webapp
2019-09-21 22:02:36 +02:00
Eike Kettner
6154e6a387 Initial application stub 2019-09-21 14:54:03 +02:00