---
layout: docs
title: Extract Text from Files
permalink: dev/adr/0011_extract_text
---

# Extract Text from Files

## Context and Problem Statement

With support for more file types there must be a way to extract text
from all of them. It is better to extract text from the source files,
in contrast to extracting the text from the converted pdf file.

There are multiple options and multiple file types. Again, most
priority is to use a java/scala library to reduce external
dependencies.

## Considered Options

### MS Office Documents

There is only one library I know: [Apache
POI](https://poi.apache.org/). It supports `doc(x)` and `xls(x)`.
However, it doesn't support open-document format (odt and ods).

### OpenDocument Format

There are two libraries:

- [Apache Tika Parser](https://tika.apache.org/)
- [ODFToolkit](https://github.com/tdf/odftoolkit)

*Tika:* The tika-parsers package contains an opendocument parser for
extracting text. But it has a huge dependency tree, since it is a
super-package containing a parser for almost every common file type.

*ODF Toolkit:* This depends on [Apache Jena](https://jena.apache.org)
and also pulls in quite some dependencies (while not as much as
tika-parser). It is not too bad, since it is a library for
manipulating opendocument files. But all I need is to only extract
text. I created tests that extracted text from my odt/ods files. It
worked at first sight, but running the tests in a loop resulted in
strange nullpointer exceptions (it only worked the first run).

### Richtext

Richtext is supported by the jdk (using `RichtextEditorKit` from
swing).

### PDF

For "image" pdf files, tesseract is used. For "text" PDF files, the
library [Apache PDFBox](https://pdfbox.apache.org) can be used.

There also is [iText](https://github.com/itext/itext7) with a AGPL
license.

### Images

For images and "image" PDF files, there is already tesseract in place.

### HTML

HTML must be converted into a PDF file before text can be extracted.

### Text/Markdown

These files can be used as-is, obviously.


## Decision Outcome

- MS Office files: POI library
- Open Document files: Tika, but integrating the few source files that
  make up the open document parser. Due to its huge dependency tree,
  the library is not added.
- PDF: Apache PDFBox. I know this library better than itext.