[
https://issues.apache.org/jira/browse/TIKA-4856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110302#comment-18110302
]
Tim Allison commented on TIKA-4856:
-----------------------------------
This is a big PR. It took me some time to try to figure out the overall goals.
I'm somewhat allergic to adding a new endpoint or parameters for tika-server
endpoint for something so specialized. The application requirements of having
equality btwn /rmeta and /unpack are non-trivial. We should be doing that
anyways, I admit.
What would you think of configuring tika-server something like this:
{noformat}
{
"server": {
"endpoints": ["rmeta", "unpack", "status"]
},
"pipes": {
"numClients": 4,
"forkedJvmArgs": ["-Xmx1g"]
},
"parsers": [
{ "pdf-parser": { "thumbnail": { "enabled": true, "dpi": 96, "imageType":
"RGB" } } },
{ "emf-parser": { "renderImage": true, "renderOnlyEmbeddedResourceTypes":
["THUMBNAIL"] } },
{ "wmf-parser": { "renderImage": true, "renderOnlyEmbeddedResourceTypes":
["THUMBNAIL"] } },
{ "default-parser": {} }
],
"parse-context": {
"unpack-config": { "includeMetadataInZip": true },
"standard-unpack-selector": { "includeEmbeddedResourceTypes":
["THUMBNAIL"] }
}
}
{noformat}
Then we wouldn't need a special endpoint or rest parameters. We'd still need
some of the underlying wiring.
My agent thinks we'd need:
{noformat}
Work list beyond the maxRenderedPages clamp
1. PDF thumbnail config + emit path (the real work).
PDFBoxRenderer.getDPI/imageType read config.getOcr() —
the renderer needs to take dpi/imageType from the request (e.g. on
RenderRequest) instead. Emit via
parseEmbedded with type THUMBNAIL. Reuse the loaded PDDocument
(setOpenContainer) rather than a second
Loader.loadPDF. Tests: THUMBNAIL present, N_PAGES unchanged, text intact,
no RENDERING entries, 0-page
PDF.
2. OCR of thumbnails. On a tesseract server every THUMBNAIL image (stored or
rendered — pre-existing for
stored) gets OCR'd. Either TesseractOCRParser skips
EMBEDDED_RESOURCE_TYPE=THUMBNAIL by default, or a
skipOcrOnEmbeddedResourceTypes knob defaulting to [THUMBNAIL]. Small, but
without it the index-time call
pays tesseract per PDF.
3. Sequence #3095 first; confirm its rendering is typed THUMBNAIL with
rendered-by. Add the
docx-with-EMF-thumbnail end-to-end test then.
4. Delete from #3096: ThumbnailDefaults, ThumbnailSelector,
/unpack/thumbnail, all renderThumbnails params,
thumbnail-defaults key + TikaJsonConfig change. Repurpose
UnpackerThumbnailTest to exercise the two
recipes end-to-end (/rmeta/config → THUMBNAIL entry; /unpack → zip
containing only the thumbnail + its
metadata json).
5. Docs: a "Thumbnails" section with the server-wide config and the two
per-request recipes;
maxRenderedPages into pdf-parser-full.json (both copies); CHANGES
rewritten.
6. Separate ticket: @JsonMerge on nested config objects (PDFParserConfig.ocr
etc.). Not required for the
redirect since thumbnail is self-contained, but the preview recipe's
"ocr": {"strategy": "NO_OCR"} still
replaces the operator's whole ocr block, and every hand-written request
config today has the same trap.{noformat}
> /unpack/thumbnail: return the document thumbnail with its metadata
> ------------------------------------------------------------------
>
> Key: TIKA-4856
> URL: https://issues.apache.org/jira/browse/TIKA-4856
> Project: Tika
> Issue Type: New Feature
> Reporter: Dominik Schmidt
> Priority: Major
>
> With TIKA-4850 through TIKA-4855 every container format that carries a
> thumbnail emits it as a THUMBNAIL embedded document, the PDF parser renders
> pages as RENDERING documents, and the EMF/WMF renderer turns the vector
> thumbnails of Office documents into raster ones. Getting "the thumbnail of
> this file" out of that still takes format knowledge on the client: the
> THUMBNAIL of a Word or Excel file is an EMF/WMF whose usable form is the
> RENDERING underneath it, a PDF has no THUMBNAIL but a page RENDERING, the
> THUMBNAIL of a DOCX inside a ZIP is not the ZIP's, and with rendering enabled
> the picture of an embedded OLE object is a RENDERING too. Plus the request
> config that switches the renderers on.
> Proposal: POST /unpack/thumbnail next to /unpack and /unpack/all, multipart
> like them. It runs the usual forked parse in unpack mode with a fixed parse
> context (PDF page 1 rendered, EMF/WMF rendered) and picks, in this order: the
> raster THUMBNAIL at depth 1; the rendering of that thumbnail; the depth-1
> RENDERING of PDF page 1. The endpoint extracts what the document carries; it
> does not resize, convert or generate previews.
> The response is JSON: the /rmeta metadata object of the selected embedded
> document, and the image as base64. Thumbnails are small, so the encoding
> overhead does not matter, and the caller gets type, dimensions, origin
> (stored thumbnail or rendering, tk:rendering:rendered-by) and path in one
> round trip without unpacking a zip. 204 when the document has no thumbnail.
> {
> "metadata": {
> "Content-Type": "image/png",
> "Content-Length": "8459",
> "tiff:ImageWidth": "800",
> "tiff:ImageLength": "1131",
> "tk:embedded-resource-type": "RENDERING",
> "tk:embedded-resource-path": "/thumbnail.emf/thumbnail.png",
> "tk:embedded-depth": "2",
> "tk:rendering:rendered-by": "poi-metafile-renderer",
> "tk:resource-name": "thumbnail.png"
> },
> "image": "iVBORw0KGgoAAAANSUhEUgAA..."
> }
> To keep the selection rule short, the metafile renderer could give the
> rendering of a THUMBNAIL the THUMBNAIL type as well (its
> tk:rendering:rendered-by tells it apart), so a raster thumbnail is a
> THUMBNAIL regardless of whether the document stored it as PNG or as EMF.
> What do you think?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
