[
https://issues.apache.org/jira/browse/TIKA-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Allison updated TIKA-5026:
------------------------------
Attachment: mobi-5026.zip
> Parse Mobipocket and Kindle KF8 ebooks (metadata, text, cover)
> --------------------------------------------------------------
>
> Key: TIKA-5026
> URL: https://issues.apache.org/jira/browse/TIKA-5026
> Project: Tika
> Issue Type: New Feature
> Reporter: Dominik Schmidt
> Priority: Major
> Attachments: mobi-5026.zip
>
>
> Tika detects application/x-mobipocket-ebook (BOOKMOBI magic at offset 60,
> globs *.prc and *.mobi) and application/vnd.amazon.ebook (glob *.azw only),
> but no parser module handles either: /rmeta on a 9.6 MB .azw3 returns the
> Content-Type and nothing else, /unpack/preset/thumbnail returns 204, while
> the EPUB of the same book yields title, author, text and the cover marked
> THUMBNAIL. The KF8 type application/vnd.amazon.mobi8-ebook (IANA, glob *.azw3
> in shared-mime-info) is missing from tika-mimetypes.xml altogether.
> The container is the same for all three: a Palm Database (PDB) whose record 0
> holds the PalmDOC header, the MOBI header and the EXTH metadata records. A
> parser next to EpubParser could do, in this order:
> 1. Metadata from EXTH: 100 creator, 101 publisher, 103 description, 104 ISBN,
> 105 subject, 106 publishing date, 108 contributor, 503 title (falls back to
> the full name in the MOBI header), 524 language; plus the MOBI header's
> locale and encoding (CP1252 or UTF-8). Hybrid files carry EXTH 121 (KF8
> boundary) and a second record 0 for the KF8 part with its own EXTH.
> 2. Cover: EXTH 201 (CoverOffset) and 202 (ThumbOffset) index into the image
> records starting at the MOBI header's first image index; the records are
> plain JPEG/GIF/PNG. Emit the cover as an embedded resource with
> tk:embedded-resource-type THUMBNAIL (TIKA-4856 marker), the thumbnail record
> as a second one, so /unpack/preset/thumbnail serves the cover like it does
> for EPUB.
> 3. Text: MOBI7 records with PalmDOC LZ77 (compression 2) or none (1),
> trailing entries stripped per the MOBI header's extra data flags, HTML handed
> to the HTML parser. HUFF/CDIC (17480) and the KF8 skeleton/fragment text
> format are more work and can be a follow-up; metadata and cover are the same
> for both generations and cover the common search and preview use cases.
> DRM (MOBI header encryption type != 0, EXTH 208/209): record 0, EXTH and the
> image records are not encrypted, so metadata and cover still come out; the
> text does not and should raise EncryptedDocumentException rather than emit
> garbage.
> Detection: add application/vnd.amazon.mobi8-ebook as a sub-type of
> application/x-mobipocket-ebook with glob *.azw3 (the magic is identical, the
> generation is only visible from the MOBI header version and EXTH 121, so
> content detection stays at the parent), and give application/vnd.amazon.ebook
> the same parent so one parser claims all three.
> Test files: Project Gutenberg offers every title as .mobi/.kf8 (public
> domain), Calibre converts EPUB to both .mobi and .azw3 for fixtures with
> known metadata and cover.
> Related: TIKA-1892 (the magic), TIKA-4856 (thumbnail marker), TIKA-4988
> (thumbnail preset).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
