dschmidt commented on PR #3520: URL: https://github.com/apache/tika/pull/3520#issuecomment-6640469435
Thanks, all fifteen addressed in b9ad505692. 1. A hybrid yields its version 6 book only; `extractKf8` (off by default, like `extractAllAlternatives`) emits the KF8 copy, with no resource type instead of `ALTERNATE_FORMAT_CHUNK`. 2. Right, checked against the four hybrids in your sample: read relative to the KF8 record 0, the indices land on `HUFF`, `FDST`, `FCIS`, `FLIS`, `INDX` and `DATP`; read from the file start, on unrelated records. The KF8 record 0 is now kept as it is (only the first image index is cleared), and the generator writes the indices that way. With `extractKf8`, the three of them that use HUFF/CDIC give 2391, 3703 and 37256 characters of KF8 text, where the KF8 half had about 60 and a warning before; the PalmDOC one hid the bug, as you suspected. 3. KF8 text ends at the end of the first FDST flow. Both KF8-only books in the sample had stylesheet text before and have none now. 4. `tika:encrypted` is set alongside the warning. 5. `supported-formats.adoc` regenerated; `testSupportedFormatsPartialIsCurrent` passes. 6. The HTML pass uses `Metadata.newInstance(context)`; the text is cut at `maxTextLength` (now 16 MB) and at 32 times the file size. 7. EXTH is read up to 1024 records and 64 values per type. 8. The KF8 half is built only when it is extracted and the extractor wants it, straight from the file's records into one array. 9. A record that does not decode keeps the text of the records before it, with a warning. 10. A file that is not a Mobipocket book (another Palm Database type, or no Palm Database at all, as a Topaz or KFX book named `.azw`) yields a warning and nothing else. PalmDOC `TEXtREAd` text would be a cheap follow-up. 11. CHANGES says so, and a detection test covers `.mobi`, `.azw`, `.azw3` and `.prc` with `BOOKMOBI` content. 12. `maxFileSize` (256 MB by default) is a setting and documented; of a larger book the beginning is read as a truncated one is, so the metadata and what fits come out. 13. EXTH 106 goes through `TikaDates.toMetadataString`: a date-only value stays a date, non-ISO dates parse. 14. Unknown and zero codepages read as windows-1252. 15. Tests for the file limit (with a small `maxFileSize`), the fuzz test with `extractKf8` and a `@Timeout`, the two tests that swallowed exceptions now assert, and a new stored-text fixture covers EXTH 524 over the locale, the title fallback and a cde-type. Also re-run against your 19-file sample and the Gutenberg books: covers match the index (the 1 MiB cut lacks its cover, which is behind the cut), no container exceptions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
