[
https://issues.apache.org/jira/browse/TIKA-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128912#comment-18128912
]
ASF GitHub Bot commented on TIKA-4980:
--------------------------------------
dschmidt opened a new pull request, #3370:
URL: https://github.com/apache/tika/pull/3370
Fixes https://issues.apache.org/jira/browse/TIKA-4980
Three TIFF-based raw formats produced no preview, each for its own reason.
**Epson ERF** is now detected as a raw by its compression code (32769) and
its Make ("SEIKO EPSON"), and `image/x-raw-epson` gains `sub-class-of
image/tiff`. The registry line is not cosmetic: without it a detector returning
that type would leave the file with no parser and nothing to fall back on,
where today TiffParser at least reads its metadata. It also makes the type
reachable by a declared Content-Type, which MimeTypes only accepts as a
refinement of what it detected.
**Olympus ORF and Panasonic RW2** are classic TIFF layouts behind their own
magic number (0x4F52/0x5352 and 0x0055), which `locateJpegPreviews()` rejected
outright. Their previews are not in the standard tags either: RW2 keeps its
JPEG in the private JpgFromRaw tag 0x002E while its strip tags are
placeholders, ORF keeps the camera thumbnail and the preview in the MakerNote,
addressed relative to the MakerNote's first byte (0x0100, and 0x2020
CameraSettings -> 0x0101/0x0102). Only the `OLYMPUS\0II` header is read; the
older `OLYMP\0` one addresses differently and is left alone.
Verified against CC0 samples from raw.pixls.us:
```
P3180165.ORF (E-M10) image/x-raw-olympus THUMBNAIL 3200x2400
INLINE 160x120
_7061961_copy.ORF (E-5) image/x-raw-olympus THUMBNAIL 3200x2400
INLINE 160x120
RAW_PANASONIC_DMC-GX-7_11 image/x-raw-panasonic THUMBNAIL 1920x1920
Lumix_DMC-LX100_1.RW2 image/x-raw-panasonic THUMBNAIL 1920x1440
```
The largest preview by JPEG length becomes the THUMBNAIL and the camera
thumbnail an INLINE image, as TIKA-4851 set it up. Make, model and dimensions
are extracted for all of them and none records a parse exception. An Epson R-D1
sample carries no embedded JPEG at all, so it stays without a thumbnail; its
detection and typing are fixed regardless.
Tests use two hand-built files, `testORF.orf` (890 bytes) and `testRW2.rw2`
(421 bytes), in the style of the existing raw test documents, plus the
detection cases. `apache-rat:check` is clean on both touched modules.
Not covered: the older Olympus `OLYMP\0` MakerNote header, Fuji RAF (not a
TIFF container), and the vendor-level mime entries that carry no `sub-class-of`
because they span several containers (`image/x-raw-canon` covers CRW, CR2 and
CR3).
> Extract previews from Olympus ORF, Panasonic RW2 and Epson ERF
> --------------------------------------------------------------
>
> Key: TIKA-4980
> URL: https://issues.apache.org/jira/browse/TIKA-4980
> Project: Tika
> Issue Type: Improvement
> Components: parser
> Reporter: Dominik Schmidt
> Priority: Major
>
> RawTiffParser (TIKA-4851, TIKA-4861) extracts the camera-generated JPEG
> previews of
> TIFF-based raw formats: Nikon NEF/NRW, Sony ARW/SRF/SR2, Pentax PEF/PTX,
> Samsung SRW,
> Adobe DNG and Canon CR2. Three further TIFF-based formats yield nothing, each
> for a
> different reason.
> h3. 1. Olympus ORF
> Detected correctly as image/x-raw-olympus (magic entry in
> tika-mimetypes.xml), but no
> parser declares the type, and RawTiffParser would refuse the file anyway: ORF
> uses its
> own magic number (0x4F52 for "IIRO"/"MMOR", 0x5352 for "IIRS") where a TIFF
> has 42, so
> locateJpegPreviews() throws "bad TIFF magic number".
> On top of that the previews are not in the IFD chain. IFD0 holds the sensor
> image; the
> previews hang in the MakerNote, whose offsets are relative to its own first
> byte:
> {code}
> EXIF IFD (0x8769) -> MakerNote (0x927C), header "OLYMPUS\0II", its IFD at +12
> 0x0100 camera thumbnail (UNDEFINED, count = length)
> 0x2020 CameraSettings sub-IFD
> 0x0101/0x0102 PreviewImageStart / PreviewImageLength
> {code}
> Sample (raw.pixls.us, CC0, Olympus E-M10, P3180165.ORF): a 3200x2400 preview
> of 918873
> bytes and a 160x120 thumbnail.
> h3. 2. Panasonic RW2
> Detected correctly as image/x-raw-panasonic, same two blockers: no parser
> declares the
> type, and the magic number is 0x0055 ("IIU\0"). The preview sits in the
> private tag
> 0x002E (JpgFromRaw) of IFD0, while the standard strip tags are placeholders
> (StripOffsets = 0xFFFFFFFF, StripByteCounts = 0).
> Sample (raw.pixls.us, CC0, Panasonic DMC-GX7): a 1920x1920 preview of 926720
> bytes.
> h3. 3. Epson ERF
> Not recognized as a raw at all. RawTiffDetector has no Epson entry, so the
> file is
> detected as image/tiff and handled by TiffParser. Declaring the type does not
> help
> either, because image/x-raw-epson has no sub-class-of in tika-mimetypes.xml:
> MimeTypes
> only accepts a declared type that is a descendant of what it detected, so the
> hint is
> dropped. Measured against an Epson R-D1 file:
> {code}
> hint image/x-raw-epson -> detected image/tiff (hint discarded)
> hint image/x-raw-adobe -> detected image/x-raw-adobe (hint accepted: it is a
> descendant)
> {code}
> The raw IFD of that file has Compression 32769 (Epson's own code, as ExifTool
> lists it)
> and PhotometricInterpretation 32803 (CFA), and IFD0 carries Make "SEIKO EPSON
> CORP.",
> which is what RawTiffDetector already keys on for the other vendors. Note this
> particular sample carries no embedded JPEG at all, so it will stay without a
> thumbnail;
> the detection and the typing are wrong regardless, and ERFs that do carry a
> preview
> cannot be reached today.
> h3. Proposed fix
> * RawTiffDetector: Epson by compression code 32769 and by Make "SEIKO
> EPSON"/"EPSON".
> * tika-mimetypes.xml: sub-class-of image/tiff for image/x-raw-epson, so the
> type is a
> TIFF for the registry, a declared type can refine to it, and the parser
> lookup has
> something to fall back to.
> * RawTiffParser: accept the ORF and RW2 magic numbers as classic TIFF
> layouts; read the
> Panasonic JpgFromRaw tag; read the Olympus MakerNote (only the
> "OLYMPUS\0II" header,
> the older "OLYMP\0" one addresses differently) for the thumbnail and the
> preview; add
> image/x-raw-olympus, image/x-raw-panasonic and image/x-raw-epson to the
> supported
> types.
> * The existing rule picks the resource type: the largest preview by JPEG
> length becomes
> the THUMBNAIL, the smaller ones INLINE images.
> Verified against real files (raw.pixls.us samples, CC0):
> {code}
> P3180165.ORF (E-M10) image/x-raw-olympus THUMBNAIL 3200x2400
> INLINE 160x120
> _7061961_copy.ORF (E-5) image/x-raw-olympus THUMBNAIL 3200x2400
> INLINE 160x120
> RAW_PANASONIC_DMC-GX-7_11 image/x-raw-panasonic THUMBNAIL 1920x1920
> Lumix_DMC-LX100_1.RW2 image/x-raw-panasonic THUMBNAIL 1920x1440
> pixls-cc0-epson-rd1.erf image/x-raw-epson (no preview in the file)
> {code}
> Make, model and dimensions are extracted for all of them, and none records a
> parse
> exception. The tests use two hand-built files of 890 and 421 bytes, in the
> style of the
> existing raw test documents.
> Not covered: the older Olympus "OLYMP\0" MakerNote header, Fuji RAF (not a
> TIFF
> container), and the vendor-level mime entries that exist without a
> sub-class-of because
> they span several containers (image/x-raw-canon covers CRW, CR2 and CR3).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
