Dominik Schmidt created TIKA-4980:
-------------------------------------
Summary: Extract previews from Olympus ORF, Panasonic RW2 and
Epson ERF
Key: TIKA-4980
URL: https://issues.apache.org/jira/browse/TIKA-4980
Project: Tika
Issue Type: Improvement
Components: parser
Reporter: Dominik Schmidt
RawTiffParser (TIKA-4851, TIKA-4861) extracts the camera-generated JPEG
previews of
TIFF-based raw formats: Nikon NEF/NRW, Sony ARW/SRF/SR2, Pentax PEF/PTX,
Samsung SRW,
Adobe DNG and Canon CR2. Three further TIFF-based formats yield nothing, each
for a
different reason.
h3. 1. Olympus ORF
Detected correctly as image/x-raw-olympus (magic entry in tika-mimetypes.xml),
but no
parser declares the type, and RawTiffParser would refuse the file anyway: ORF
uses its
own magic number (0x4F52 for "IIRO"/"MMOR", 0x5352 for "IIRS") where a TIFF has
42, so
locateJpegPreviews() throws "bad TIFF magic number".
On top of that the previews are not in the IFD chain. IFD0 holds the sensor
image; the
previews hang in the MakerNote, whose offsets are relative to its own first
byte:
{code}
EXIF IFD (0x8769) -> MakerNote (0x927C), header "OLYMPUS\0II", its IFD at +12
0x0100 camera thumbnail (UNDEFINED, count = length)
0x2020 CameraSettings sub-IFD
0x0101/0x0102 PreviewImageStart / PreviewImageLength
{code}
Sample (raw.pixls.us, CC0, Olympus E-M10, P3180165.ORF): a 3200x2400 preview of
918873
bytes and a 160x120 thumbnail.
h3. 2. Panasonic RW2
Detected correctly as image/x-raw-panasonic, same two blockers: no parser
declares the
type, and the magic number is 0x0055 ("IIU\0"). The preview sits in the private
tag
0x002E (JpgFromRaw) of IFD0, while the standard strip tags are placeholders
(StripOffsets = 0xFFFFFFFF, StripByteCounts = 0).
Sample (raw.pixls.us, CC0, Panasonic DMC-GX7): a 1920x1920 preview of 926720
bytes.
h3. 3. Epson ERF
Not recognized as a raw at all. RawTiffDetector has no Epson entry, so the file
is
detected as image/tiff and handled by TiffParser. Declaring the type does not
help
either, because image/x-raw-epson has no sub-class-of in tika-mimetypes.xml:
MimeTypes
only accepts a declared type that is a descendant of what it detected, so the
hint is
dropped. Measured against an Epson R-D1 file:
{code}
hint image/x-raw-epson -> detected image/tiff (hint discarded)
hint image/x-raw-adobe -> detected image/x-raw-adobe (hint accepted: it is a
descendant)
{code}
The raw IFD of that file has Compression 32769 (Epson's own code, as ExifTool
lists it)
and PhotometricInterpretation 32803 (CFA), and IFD0 carries Make "SEIKO EPSON
CORP.",
which is what RawTiffDetector already keys on for the other vendors. Note this
particular sample carries no embedded JPEG at all, so it will stay without a
thumbnail;
the detection and the typing are wrong regardless, and ERFs that do carry a
preview
cannot be reached today.
h3. Proposed fix
* RawTiffDetector: Epson by compression code 32769 and by Make "SEIKO
EPSON"/"EPSON".
* tika-mimetypes.xml: sub-class-of image/tiff for image/x-raw-epson, so the
type is a
TIFF for the registry, a declared type can refine to it, and the parser
lookup has
something to fall back to.
* RawTiffParser: accept the ORF and RW2 magic numbers as classic TIFF layouts;
read the
Panasonic JpgFromRaw tag; read the Olympus MakerNote (only the "OLYMPUS\0II"
header,
the older "OLYMP\0" one addresses differently) for the thumbnail and the
preview; add
image/x-raw-olympus, image/x-raw-panasonic and image/x-raw-epson to the
supported
types.
* The existing rule picks the resource type: the largest preview by JPEG length
becomes
the THUMBNAIL, the smaller ones INLINE images.
Verified against real files (raw.pixls.us samples, CC0):
{code}
P3180165.ORF (E-M10) image/x-raw-olympus THUMBNAIL 3200x2400 INLINE
160x120
_7061961_copy.ORF (E-5) image/x-raw-olympus THUMBNAIL 3200x2400 INLINE
160x120
RAW_PANASONIC_DMC-GX-7_11 image/x-raw-panasonic THUMBNAIL 1920x1920
Lumix_DMC-LX100_1.RW2 image/x-raw-panasonic THUMBNAIL 1920x1440
pixls-cc0-epson-rd1.erf image/x-raw-epson (no preview in the file)
{code}
Make, model and dimensions are extracted for all of them, and none records a
parse
exception. The tests use two hand-built files of 890 and 421 bytes, in the
style of the
existing raw test documents.
Not covered: the older Olympus "OLYMP\0" MakerNote header, Fuji RAF (not a TIFF
container), and the vendor-level mime entries that exist without a sub-class-of
because
they span several containers (image/x-raw-canon covers CRW, CR2 and CR3).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
