[
https://issues.apache.org/jira/browse/TIKA-4777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094958#comment-18094958
]
ASF GitHub Bot commented on TIKA-4777:
--------------------------------------
dschmidt opened a new pull request, #2936:
URL: https://github.com/apache/tika/pull/2936
JIRA: https://issues.apache.org/jira/browse/TIKA-4777
Stacked on #2935 (TIKA-2861), only the last commit is new here. Marked as
draft until that PR is merged; after a rebase only the TIKA-4777 commit
will remain.
QuickTime timed metadata tracks (`mebx` sample descriptions, ISO 14496-12
boxed metadata) declare their key names in the file, and a track's
presentation start is defined by its edit list: a leading empty edit
delays the media by its duration. This parses the mebx key declarations
(keys/keyd) and emits, for every declared key of a delayed track, the
presentation start in microseconds as `<key name>.track-start-us`.
Undelayed tracks are not reported, their start carries no information.
No key names are hardcoded. The main use case are Apple Live Photos,
which mark the moment the paired still image was captured as the single
one-tick sample of the `com.apple.quicktime.still-image-time` track,
shifted there by an empty edit. The sample value itself is a constant -1
marker, so the moov boxes alone are sufficient and mdat is never read.
Apple's own LivePhotosKit JS player derives its `photoTime` the same way
(it parses trak/elst/mebx client-side).
Verified against a real Live Photo video (iPhone 15 Pro, iOS 18.5):
`com.apple.quicktime.still-image-time.track-start-us = 1233333` (1.2333s,
matching the empty edit of 740/600s), likewise the transform keys
declared by the same track; the undelayed video-orientation and
live-photo-info tracks are not reported.
Testing: the crafted `testMP4_QuickTimeMetadata.mov` fixture gains a
timed metadata track with a proper mebx keys/keyd declaration and an
empty edit; `MP4ParserTest` asserts the emitted track start.
> Extract QuickTime timed metadata (mebx) tracks
> ----------------------------------------------
>
> Key: TIKA-4777
> URL: https://issues.apache.org/jira/browse/TIKA-4777
> Project: Tika
> Issue Type: Improvement
> Reporter: Dominik Schmidt
> Priority: Major
>
> Apple Live Photo videos mark the moment the paired still image was
> captured in a timed metadata track: the sample description declares the
> format 'mebx' with a keys structure containing
> com.apple.quicktime.still-image-time. The item value itself is always -1;
> the actual time is the presentation time of that sample, derived from the
> track's sample tables.
> Everything needed lives in the moov box, no mdat access is required:
> - mdhd: track timescale
> - stsd: detect the mebx sample entry and its still-image-time key
> - stts and elst: an empty edit shifts the single one-tick sample to the
> still moment
> Example from a real Live Photo video (iPhone 15 Pro, iOS 18.5): the
> still-image-time track has timescale 600, an edit list with an empty edit
> of 740 units followed by the sample, giving a still time of 1.2333 s.
> Neither the current MP4 handler (which reads the static keys/ilst
> metadata, TIKA-2861) nor the underlying metadata-extractor exposes timed
> metadata tracks; ExifTool only surfaces the constant -1 marker value.
> Note that the Apple maker note tag 0x0017 in the paired image
> (historically documented as "LivePhotoVideoIndex") does not encode a
> usable time on current iOS versions, so the video track is the only
> reliable source for this timestamp.
> Proposal: extend TikaMp4BoxHandler to accept the relevant trak child
> boxes, correlate mdhd/stsd/stts/elst per track, and emit the still time
> as metadata (e.g. com.apple.quicktime.still-image-time, in
> microseconds).
> Related: TIKA-2861 (QuickTime item-list metadata), TIKA-4776 (image-side
> Apple maker note, LONG8), TIKA-852.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
