Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 9d7e16e79a52be192a09a6135eb479e00d653e73
https://github.com/WebKit/WebKit/commit/9d7e16e79a52be192a09a6135eb479e00d653e73
Author: Jean-Yves Avenard <[email protected]>
Date: 2026-05-15 (Fri, 15 May 2026)
Changed paths:
A
LayoutTests/media/media-source/media-source-append-bframe-orphan-post-reset-expected.txt
A
LayoutTests/media/media-source/media-source-append-bframe-orphan-post-reset.html
A
LayoutTests/media/media-source/media-source-append-overlapping-equal-decode-key-expected.txt
A
LayoutTests/media/media-source/media-source-append-overlapping-equal-decode-key.html
A
LayoutTests/media/media-source/media-source-append-pt-collision-tail-expected.txt
A LayoutTests/media/media-source/media-source-append-pt-collision-tail.html
A
LayoutTests/media/media-source/media-source-coded-frame-group-reset-overlapping-sample-orphan-expected.txt
A
LayoutTests/media/media-source/media-source-coded-frame-group-reset-overlapping-sample-orphan.html
A
LayoutTests/media/media-source/media-source-pt-collision-cleanup-expected.txt
A LayoutTests/media/media-source/media-source-pt-collision-cleanup.html
M LayoutTests/media/media-source/media-source-video-playback-quality.html
M LayoutTests/platform/glib/TestExpectations
M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
M Source/WebCore/platform/graphics/TrackBuffer.cpp
Log Message:
-----------
[MSE] processMediaSample: close Coded Frame Processing spec gaps proposed in
w3c/media-source#374 and #375
https://bugs.webkit.org/show_bug.cgi?id=314723
rdar://176971800
Reviewed by OOPs
The "Coded Frame Processing" algorithm in the MSE spec has two cleanup
gaps when an incoming coded frame arrives mid-stream. Each is tracked
as a proposed spec amendment:
- w3c/media-source#374 (SAP Type 2 decode-shadowed orphans).
Non-sync coded frames whose decode timestamp is strictly greater
than the incoming frame's decode timestamp and whose presentation
timestamp is less than the incoming frame's survive every existing
cleanup step. They decode after the incoming random access point
but present before it, and no random access point remains at or
before their presentation timestamps once the incoming one takes
over. They become unreachable on seek.
- w3c/media-source#375 (presentation-timestamp coincidence).
Coded frames whose presentation timestamp coincides (exactly, or
within 1 microsecond) with the incoming frame's can coexist in the
track buffer with the incoming frame, producing downstream
ambiguity in algorithms that refer to "the coded frame in track
buffer with a presentation interval that contains t".
Place the #375 amendment between step 1.13 ("overlap check") and step
1.14 ("Remove existing coded frames in track buffer"):
Look up the presentation-order range
[incoming.pts - 1us, incoming.pts + 1us)
via findSamplesBetweenPresentationTimes (O(log n)) and add every
sample it yields to erasedSamples. Runs unconditionally: a mid-
stream PT collision can reach "Add the coded frame" with all other
cleanup steps no-opping, so #375's lookup must not be gated on
erasedSamples being non-empty.
Place the #374 amendment inside step 1.15's existing "Remove all
possible decoding dependencies" block, after the main dependent sweep:
Call findSamplesBetweenDecodeKeys(incomingDecodeKey,
erasedSamples.decodeOrder().begin()->first) and, for each returned
sample, insert into dependentSamples if it is non-sync and its
presentation timestamp is less than the incoming frame's.
Using erasedSamples.begin()->first as the upper bound is safe:
samples in [erasedSamples.begin(), nextSyncIter) are already
scheduled for removal by the step 1.15 main sweep immediately
above (dependentSamples.insert(firstDecodeIter, nextSyncIter)),
so #374 only needs to cover the gap [incomingDecodeKey,
erasedSamples.begin()). The amendment filed on #374 includes an
optimization note permitting implementations to bound the scan
short of the next random access point; this bound satisfies the
note, and the predicate (non-sync AND pts < incoming.pts) keeps
sync frames and samples with pts >= incoming.pts preserved.
Remove the samplesWithHigherDecodeTimes cleanup previously located
inside step 1.15's dependent sweep: its responsibilities are now
covered by #375 (equal-key collision, equal-PT collision) and by the
Simplify the step-1.14 B-frame cascade's DTS re-key guard. The
three-predicate condition
nextSampleInDecodeOrderRef->presentationTime() < sample->presentationTime()
&& nextSyncSample != trackBuffer.samples().decodeOrder().begin()
&& trackBuffer.samples().decodeOrder().size() > 0
collapses to its semantic predicate
nextSampleInDecodeOrderRef->presentationTime() < sample->presentationTime()
because size() > 0 is implied by the earlier early-break on
nextSampleInDecodeOrder == decodeOrder().end(), and nextSyncSample
can never equal begin() because findSyncSampleAfterDecodeIterator
pre-increments its argument before scanning (SampleMap.cpp:
std::find_if(++currentSampleDTS, end(), ...)). The invariant is
documented inline.
*
LayoutTests/media/media-source/media-source-append-bframe-orphan-post-reset-expected.txt:
Added.
*
LayoutTests/media/media-source/media-source-append-bframe-orphan-post-reset.html:
Added.
Real-MP4 regression guard for #374. Seg 1 is a B-frame GOP anchored
past t=1s (so seg 2's non-zero DTS avoids
Source/WebCore/platform/graphics/gstreamer/mse/AppendPipeline.cpp's
extendToTheBeginning hack, which fires on DTS==0 && PTS>0 && sync);
seg 2's single sync at DTS=1.000 PTS=1.075 lands inside an existing
B-frame's presentation interval. With the new passes: the legitimate
next-GOP I-frame at (1.050, 1.100) is preserved (pts > incoming.pts);
the orphan B-frame at (1.150, 1.025) is evicted (pts < incoming.pts,
non-sync); final buffered length = 3.
*
LayoutTests/media/media-source/media-source-append-overlapping-equal-decode-key-expected.txt:
Added.
*
LayoutTests/media/media-source/media-source-append-overlapping-equal-decode-key.html:
Added.
Real-MP4 regression guard for #375. Seg 2 is split into two appends
with independent baseDecodeTime so the setup samples push highestPT
past 200, leaving the #375 presentation-timestamp collision cleanup
as the only code path that can evict the existing sample at DTS=200
when seg 2B's incoming sync at DTS=200 is processed. The sample in
the buffer at DTS=0.200 must carry seg 2B's duration (120), not
seg 1's (100).
*
LayoutTests/media/media-source/media-source-coded-frame-group-reset-overlapping-sample-orphan-expected.txt:
Added.
*
LayoutTests/media/media-source/media-source-coded-frame-group-reset-overlapping-sample-orphan.html:
Added.
Step 1.6 + step 1.14 cascade DTS re-key-decline + fallthrough-erase
coverage. Timestamps shifted past t=1s so seg 2's DTS is non-zero,
avoiding the GStreamer extendToTheBeginning hack. Seg 1 is a single
sync at baseDecodeTime=1100 dur=100; seg 2 is a single sync at
baseDecodeTime=1000 dur=50 CTO=150 (PTS=1150 inside seg 1's interval
[1100, 1200)). Cascade's DTS re-key declines because the candidate
safeDecodeTime lands past the incoming frame's duration window, and
the fallthrough addRange erases the overlapping seg 1 sample.
Buffered collapses to [1.15, 1.2].
* LayoutTests/media/media-source/media-source-video-playback-quality.html: the
test actually exposed bug #375
it added two samples with the same key pts/dts and would have been silently
dropped upon adding to the SampleMap.
* LayoutTests/platform/glib/TestExpectations:
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::processMediaSample):
Inserted the #375 unconditional pass between step 1.13 and step 1.14;
inserted the #374 bounded pass inside step 1.15's dependent sweep;
removed the obsolete samplesWithHigherDecodeTimes block; simplified
the B-frame cascade's DTS re-key guard.
Canonical link: https://commits.webkit.org/313296@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
