Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 93e7a14ff5bc2aad775dd5b2d53cc67499adac10
https://github.com/WebKit/WebKit/commit/93e7a14ff5bc2aad775dd5b2d53cc67499adac10
Author: Jean-Yves Avenard <[email protected]>
Date: 2026-05-14 (Thu, 14 May 2026)
Changed paths:
A
LayoutTests/media/media-source/media-source-append-b-frame-tail-overshoot-expected.txt
A
LayoutTests/media/media-source/media-source-append-b-frame-tail-overshoot.html
M Source/WebCore/Modules/mediasource/SampleMap.cpp
M Source/WebCore/Modules/mediasource/SampleMap.h
M Source/WebCore/platform/MediaSample.h
M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
M Source/WebCore/platform/graphics/SourceBufferPrivate.h
M Source/WebCore/platform/graphics/TrackBuffer.cpp
M Source/WebCore/platform/graphics/TrackBuffer.h
M Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h
M Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm
M Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.cpp
M Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.h
M Source/WebCore/platform/mock/mediasource/MockBox.h
M Source/WebCore/platform/mock/mediasource/MockSourceBufferPrivate.cpp
Log Message:
-----------
Twitch.tv may stall at ad transition during live streaming.
https://bugs.webkit.org/show_bug.cgi?id=314306
rdar://176446372
Reviewed by Jer Noble
Avoid buffered-range gap when a reordered B-frame tail slightly overshoots
the next buffered sample
To play an ad, Twitch will insert video content over an existing MediaSource.
The last video frame's timestamp of the ad is then adjusted to provide
continuous playback with the live content.
At the tail of such fMP4 stream, sometimes the last frame to be visible is
a b-frame (pts > dts); the trun.sample_duration of this last-in-
presentation-order sample is a decode-to-next-decode placeholder rather
than a real presentation duration as there's no sample to present after.
The sample declared frame_end (pts + duration) can overshoot the
next buffered sample's pts by a few milliseconds — not a true overlap,
just a carryover from the decode grid.
Taken at face value, MSE "Coded Frame Processing" step 1.14 treats the
overshoot as a real overlap and removes the overlapped frame, and step
1.15 extends that removal in dts order up to the next sync sample. The
result is a gap in the buffered range that stalls playback. Observed
repeatedly at Twitch content-to-ad splice boundaries on Safari and
Firefox (Chrome tolerates the overshoot so does not stall).
Fix: if the per-track presentation tail is a B-frame and its overshoot
of the next buffered sample's pts is less than timeFudgeFactor(), treat
the overshoot as placeholder noise and shift the overlapped sample
forward by the overshoot — pts and dts shift equally, duration shrinks,
presentationEndTime is preserved. Larger overshoots are kept on the
default path.
Implementation:
- Add MediaSample::createCopyWithAdjustedStartTime(const MediaTime& offset):
new virtual producing an immutable copy with pts/dts shifted forward
by `offset` (clamped to duration) and duration correspondingly
shrunk. Overridden in MediaSampleAVFObjC (via
CMSampleBufferCreateCopyWithNewTiming, applying the offset to each
sub-sample's timing so multi-sample buffers are preserved),
MediaSampleGStreamer (via gst_buffer_copy + timestamp update on the
copy to avoid corrupting the original's shared GstBuffer), and
MockMediaSample.
- Add TrackBuffer::adjustSampleStartTime(original, offset): creates the
adjusted copy via createCopyWithAdjustedStartTime, subtracts the
original's [pts, presentationEndTime) from m_buffered, swaps the
sample in the SampleMap and m_decodeQueue (if present), and re-adds
the adjusted range. Keeping m_decodeQueue in sync with the SampleMap
is required so a subsequent step-1.14/1.15 removeSamples cascade —
which erases m_decodeQueue by the current decode key — matches the
adjusted entry rather than leaving the pre-adjustment key behind as
an orphan.
- Add SampleMap::replaceSample: swap an already-buffered sample for an
adjusted copy carrying nearly-identical keys. Uses erase() + insert
with the erase-return iterator as the insertion hint in both the
presentation-order and decode-order submaps, keeping the operation
to two map ops rather than two full find+erase+insert cycles.
- In SourceBufferPrivate::processPendingMediaSamples, compute the
per-track presentation tail (sample with the highest
presentationEndTime in each track) and pass isPresentationTail to
processMediaSample. Per-track keying is required: an audio sample's
presentationEndTime often exceeds the last video sample's, and a
global-max tail would deny the video tail its shift-eligibility.
- In processMediaSample, inside the step-1.14 overlap handling: if
the incoming sample is isPresentationTail and reordered (pts > dts),
search the buffered samples whose pts lies in (presentationTimestamp,
frameEndTimestamp) and within timeFudgeFactor of frameEndTimestamp;
if one is found and is not fully overwritten by the incoming frame,
shift it forward via TrackBuffer::adjustSampleStartTime. Fully-
overwritten samples are left to the default 1.14 removal.
Test: media/media-source/media-source-append-b-frame-tail-overshoot.html
*
LayoutTests/media/media-source/media-source-append-b-frame-tail-overshoot-expected.txt:
Added.
*
LayoutTests/media/media-source/media-source-append-b-frame-tail-overshoot.html:
Added.
Canonical link: https://commits.webkit.org/313253@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
