Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 2fcceeb9c6b0ef543e182c5dc28e64dae6281314
https://github.com/WebKit/WebKit/commit/2fcceeb9c6b0ef543e182c5dc28e64dae6281314
Author: Jean-Yves Avenard <[email protected]>
Date: 2026-04-01 (Wed, 01 Apr 2026)
Changed paths:
M LayoutTests/ipc/serialized-type-info.html
A
LayoutTests/media/media-source/media-source-seek-during-pending-seek-expected.txt
A LayoutTests/media/media-source/media-source-seek-during-pending-seek.html
M Source/WTF/wtf/NativePromise.h
M Source/WebCore/platform/PlatformMediaError.cpp
M Source/WebCore/platform/PlatformMediaError.h
M Source/WebCore/platform/graphics/AudioVideoRenderer.h
M Source/WebCore/platform/graphics/MediaSourcePrivate.cpp
M Source/WebCore/platform/graphics/MediaSourcePrivate.h
M Source/WebCore/platform/graphics/SourceBufferPrivate.cpp
M Source/WebCore/platform/graphics/SourceBufferPrivate.h
M Source/WebCore/platform/graphics/avfoundation/AudioVideoRendererAVFObjC.h
M Source/WebCore/platform/graphics/avfoundation/AudioVideoRendererAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h
M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm
M
Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp
M
Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp
M
Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h
M
Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp
M
Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h
M Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp
M Source/WebKit/GPUProcess/media/RemoteAudioVideoRendererProxyManager.cpp
M Source/WebKit/GPUProcess/media/RemoteAudioVideoRendererProxyManager.h
M
Source/WebKit/GPUProcess/media/RemoteAudioVideoRendererProxyManager.messages.in
M Source/WebKit/Platform/IPC/Connection.h
M Source/WebKit/Scripts/webkit/messages.py
M Source/WebKit/Shared/WTFArgumentCoders.serialization.in
M Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in
M Source/WebKit/WebProcess/GPU/media/AudioVideoRendererRemote.cpp
M Source/WebKit/WebProcess/GPU/media/AudioVideoRendererRemote.h
M Source/WebKit/WebProcess/GPU/media/RemoteAudioVideoRendererState.h
M
Source/WebKit/WebProcess/GPU/media/RemoteAudioVideoRendererState.serialization.in
M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp
M Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h
Log Message:
-----------
Scrubbing a YouTube video in full-screen causes it to exit full-screen mode.
https://bugs.webkit.org/show_bug.cgi?id=311100
rdar://172682230
Reviewed by Jer Noble.
There were two issues at play:
1- The seeking code relied on a key assumption ; the AudioVideoRenderer will
be flushed and then new samples will be enqueued serially during a seeking
operation:
- No new flush will be issued while samples are being enqueued.
This guarantees that after a flush, the first sample enqueued to the renderer
will always be a keyframe.
When a new seek is started before the prior one completed, it was assumed that
the second time
MediaSourcePrivate::waitForTarget was called, any pending promise would be
rejected by the MediaSource
which would stop the processing flow of the first seek.
However, the MediaSourcePrivate::waitForTarget isn't synchronous and isn't
atomic; it will always
take at least two event loop to complete.
It was possible when the new seek came, that the previous waitForTarget promise
had been resolved
and was already in-flight. Once received by the
MediaPlayerPrivateMediaSourceAVFObjC
it would then seek the AudioVideoRenderer and then enqueue new frames on the
SourceBuffer's workqueue.
The 2nd call to MediaSourcePrivate::waitForTarget would then resolve, and also
start
seeking the renderer and once again flush the renderer; all while the
SourceBuffer is in the
process on enqueueing new samples.
We properly cancel the MediaSourcePrivate::waitForTarget current operation by
using
a NativePromiseRequest to track the completion, and disconnect is as needed.
2- The flush operation was started on the main thread while the samples
re-enqueueing was occurring
on the SourceBuffer, the tasks dispatched to flush the renderer and re-enqueue
the content could run
out of order.
We restructure the seek operations and
MediaPlayerPrivateMediaSourceAVFObjC::startSeek() was restructured
to not require the inner promise resolver to call back into startSeek() again.
AudioVideoRenderer prepareToSeek(MediaTime) , now takes the seek time and
return a MediaTimePromise.
If no seeking is required by the AudioVideoRenderer's synchronizer, then the
promise
is resolved with the current time value to indicate the caller can short
circuit and finish the seek early.
Otherwise the promise is resolved with MediaTime::isIndefinite() which
indicates that samples need
to be re-enqueued following a flush on their respective tracks.
The AudioVideoRenderer no longer needs to be fully flushed; instead all current
tracks
much be individually flushed for the seek operation to continue.
The new flow is more asynchronous and parallelise two tasks:
- m_mediaSourcePrivate)->reenqueueMediaForTime(seekTime),
- m_renderer->finishSeek(seekTime)
so that the renderer can start waiting for the first sample decoding to
complete before all frames have been enqueued.
It also allows the seek to be more easily interrupted and another seek started.
We now track the seek state from within AudioVideoRendereRemote as
AudioVideoRendererRemote::seeking() could
have returned a stale value. It also allows to better assert that the API usage
of seeking with the AudioVideoRenderer
is correct.
Fly-By: the m_rendererSeekRequest wasn't always disconnected/completed should
an error occur
(such as the GPU process crashing), which would have caused an assertion on
debug build.
We add some assertions and logging to more easily detect should the first frame
received after a flush not be a keyframe.
Added test.
* LayoutTests/ipc/serialized-type-info.html:
*
LayoutTests/media/media-source/media-source-seek-during-pending-seek-expected.txt:
Added.
* LayoutTests/media/media-source/media-source-seek-during-pending-seek.html:
Added.
* Source/WTF/wtf/NativePromise.h:
(WTF::GenericPromiseConverter::convertError): Add converter to send a
GenericPromise over IPC.
* Source/WebCore/platform/PlatformMediaError.cpp:
(WebCore::convertEnumerationToString):
* Source/WebCore/platform/PlatformMediaError.h:
* Source/WebCore/platform/graphics/AudioVideoRenderer.h:
(WebCore::SynchronizerInterface::prepareToSeek): Deleted.
* Source/WebCore/platform/graphics/MediaSourcePrivate.cpp:
(WebCore::MediaSourcePrivate::waitForTarget):
(WebCore::MediaSourcePrivate::reenqueueMediaForTime):
(WebCore::MediaSourcePrivate::seekToTime): Deleted.
* Source/WebCore/platform/graphics/MediaSourcePrivate.h:
* Source/WebCore/platform/graphics/SourceBufferPrivate.cpp:
(WebCore::SourceBufferPrivate::reenqueueMediaForTime):
(WebCore::SourceBufferPrivate::isReenqueuePending const):
(WebCore::SourceBufferPrivate::provideMediaData):
(WebCore::SourceBufferPrivate::attach):
(WebCore::SourceBufferPrivate::seekToTime): Deleted.
* Source/WebCore/platform/graphics/SourceBufferPrivate.h:
(WebCore::SourceBufferPrivate::timeFudgeFactor const):
(WebCore::SourceBufferPrivate::isSeeking const): Deleted.
* Source/WebCore/platform/graphics/avfoundation/AudioVideoRendererAVFObjC.h:
* Source/WebCore/platform/graphics/avfoundation/AudioVideoRendererAVFObjC.mm:
(WebCore::AudioVideoRendererAVFObjC::enqueueSample):
(WebCore::AudioVideoRendererAVFObjC::isReadyForMoreSamples):
(WebCore::AudioVideoRendererAVFObjC::requestMediaDataWhenReady):
(WebCore::AudioVideoRendererAVFObjC::flush):
(WebCore::AudioVideoRendererAVFObjC::flushTrack):
(WebCore::AudioVideoRendererAVFObjC::setTimeObserver):
(WebCore::AudioVideoRendererAVFObjC::prepareToSeek):
(WebCore::AudioVideoRendererAVFObjC::finishSeek):
(WebCore::AudioVideoRendererAVFObjC::maybeCompleteSeek):
(WebCore::AudioVideoRendererAVFObjC::flushVideo):
(WebCore::AudioVideoRendererAVFObjC::flushAudio):
(WebCore::AudioVideoRendererAVFObjC::flushAudioTrack):
(WebCore::AudioVideoRendererAVFObjC::cancelSeekingPromiseIfNeeded):
(WebCore::AudioVideoRendererAVFObjC::toString const):
(WebCore::AudioVideoRendererAVFObjC::seekTo): Deleted.
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::~MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::seekInternal):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::continueSeek):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::reenqueueMediaForTimeAndFinishSeek):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::cancelPendingSeek):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::startSeek):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::flush): Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::reenqueueMediaForTime): Deleted.
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.h:
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm:
(WebCore::MediaPlayerPrivateWebM::seekInternal):
(WebCore::MediaPlayerPrivateWebM::reenqueueMediaForTime):
(WebCore::MediaPlayerPrivateWebM::startSeek): Deleted.
*
Source/WebCore/platform/graphics/gstreamer/mse/MediaPlayerPrivateGStreamerMSE.cpp:
(WebCore::MediaPlayerPrivateGStreamerMSE::doSeek):
*
Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.cpp:
(WebCore::MediaSourcePrivateGStreamer::willSeek): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/MediaSourcePrivateGStreamer.h:
*
Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.cpp:
(WebCore::SourceBufferPrivateGStreamer::willSeek): Deleted.
(WebCore::SourceBufferPrivateGStreamer::isSeeking const): Deleted.
(WebCore::SourceBufferPrivateGStreamer::seekToTime): Deleted.
* Source/WebCore/platform/graphics/gstreamer/mse/SourceBufferPrivateGStreamer.h:
* Source/WebCore/platform/mock/mediasource/MockMediaPlayerMediaSource.cpp:
(WebCore::MockMediaPlayerMediaSource::seekToTarget):
* Source/WebKit/GPUProcess/media/RemoteAudioVideoRendererProxyManager.cpp:
(WebKit::RemoteAudioVideoRendererProxyManager::prepareToSeek):
(WebKit::RemoteAudioVideoRendererProxyManager::finishSeek):
(WebKit::RemoteAudioVideoRendererProxyManager::stateFor const):
(WebKit::RemoteAudioVideoRendererProxyManager::seekTo): Deleted.
* Source/WebKit/GPUProcess/media/RemoteAudioVideoRendererProxyManager.h:
*
Source/WebKit/GPUProcess/media/RemoteAudioVideoRendererProxyManager.messages.in:
* Source/WebKit/Platform/IPC/Connection.h:
* Source/WebKit/Scripts/webkit/messages.py:
(types_that_cannot_be_forward_declared):
(headers_for_type):
* Source/WebKit/Shared/WTFArgumentCoders.serialization.in:
* Source/WebKit/Shared/WebCoreArgumentCoders.serialization.in:
* Source/WebKit/WebProcess/GPU/media/AudioVideoRendererRemote.cpp:
(WebKit::AudioVideoRendererRemote::AudioVideoRendererRemote):
(WebKit::AudioVideoRendererRemote::cancelPendingSeek):
(WebKit::AudioVideoRendererRemote::prepareToSeek):
(WebKit::AudioVideoRendererRemote::finishSeek):
(WebKit::AudioVideoRendererRemote::seeking const):
(WebKit::AudioVideoRendererRemote::enqueueSample):
(WebKit::AudioVideoRendererRemote::flush):
(WebKit::AudioVideoRendererRemote::~AudioVideoRendererRemote): Deleted.
(WebKit::AudioVideoRendererRemote::seekTo): Deleted.
* Source/WebKit/WebProcess/GPU/media/AudioVideoRendererRemote.h:
* Source/WebKit/WebProcess/GPU/media/RemoteAudioVideoRendererState.h:
*
Source/WebKit/WebProcess/GPU/media/RemoteAudioVideoRendererState.serialization.in:
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.cpp:
(WebKit::SourceBufferPrivateRemote::reenqueueMediaForTime):
(WebKit::SourceBufferPrivateRemote::seekToTime): Deleted.
* Source/WebKit/WebProcess/GPU/media/SourceBufferPrivateRemote.h:
Canonical link: https://commits.webkit.org/310415@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
