Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 7372c4293daa59212c8c34bc27da8ec9f6d3251b
https://github.com/WebKit/WebKit/commit/7372c4293daa59212c8c34bc27da8ec9f6d3251b
Author: Jean-Yves Avenard <[email protected]>
Date: 2024-12-02 (Mon, 02 Dec 2024)
Changed paths:
M LayoutTests/http/tests/media/media-source/mediasource-rvfc.html
M LayoutTests/media/media-rvfc-playing-webm.html
A LayoutTests/media/media-source/media-source-rvfc-paused-expected.txt
A
LayoutTests/media/media-source/media-source-rvfc-paused-offscreen-expected.txt
A LayoutTests/media/media-source/media-source-rvfc-paused-offscreen.html
A LayoutTests/media/media-source/media-source-rvfc-paused.html
A LayoutTests/media/media-source/media-source-rvfc-playing-expected.txt
A
LayoutTests/media/media-source/media-source-rvfc-playing-offscreen-expected.txt
A LayoutTests/media/media-source/media-source-rvfc-playing-offscreen.html
A LayoutTests/media/media-source/media-source-rvfc-playing.html
M LayoutTests/media/media-source/media-source-vp8-webm-error-offscreen.html
M LayoutTests/platform/glib/TestExpectations
M LayoutTests/platform/mac/TestExpectations
M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml
M Source/WebCore/html/HTMLMediaElement.cpp
M Source/WebCore/platform/graphics/MediaPlayer.cpp
M Source/WebCore/platform/graphics/MediaPlayer.h
M Source/WebCore/platform/graphics/MediaPlayerPrivate.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h
M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm
M Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp
Log Message:
-----------
requestVideoFrameCallback is never called when video element is paused or
non-visible
https://bugs.webkit.org/show_bug.cgi?id=282782
rdar://139473124
Reviewed by Jer Noble.
In continuation of bug 283360, and similar to the work done on
MediaPlayerPrivateWebM we stop using
directly a WebCoreDecompressionSession whenever the video element isn't
displayed or is not in the DOM.
Instead we will use the new capabilities of the VideoMediaSampleRenderer to
track and provide the
currently displayed video frames as needed.
We introduce two new preferences (both disabled by default) to tune this
behaviour.
- MediaSourcePrefersDecompressionSession: if set, the VideoMediaSampleRenderer
will always use a WebCoreDecompressionSession
to perform the decoding if it is possible (e.g content isn't encrypted). With
this is turned on, the
VideoMediaSampleRenderer will first decode the video frame and then enqueue it
to the AVSampleBufferDisplayLayer.
- MediaSourceCanFallbackToDecompressionSession: if set, the
VideoMediaSampleRenderer will only use a WebCoreDecompressionSession
if it's used in combination with the requestVideoFrameCallback API. As soon as
the HTMLMediaElement requests to capture the video
frame metadata, we will turn on decoding independently of the
AVSampleBufferDisplayLayer.
With those preferences off, the behaviour is close to identical to the previous
one: just like we did with
the old WebCoreDecompressionSession, we will tear down the existing
AVSampleBufferDisplayRenderer or AVSampleBufferVideoRenderer (visionOS)
whenever the video element is not or removed from the DOM.
Added new tests checking how rVFC behaves with a MSE element, being paused or
playing, visible or offscreen.
Also re-enabling tests that used to fail.
Covered by other existing tests.
* LayoutTests/http/tests/media/media-source/mediasource-rvfc.html:
* LayoutTests/media/media-rvfc-playing-webm.html:
* LayoutTests/media/media-source/media-source-rvfc-paused-expected.txt: Added.
*
LayoutTests/media/media-source/media-source-rvfc-paused-offscreen-expected.txt:
Added.
* LayoutTests/media/media-source/media-source-rvfc-paused-offscreen.html: Added.
* LayoutTests/media/media-source/media-source-rvfc-paused.html: Added.
* LayoutTests/media/media-source/media-source-rvfc-playing-expected.txt: Added.
*
LayoutTests/media/media-source/media-source-rvfc-playing-offscreen-expected.txt:
Added.
* LayoutTests/media/media-source/media-source-rvfc-playing-offscreen.html:
Copied from LayoutTests/media/media-rvfc-playing-webm.html.
* LayoutTests/media/media-source/media-source-rvfc-playing.html: Copied from
LayoutTests/media/media-rvfc-playing-webm.html.
* LayoutTests/media/media-source/media-source-vp8-webm-error-offscreen.html:
* LayoutTests/platform/glib/TestExpectations:
* LayoutTests/platform/mac/TestExpectations:
* LayoutTests/platform/ios/TestExpectations:
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WebCore/html/HTMLMediaElement.cpp:
* Source/WebCore/platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::setDecompressionSessionPreferences):
(WebCore::MediaPlayer::loadWithNextMediaEngine):
* Source/WebCore/platform/graphics/MediaPlayer.h:
* Source/WebCore/platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::setDecompressionSessionPreferences):
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.h:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::decompressionSession const):
Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::hasVideoRenderer const):
Deleted.
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaPlayerPrivateMediaSourceAVFObjC.mm:
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::~MediaPlayerPrivateMediaSourceAVFObjC):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::load):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateLastPixelBuffer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::shouldEnsureLayerOrVideoRenderer
const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setPresentationSize):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVideoLayerSizeFenced):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::acceleratedRenderingStateChanged):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateDisplayLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::videoPlaybackQualityMetrics):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayerOrVideoRendererAndCreateRenderlessVideoMediaSampleRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::configureLayerOrVideoRenderer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::createVideoMediaSampleRendererForRendererer):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::canUseDecompressionSession
const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::isUsingDecompressionSession
const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::willUseDecompressionSessionIfNeeded
const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::cdmInstanceAttached):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::cdmInstanceDetached):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::needsVideoLayerChanged):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::startVideoFrameMetadataGathering):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::checkNewVideoFrameMetadata):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::stopVideoFrameMetadataGathering):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::playerContentBoxRectChanged):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::hasVideoRenderer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::layerOrVideoRenderer const):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::setVideoTarget):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::isInFullscreenOrPictureInPictureChanged):
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::updateDisplayLayerAndDecompressionSession):
Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::ensureDecompressionSession):
Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyDecompressionSession):
Deleted.
(WebCore::MediaPlayerPrivateMediaSourceAVFObjC::destroyLayerOrVideoRenderer):
Deleted.
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/MediaSourcePrivateAVFObjC.mm:
(WebCore::MediaSourcePrivateAVFObjC::setVideoRenderer):
(WebCore::MediaSourcePrivateAVFObjC::stageVideoRenderer):
(WebCore::MediaSourcePrivateAVFObjC::videoRendererWillReconfigure):
(WebCore::MediaSourcePrivateAVFObjC::videoRendererDidReconfigure):
(WebCore::MediaSourcePrivateAVFObjC::setSourceBufferWithSelectedVideo):
(WebCore::MediaSourcePrivateAVFObjC::setDecompressionSession): Deleted.
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.h:
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::destroyRenderers):
(WebCore::SourceBufferPrivateAVFObjC::trackDidChangeSelected):
(WebCore::SourceBufferPrivateAVFObjC::setCDMInstance):
(WebCore::SourceBufferPrivateAVFObjC::flushIfNeeded):
(WebCore::SourceBufferPrivateAVFObjC::flushVideo):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSample):
(WebCore::SourceBufferPrivateAVFObjC::enqueueSampleBuffer):
(WebCore::SourceBufferPrivateAVFObjC::isReadyForMoreSamples):
(WebCore::SourceBufferPrivateAVFObjC::didBecomeReadyForMoreSamples):
(WebCore::SourceBufferPrivateAVFObjC::notifyClientWhenReadyForMoreSamples):
(WebCore::SourceBufferPrivateAVFObjC::configureVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::invalidateVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::setVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::stageVideoRenderer):
(WebCore::SourceBufferPrivateAVFObjC::videoRendererWillReconfigure):
(WebCore::SourceBufferPrivateAVFObjC::videoRendererDidReconfigure):
(WebCore::SourceBufferPrivateAVFObjC::setDecompressionSession): Deleted.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h:
(WebCore::VideoMediaSampleRenderer::prefersDecompressionSession const):
(WebCore::VideoMediaSampleRenderer::prefersDecompressionSession): Deleted.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm:
(WebCore::VideoMediaSampleRenderer::isUsingDecompressionSession const):
(WebCore::VideoMediaSampleRenderer::displayLayer const): Deleted.
* Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::loadMediaSource):
Canonical link: https://commits.webkit.org/287259@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
