Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 7611df14b62fd93b69613fe5c5e411b1e9b7ab30
https://github.com/WebKit/WebKit/commit/7611df14b62fd93b69613fe5c5e411b1e9b7ab30
Author: Jer Noble <[email protected]>
Date: 2024-11-04 (Mon, 04 Nov 2024)
Changed paths:
M Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.h
M Source/WebCore/platform/graphics/avfoundation/objc/MediaSampleAVFObjC.mm
M
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm
M Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm
M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h
M Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm
M Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.h
M Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm
M Source/WebCore/platform/libwebrtc/LibWebRTCVPXVideoDecoder.cpp
M Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.cpp
M Source/WebCore/platform/mediastream/libwebrtc/VideoFrameLibWebRTC.h
Log Message:
-----------
Refactor VideoMediaSampleRenderer to support enqueuing decoded samples
https://bugs.webkit.org/show_bug.cgi?id=281497
rdar://137964106
Reviewed by Jean-Yves Avenard.
Currently, WebCoreDecompressionSession maintains a queue of decoded images.
However,
when WebCoreDecompressionSession is used in a mode where those decoded images
are
delivered to its clients, the queue is bypassed. VideoMediaSampleRenderer is a
more
natural place for such a cue to belong, and eventually, all queueing behavior
should
be migrated from WebCoreDecompressionSession to VideoMediaSampleRenderer. As a
first
step, expose the promise-based WebCoreDecompressionSession::enqueueSample()
function,
and expose a static method to create a CoreMedia buffer queue. Use these
primitives
inside VideoMediaSampleRenderer to enqueue decoded images for retrieval by
clients
later.
In this mode of operation, a VideoMediaSampleRenderer will feed decoded images
to
its associated AVSampleBufferDisplayLayer or AVSampleBufferVideoRenderer, and
store
those images in a queue for retrieval via copyDisplayedPixelBuffer(). This
requires
a CMTimebaseRef, and to ensure the main thread doesn't excessively block these
operations, move most operations to a WorkQueue.
In the future, VideoMediaSampleRenderer can be extended to support having
multiple
AVSBDLs or AVSBVRs simultaneously, allowing reconfigurations of layers or
renderers
without having to tear down and re-build decoder pipelines.
Drive-by fix: One .webm file in web-platform-tests generates decoded frames of
type
I444. This causes a debug assertion to be hit, as VideoFrameLibWebRTC only
expects
buffers to be of type I420 or I010. This assertion is now invalid so convert the
VideoFrameLibWebRTC::create() method to return a RefPtr rather than a Ref and
check the input buffer's type in that method.
*
Source/WebCore/platform/graphics/avfoundation/objc/SourceBufferPrivateAVFObjC.mm:
(WebCore::SourceBufferPrivateAVFObjC::enqueueSampleBuffer):
* Source/WebCore/platform/graphics/cocoa/MediaPlayerPrivateWebM.mm:
(WebCore::MediaPlayerPrivateWebM::enqueueSample):
(WebCore::MediaPlayerPrivateWebM::setVideoRenderer):
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.h:
(WebCore::VideoMediaSampleRenderer::prefersDecompressionSession):
(WebCore::VideoMediaSampleRenderer::timebase const):
(WebCore::VideoMediaSampleRenderer::setResourceOwner): Deleted.
* Source/WebCore/platform/graphics/cocoa/VideoMediaSampleRenderer.mm:
(WebCore::VideoMediaSampleRenderer::VideoMediaSampleRenderer):
(WebCore::VideoMediaSampleRenderer::~VideoMediaSampleRenderer):
(WebCore::VideoMediaSampleRenderer::isReadyForMoreMediaData const):
(WebCore::VideoMediaSampleRenderer::stopRequestingMediaData):
(WebCore::VideoMediaSampleRenderer::setPrefersDecompressionSession):
(WebCore::VideoMediaSampleRenderer::setPrefersDecompressionSessionInternal):
(WebCore::VideoMediaSampleRenderer::setTimebase):
(WebCore::VideoMediaSampleRenderer::setTimebaseInternal):
(WebCore::VideoMediaSampleRenderer::enqueueSample):
(WebCore::VideoMediaSampleRenderer::initializeDecompressionSession):
(WebCore::VideoMediaSampleRenderer::decodedFrameAvailable):
(WebCore::VideoMediaSampleRenderer::purgeDecodedSampleQueue):
(WebCore::VideoMediaSampleRenderer::ensureDecodedSampleQueue):
(WebCore::VideoMediaSampleRenderer::flush):
(WebCore::VideoMediaSampleRenderer::resetReadyForMoreSample):
(WebCore::VideoMediaSampleRenderer::expectMinimumUpcomingSampleBufferPresentationTime):
(WebCore::VideoMediaSampleRenderer::copyDisplayedPixelBuffer):
(WebCore::VideoMediaSampleRenderer::totalVideoFrames const):
(WebCore::VideoMediaSampleRenderer::droppedVideoFrames const):
(WebCore::VideoMediaSampleRenderer::corruptedVideoFrames const):
(WebCore::VideoMediaSampleRenderer::totalFrameDelay const):
(WebCore::VideoMediaSampleRenderer::setResourceOwner):
(WebCore::VideoMediaSampleRenderer::copyDisplayedPixelBuffer const): Deleted.
* Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.h:
* Source/WebCore/platform/graphics/cocoa/WebCoreDecompressionSession.mm:
(WebCore::WebCoreDecompressionSession::createBufferQueue):
(WebCore::WebCoreDecompressionSession::enqueueSample):
Canonical link: https://commits.webkit.org/286106@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
