Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 72be95a9c52756b4412dacc18062120d5c784889
https://github.com/WebKit/WebKit/commit/72be95a9c52756b4412dacc18062120d5c784889
Author: Jean-Yves Avenard <[email protected]>
Date: 2024-10-26 (Sat, 26 Oct 2024)
Changed paths:
M
LayoutTests/http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable-gpuprocess.html
M LayoutTests/http/wpt/mediarecorder/MediaRecorder-first-frame-expected.txt
M LayoutTests/http/wpt/mediarecorder/MediaRecorder-first-frame.html
M LayoutTests/http/wpt/mediarecorder/MediaRecorder-requestData.html
M LayoutTests/http/wpt/mediarecorder/mimeType.html
M
LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-mimetype-expected.txt
M
LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-mimetype.html
R LayoutTests/ipc/shared-video-frame-size-expected.txt
R LayoutTests/ipc/shared-video-frame-size.html
M LayoutTests/platform/ios/TestExpectations
M LayoutTests/platform/mac-wk1/TestExpectations
M LayoutTests/platform/mac-wk2/TestExpectations
M LayoutTests/platform/mac/TestExpectations
M LayoutTests/platform/wk2/TestExpectations
M Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml
M Source/WTF/wtf/PlatformEnableCocoa.h
M Source/WebCore/Headers.cmake
M Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp
M Source/WebCore/Modules/mediarecorder/MediaRecorder.h
R Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp
R Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h
M Source/WebCore/PlatformMac.cmake
M Source/WebCore/SaferCPPExpectations/UncountedCallArgsCheckerExpectations
M Source/WebCore/Sources.txt
M Source/WebCore/SourcesCocoa.txt
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/loader/EmptyClients.cpp
M Source/WebCore/page/Page.cpp
M Source/WebCore/page/Page.h
M Source/WebCore/page/PageConfiguration.cpp
M Source/WebCore/page/PageConfiguration.h
M Source/WebCore/platform/MediaSample.h
M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp
M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.h
A Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp
A Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.h
M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.cpp
M Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.h
A
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterAVFObjC.h
A
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterAVFObjC.mm
R
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h
R
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm
A
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.cpp
M
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.h
R
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.mm
M Source/WebCore/testing/Internals.cpp
M Source/WebKit/CMakeLists.txt
M Source/WebKit/DerivedSources-input.xcfilelist
M Source/WebKit/DerivedSources-output.xcfilelist
M Source/WebKit/DerivedSources.make
M Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp
M Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h
R Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp
R Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h
R Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in
R Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.cpp
R Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.h
R Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in
M Source/WebKit/SaferCPPExpectations/UncountedCallArgsCheckerExpectations
M Source/WebKit/Scripts/webkit/messages.py
M Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp
M Source/WebKit/Shared/Cocoa/WebCoreArgumentCodersCocoa.serialization.in
M Source/WebKit/Shared/WTFArgumentCoders.serialization.in
M Source/WebKit/Sources.txt
M Source/WebKit/SourcesCocoa.txt
M Source/WebKit/WebKit.xcodeproj/project.pbxproj
R Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderIdentifier.h
R Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp
R Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h
R Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.cpp
R Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.h
M Source/WebKit/WebProcess/WebPage/WebPage.cpp
M Source/WebKitLegacy/mac/WebView/WebView.mm
Log Message:
-----------
All MediaRecorderPrivate should be running in the web content process.
https://bugs.webkit.org/show_bug.cgi?id=281816
rdar://138239763
Reviewed by Jer Noble.
We extend on the work started with the MediaRecorderPrivateWriterWebM
introduced in 285061@main.
We add a new MediaRecorderPrivateEncoder that performs the encoding, encoded
frames interleaving management
and writing through the specialised MediaRecorderPrivateWriter.
This new MediaRecorderPrivate can encode into any audio codec supported by
AVFoundation (though only opus and AAC is enabled for now)
and all video codecs supported by VideoEncoder with webm (vp8, vp9) and mp4
(h264, h265, av1).
The code runs in its dedicated WorkQueue.
One of the key change which added a significant amount of technical complexity
is that while we will generate 2s GOP
a call to MediaRecorder.requestData() will return fully playable media segment,
formed such as each new segment starts
with a keyframe. The generated media is MSE compatible.
This way, the page fully controls the duration of the media segments generated
by either setting
a time slice when creating a MediaRecorder or calling requestData() at regular
interval.
When a new segment is required by the page, we flush all pending frames
received to that point. As the audio encoder
has a certain latency, we may need to wait for a future audio frame to come
which would allow the audio for the
current time to be available. When we are streaming, this is not a problem ;
however if the recording has been paused
no audio will be coming. So instead will fully drain the encoder. As a side
effect, resuming the recording
may cause an audible silence for the duration of the codec delay (55ms with
Opus content).
In practice, this may not be noticeable as pausing and resuming a recording
would produce noticeable
artifacts with audio.
Another technical change with the previous webm recorder is that the video time
is now set to use
the start time of the last appended audio frame rather than the end. This too
is also a consequence of the audio encoder
having a set latency and allowed for some simplifications in the code.
The new MediaRecorder add supports for the codec to be set and the mimetype for
the content generated will
be mostly accurate (we use static values with common profiles rather than
attempting to determing accurately
things like the codec level and profile.
The RemoteMediaRecorder being no longer used is fully removed along the related
files.
The class MediaRecorderProvider and its overrides are removed.
Tests amended and corrected to reflect the more accurate behaviors.
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-first-frame-expected.txt:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-first-frame.html:
* LayoutTests/http/wpt/mediarecorder/MediaRecorder-requestData.html:
* LayoutTests/http/wpt/mediarecorder/mimeType.html:
*
LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-mimetype-expected.txt:
*
LayoutTests/imported/w3c/web-platform-tests/mediacapture-record/MediaRecorder-mimetype.html:
* LayoutTests/ipc/shared-video-frame-size-expected.txt: Removed.
* LayoutTests/ipc/shared-video-frame-size.html: Removed.
* LayoutTests/platform/ios/TestExpectations:
* LayoutTests/platform/mac/TestExpectations:
* LayoutTests/platform/mac-wk1/TestExpectations: Amend test, was failing
before, just failed a different way. There's no MediaStream support on WK1
* Source/WTF/Scripts/Preferences/UnifiedWebPreferences.yaml:
* Source/WTF/wtf/PlatformEnableCocoa.h:
* Source/WebCore/Headers.cmake:
* Source/WebCore/Modules/mediarecorder/MediaRecorder.cpp:
(WebCore::MediaRecorder::isTypeSupported):
(WebCore::MediaRecorder::createMediaRecorderPrivate):
(WebCore::MediaRecorder::startRecording):
* Source/WebCore/Modules/mediarecorder/MediaRecorder.h:
* Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.cpp: Removed.
* Source/WebCore/Modules/mediarecorder/MediaRecorderProvider.h: Removed.
* Source/WebCore/PlatformMac.cmake:
* Source/WebCore/SaferCPPExpectations/UncountedCallArgsCheckerExpectations:
* Source/WebCore/Sources.txt:
* Source/WebCore/SourcesCocoa.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/loader/EmptyClients.cpp:
(WebCore::pageConfigurationWithEmptyClients):
* Source/WebCore/page/Page.cpp:
(WebCore::Page::Page):
* Source/WebCore/page/Page.h:
(WebCore::Page::socketProvider):
(WebCore::Page::mediaRecorderProvider): Deleted.
* Source/WebCore/page/PageConfiguration.cpp:
(WebCore::PageConfiguration::PageConfiguration):
* Source/WebCore/page/PageConfiguration.h:
* Source/WebCore/platform/MediaSample.h:
(WebCore::MediaSample::presentationEndTime const):
* Source/WebCore/platform/graphics/cocoa/CMUtilities.mm:
(WebCore::createFormatDescriptionFromTrackInfo):
(WebCore::toCMSampleBuffer):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.cpp:
(WebCore::MediaRecorderPrivateAVFImpl::isTypeSupported):
(WebCore::MediaRecorderPrivateAVFImpl::create):
(WebCore::MediaRecorderPrivateAVFImpl::MediaRecorderPrivateAVFImpl):
(WebCore::MediaRecorderPrivateAVFImpl::~MediaRecorderPrivateAVFImpl):
(WebCore::MediaRecorderPrivateAVFImpl::startRecording):
(WebCore::MediaRecorderPrivateAVFImpl::videoFrameAvailable):
(WebCore::MediaRecorderPrivateAVFImpl::audioSamplesAvailable):
(WebCore::MediaRecorderPrivateAVFImpl::stopRecording):
(WebCore::MediaRecorderPrivateAVFImpl::pauseRecording):
(WebCore::MediaRecorderPrivateAVFImpl::resumeRecording):
(WebCore::MediaRecorderPrivateAVFImpl::fetchData):
(WebCore::MediaRecorderPrivateAVFImpl::mimeType const):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateAVFImpl.h:
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.cpp: Added.
(WebCore::MediaRecorderPrivateEncoder::Listener::create):
(WebCore::MediaRecorderPrivateEncoder::Listener::Listener):
(WebCore::MediaRecorderPrivateEncoder::create):
(WebCore::MediaRecorderPrivateEncoder::queueSingleton):
(WebCore::MediaRecorderPrivateEncoder::MediaRecorderPrivateEncoder):
(WebCore::codecStringForMediaVideoCodecId):
(WebCore::MediaRecorderPrivateEncoder::compressedAudioOutputBufferCallback):
(WebCore::MediaRecorderPrivateEncoder::initialize):
(WebCore::MediaRecorderPrivateEncoder::generateMIMEType):
(WebCore::MediaRecorderPrivateEncoder::pause):
(WebCore::MediaRecorderPrivateEncoder::resume):
(WebCore::MediaRecorderPrivateEncoder::close):
(WebCore::MediaRecorderPrivateEncoder::mimeType const):
(WebCore::MediaRecorderPrivateEncoder::audioBitRate const):
(WebCore::MediaRecorderPrivateEncoder::videoBitRate const):
(WebCore::MediaRecorderPrivateEncoder::appendAudioSampleBuffer):
(WebCore::MediaRecorderPrivateEncoder::appendVideoFrame):
(WebCore::MediaRecorderPrivateEncoder::nextVideoFrameTime):
(WebCore::MediaRecorderPrivateEncoder::appendData):
(WebCore::MediaRecorderPrivateEncoder::flushDataBuffer):
(WebCore::MediaRecorderPrivateEncoder::hasMuxedDataSinceEndSegment const):
(WebCore::MediaRecorderPrivateEncoder::takeData):
(WebCore::MediaRecorderPrivateEncoder::enqueueCompressedAudioSampleBuffers):
(WebCore::MediaRecorderPrivateEncoder::maybeStartWriter):
(WebCore::MediaRecorderPrivateEncoder::encodePendingVideoFrames):
(WebCore::MediaRecorderPrivateEncoder::processVideoEncoderActiveConfiguration):
(WebCore::MediaRecorderPrivateEncoder::enqueueCompressedVideoFrame):
(WebCore::MediaRecorderPrivateEncoder::partiallyFlushEncodedQueues):
(WebCore::MediaRecorderPrivateEncoder::waitForMatchingAudio):
(WebCore::MediaRecorderPrivateEncoder::flushToEndSegment):
(WebCore::MediaRecorderPrivateEncoder::flushAllEncodedQueues):
(WebCore::MediaRecorderPrivateEncoder::muxNextFrame):
(WebCore::MediaRecorderPrivateEncoder::stopRecording):
(WebCore::MediaRecorderPrivateEncoder::fetchData):
(WebCore::MediaRecorderPrivateEncoder::flushPendingData):
(WebCore::MediaRecorderPrivateEncoder::currentTime const):
(WebCore::MediaRecorderPrivateEncoder::currentEndTime const):
(WebCore::MediaRecorderPrivateEncoder::audioCompressor const):
(WebCore::MediaRecorderPrivateEncoder::listener):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateEncoder.h: Added.
(WebCore::MediaRecorderPrivateEncoder::hasAudio const):
(WebCore::MediaRecorderPrivateEncoder::hasVideo const):
(WebCore::MediaRecorderPrivateEncoder::WTF_GUARDED_BY_CAPABILITY):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.cpp:
(WebCore::MediaRecorderPrivateWriter::create):
* Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.h:
(): Deleted.
*
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterAVFObjC.h:
Copied from Source/WebCore/platform/mediarecorder/MediaRecorderPrivateWriter.h.
*
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterAVFObjC.mm:
Added.
(-[WebAVAssetWriterDelegate initWithWriter:]):
(-[WebAVAssetWriterDelegate assetWriter:didOutputSegmentData:segmentType:]):
(WebCore::MediaRecorderPrivateWriterAVFObjC::create):
(WebCore::MediaRecorderPrivateWriterAVFObjC::MediaRecorderPrivateWriterAVFObjC):
(WebCore::MediaRecorderPrivateWriterAVFObjC::addAudioTrack):
(WebCore::MediaRecorderPrivateWriterAVFObjC::addVideoTrack):
(WebCore::MediaRecorderPrivateWriterAVFObjC::allTracksAdded):
(WebCore::MediaRecorderPrivateWriterAVFObjC::muxFrame):
(WebCore::appendEndsPreviousSampleDurationMarker):
(WebCore::MediaRecorderPrivateWriterAVFObjC::forceNewSegment):
(WebCore::MediaRecorderPrivateWriterAVFObjC::close):
*
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.h:
Removed.
*
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:
Removed.
*
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.cpp:
Added.
(WebCore::MediaRecorderPrivateWriterWebMDelegate::MediaRecorderPrivateWriterWebMDelegate):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addAudioTrack):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addVideoTrack):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::addFrame):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::forceNewClusterOnNextFrame):
(WebCore::MediaRecorderPrivateWriterWebMDelegate::finalize):
(WebCore::MediaRecorderPrivateWriterWebM::create):
(WebCore::mkvCodeIcForMediaVideoCodecId):
(WebCore::MediaRecorderPrivateWriterWebM::MediaRecorderPrivateWriterWebM):
(WebCore::MediaRecorderPrivateWriterWebM::addAudioTrack):
(WebCore::MediaRecorderPrivateWriterWebM::addVideoTrack):
(WebCore::MediaRecorderPrivateWriterWebM::muxFrame):
(WebCore::MediaRecorderPrivateWriterWebM::forceNewSegment):
(WebCore::MediaRecorderPrivateWriterWebM::close):
* Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.h:
*
Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterWebM.mm:
Removed.
* Source/WebCore/testing/Internals.cpp:
* Source/WebKit/CMakeLists.txt:
* Source/WebKit/DerivedSources-input.xcfilelist:
* Source/WebKit/DerivedSources-output.xcfilelist:
* Source/WebKit/DerivedSources.make:
* Source/WebKit/GPUProcess/GPUConnectionToWebProcess.cpp:
(WebKit::GPUConnectionToWebProcess::allowsExitUnderMemoryPressure const):
(WebKit::GPUConnectionToWebProcess::dispatchMessage):
(WebKit::GPUConnectionToWebProcess::mediaRecorderManager): Deleted.
* Source/WebKit/GPUProcess/GPUConnectionToWebProcess.h:
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.cpp: Removed.
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.h: Removed.
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorder.messages.in: Removed.
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.cpp: Removed.
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.h: Removed.
* Source/WebKit/GPUProcess/webrtc/RemoteMediaRecorderManager.messages.in:
Removed.
* Source/WebKit/SaferCPPExpectations/UncountedCallArgsCheckerExpectations:
* Source/WebKit/Scripts/webkit/messages.py:
(serialized_identifiers):
* Source/WebKit/Scripts/webkit/tests/MessageArgumentDescriptions.cpp:
(IPC::serializedIdentifiers):
* Source/WebKit/Shared/Cocoa/WebCoreArgumentCodersCocoa.serialization.in:
* Source/WebKit/Shared/WTFArgumentCoders.serialization.in:
* Source/WebKit/Sources.txt:
* Source/WebKit/SourcesCocoa.txt:
* Source/WebKit/WebKit.xcodeproj/project.pbxproj:
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderIdentifier.h: Removed.
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.cpp: Removed.
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderPrivate.h: Removed.
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.cpp: Removed.
* Source/WebKit/WebProcess/GPU/webrtc/MediaRecorderProvider.h: Removed.
* Source/WebKit/WebProcess/WebPage/WebPage.cpp:
(WebKit::m_textAnimationController):
* Source/WebKitLegacy/mac/WebView/WebView.mm:
(-[WebView _commonInitializationWithFrameName:groupName:]):
(-[WebView initSimpleHTMLDocumentWithStyle:frame:preferences:groupName:]):
Canonical link: https://commits.webkit.org/285726@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
