Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 149d8d7a3cfd0970497eb25ebdf41baecd09e968
https://github.com/WebKit/WebKit/commit/149d8d7a3cfd0970497eb25ebdf41baecd09e968
Author: Youenn Fablet <[email protected]>
Date: 2023-05-06 (Sat, 06 May 2023)
Changed paths:
M LayoutTests/fast/mediastream/constraint-intrinsic-size.html
M LayoutTests/fast/mediastream/getDisplayMedia-frame-rate-expected.txt
M LayoutTests/fast/mediastream/getDisplayMedia-frame-rate.html
M LayoutTests/fast/mediastream/getDisplayMedia-size-expected.txt
A LayoutTests/fast/mediastream/getUserMedia-frame-rate-expected.txt
A LayoutTests/fast/mediastream/getUserMedia-frame-rate.html
M LayoutTests/platform/glib/TestExpectations
M LayoutTests/platform/mac-wk2/TestExpectations
M Source/WebCore/Headers.cmake
M Source/WebCore/Sources.txt
M Source/WebCore/WebCore.xcodeproj/project.pbxproj
M Source/WebCore/platform/mediastream/MediaConstraints.cpp
M Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp
M Source/WebCore/platform/mediastream/RealtimeMediaSource.h
M Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp
M Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h
R Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp
R Source/WebCore/platform/mediastream/RealtimeVideoSource.h
M Source/WebCore/platform/mediastream/cocoa/DisplayCaptureSourceCocoa.cpp
M Source/WebCore/platform/mediastream/cocoa/DisplayCaptureSourceCocoa.h
M
Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp
M Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm
M Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm
M Source/WebCore/platform/mock/MockMediaDevice.h
M Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp
M Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp
M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp
M Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h
M Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp
Log Message:
-----------
Add support for getDisplayMedia video track clone resizing
https://bugs.webkit.org/show_bug.cgi?id=256250
rdar://problem/108834643
Reviewed by Eric Carlson.
Remove RealtimeVideoSource which only handles preset based video capture
sources.
Instead, we make RealtimeMediaSource::VideoFrameObserver able to state which
size and frame rate it wants.
RealtimeMediaSource will then update video frame sending accordingly.
We enable DisplayCaptureSourceCocoa on iOS as well to be able to run tests on
simulator properly.
On WebProcess side, all VideoFrameObservers will not provide desired size/frame
rates since they only consume video frames.
On GPUProcess side, VideoFrameObservers created by UserMediaCaptureManagerProxy
will provide desired size/frame rates from provided constraints.
The computation of desired size/frame rates from constraints is done in two
steps:
- VideoFrameObservers will provide ideal width/height/frameRate values from the
MediaConstraints using RealtimeMediaSource::extractVideoFrameSizeConstraints.
- RealtimeMediaSource will compute the desired sizes based on
RealtimeMediaSource::computeResizedVideoFrameSize.
Camera sources will allow aspect ratio changes, see
RealtimeMediaSource::computeResizedVideoFrameSize.
Display sources will not allow aspect ratio changes, see
DisplayCaptureSourceCocoa::computeResizedVideoFrameSize.
We start changing how aspectRatio is handled as well.
It is used for camera sources when validating constraints and when computing
the width/height ideal values.
In the future, RealtimeMediaSource may not need to store aspectRatio at all.
For now, if aspectRatio constraint is provided, we will not set the default
height constraint to 480, since it can be computed from the width and
aspectRatio.
By using size/frame rates as part of VideoFrameObservers, observers can receive
video frames with different size/frame rate than the source is exposing via
getSettings().
To keep consistent settings values, UserMediaCaptureManagerProxy is overriding
source width/height/frameRate based on size/frame rates passed to the
VideoFrameObserver.
In the future, we might be able to remove RealtimeMediaSource size and only
keep intrinsicSize, which will simplify the class.
The display source is now capturing at the native size and resizing is done as
a follow-up step when sending video frame to observers.
Covered by existing tests for getUserMedia and by new tests for getDisplayMedia.
* LayoutTests/fast/mediastream/constraint-intrinsic-size.html:
* LayoutTests/fast/mediastream/getDisplayMedia-frame-rate-expected.txt:
* LayoutTests/fast/mediastream/getDisplayMedia-frame-rate.html:
* LayoutTests/fast/mediastream/getDisplayMedia-size-expected.txt:
* LayoutTests/fast/mediastream/getUserMedia-frame-rate-expected.txt: Added.
* LayoutTests/fast/mediastream/getUserMedia-frame-rate.html: Added.
* LayoutTests/platform/glib/TestExpectations:
* LayoutTests/platform/mac-wk2/TestExpectations:
* Source/WebCore/Headers.cmake:
* Source/WebCore/Sources.txt:
* Source/WebCore/WebCore.xcodeproj/project.pbxproj:
* Source/WebCore/platform/mediastream/MediaConstraints.cpp:
(WebCore::addDefaultVideoConstraints):
(WebCore::MediaConstraints::setDefaultVideoConstraints):
* Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::VideoFrameAdaptor::VideoFrameAdaptor):
(WebCore::RealtimeMediaSource::~RealtimeMediaSource):
(WebCore::RealtimeMediaSource::addVideoFrameObserver):
(WebCore::RealtimeMediaSource::removeVideoFrameObserver):
(WebCore::RealtimeMediaSource::forEachVideoFrameObserver):
(WebCore::RealtimeMediaSource::videoFrameRotation const):
(WebCore::adaptVideoFrame):
(WebCore::RealtimeMediaSource::computeResizedVideoFrameSize):
(WebCore::RealtimeMediaSource::videoFrameAvailable):
(WebCore::RealtimeMediaSource::extractVideoFrameSizeConstraints):
(WebCore::RealtimeMediaSource::applyConstraints):
* Source/WebCore/platform/mediastream/RealtimeMediaSource.h:
* Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp:
(WebCore::RealtimeVideoCaptureSource::updateCapabilities):
(WebCore::RealtimeVideoCaptureSource::bestSupportedSizeFrameRateAndZoom):
(WebCore::RealtimeVideoCaptureSource::clientUpdatedSizeFrameRateAndZoom):
(WebCore::RealtimeVideoCaptureSource::setSizeFrameRateAndZoom):
(WebCore::RealtimeVideoCaptureSource::videoFrameRotation const): Deleted.
* Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h:
* Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp: Removed.
* Source/WebCore/platform/mediastream/RealtimeVideoSource.h: Removed.
* Source/WebCore/platform/mediastream/cocoa/DisplayCaptureSourceCocoa.cpp:
(WebCore::DisplayCaptureSourceCocoa::computeResizedVideoFrameSize):
(WebCore::DisplayCaptureSourceCocoa::setSizeFrameRateAndZoom):
(WebCore::DisplayCaptureSourceCocoa::emitFrame):
(WebCore::DisplayCaptureSourceCocoa::updateFrameSize): Deleted.
* Source/WebCore/platform/mediastream/cocoa/DisplayCaptureSourceCocoa.h:
*
Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp:
(WebCore::MockRealtimeVideoSource::create):
(WebCore::MockDisplayCaptureSourceGStreamer::create):
* Source/WebCore/platform/mediastream/mac/AVVideoCaptureSource.mm:
(WebCore::AVVideoCaptureSource::create):
* Source/WebCore/platform/mediastream/mac/MockRealtimeVideoSourceMac.mm:
(WebCore::MockRealtimeVideoSource::create):
* Source/WebCore/platform/mock/MockMediaDevice.h:
* Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp:
* Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp:
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.cpp:
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::applyConstraints):
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::settings):
(WebKit::UserMediaCaptureManagerProxy::SourceProxy::copySettings):
(WebKit::UserMediaCaptureManagerProxy::createCameraSource):
(WebKit::UserMediaCaptureManagerProxy::createMediaSourceForCaptureDeviceWithConstraints):
(WebKit::UserMediaCaptureManagerProxy::applyConstraints):
(WebKit::UserMediaCaptureManagerProxy::clone):
* Source/WebKit/UIProcess/Cocoa/UserMediaCaptureManagerProxy.h:
* Source/WebKit/WebProcess/cocoa/UserMediaCaptureManager.cpp:
Canonical link: https://commits.webkit.org/263750@main
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
