Title: [279940] trunk
Revision
279940
Author
[email protected]
Date
2021-07-15 04:48:58 -0700 (Thu, 15 Jul 2021)

Log Message

[GStreamer][Pipewire] Implement getDisplayMedia() backend
https://bugs.webkit.org/show_bug.cgi?id=210926

Reviewed by Xabier Rodriguez-Calvar.

.:

* Source/cmake/FindLIBPORTAL.cmake: Added.
* Source/cmake/GStreamerChecks.cmake: Enable Pipewire support if libportal was found.

Source/WebCore:

Display capture is now supported by GStreamer ports. When requested, the capture device
manager queries the host through the libportal API. The user is presented with a prompt
allowing to select either a monitor or an application window. Once selected, the manager
creates the associated CaptureDevice, passing the pipewire file descriptor along. The
existing GStreamer video capture source infrastructure is reused, leveraging the GStreamer
pipewiresrc element to generate a live video stream of the display device.

As display capture devices need to respect the aspect-ratio of the video frame, taking into
account max-{widther,height}, some code was reused from the Cocoa display capture code and
placed in RealtimeVideoCaptureSource as ensureIntrinsicSizeMaintainsAspectRatio().

Unfortunately this aspect-ratio enforcing feature is currently used only for mock capture
sources. The GStreamer pipewiresrc doesn't support caps renegotiation well yet, so we can't
comply with the aspect-ratio enforcing, for the time being.

This patch is covered by newly unskipped layout tests and API tests.

* platform/GStreamer.cmake:
* platform/mediastream/RealtimeMediaSource.cpp:
(WebCore::RealtimeMediaSource::setIntrinsicSize):
(WebCore::RealtimeMediaSource::hashedId const):
* platform/mediastream/RealtimeMediaSource.h:
* platform/mediastream/RealtimeVideoCaptureSource.cpp:
(WebCore::RealtimeVideoCaptureSource::ensureIntrinsicSizeMaintainsAspectRatio):
* platform/mediastream/RealtimeVideoCaptureSource.h:
* platform/mediastream/RealtimeVideoSource.cpp:
(WebCore::RealtimeVideoSource::sourceSettingsChanged):
* platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp:
(WebCore::GStreamerAudioCapturer::GStreamerAudioCapturer):
* platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp:
(WebCore::GStreamerDisplayCaptureDeviceManager::GStreamerDisplayCaptureDeviceManager):
(WebCore::GStreamerDisplayCaptureDeviceManager::~GStreamerDisplayCaptureDeviceManager):
(WebCore::GStreamerDisplayCaptureDeviceManager::computeCaptureDevices):
(WebCore::GStreamerDisplayCaptureDeviceManager::setSession):
(WebCore::GStreamerDisplayCaptureDeviceManager::sessionStarted):
(WebCore::GStreamerDisplayCaptureDeviceManager::notifyClient):
(WebCore::GStreamerDisplayCaptureDeviceManager::sessionWasClosed):
* platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h:
* platform/mediastream/gstreamer/GStreamerCapturer.cpp:
(WebCore::GStreamerCapturer::GStreamerCapturer):
(WebCore::GStreamerCapturer::Observer::~Observer):
(WebCore::GStreamerCapturer::addObserver):
(WebCore::GStreamerCapturer::removeObserver):
(WebCore::GStreamerCapturer::forEachObserver):
(WebCore::GStreamerCapturer::createSource):
* platform/mediastream/gstreamer/GStreamerCapturer.h:
(WebCore::GStreamerCapturer::Observer::sourceCapsChanged):
(WebCore::GStreamerCapturer::deviceType const):
* platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp:
(WebCore::GStreamerVideoCaptureSource::create):
(WebCore::GStreamerVideoCaptureSource::createPipewireSource):
(WebCore::GStreamerVideoCaptureSource::GStreamerVideoCaptureSource):
(WebCore::m_deviceType):
(WebCore::GStreamerVideoCaptureSource::~GStreamerVideoCaptureSource):
(WebCore::GStreamerVideoCaptureSource::sourceCapsChanged):
(WebCore::GStreamerVideoCaptureSource::startProducingData):
(WebCore::GStreamerVideoCaptureSource::stopProducingData):
(WebCore::m_capturer): Deleted.
* platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h:
* platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp:
(WebCore::initializeDebugCategory):
(WebCore::GStreamerVideoCapturer::GStreamerVideoCapturer):
(WebCore::GStreamerVideoCapturer::createSource):
(WebCore::GStreamerVideoCapturer::setPipewireFD):
(WebCore::GStreamerVideoCapturer::setSize):
(WebCore::GStreamerVideoCapturer::setFrameRate):
* platform/mediastream/gstreamer/GStreamerVideoCapturer.h:
* platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp:
(WebCore::MockRealtimeVideoSource::create):
(WebCore::MockRealtimeVideoSourceGStreamer::createMockDisplayCaptureSource):
* platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.h:
* platform/mock/MockRealtimeMediaSourceCenter.cpp:
* platform/mock/MockRealtimeVideoSource.cpp:
(WebCore::MockRealtimeVideoSource::capabilities):
(WebCore::MockRealtimeVideoSource::generateFrame):
* platform/mock/MockRealtimeVideoSource.h:

Source/WTF:

* Scripts/Preferences/WebPreferencesExperimental.yaml: Enable screen-capture if libportal
was found.

LayoutTests:

* platform/glib/TestExpectations: Unskip now-passing getDisplayMedia tests.

Modified Paths

Added Paths

Diff

Modified: trunk/ChangeLog (279939 => 279940)


--- trunk/ChangeLog	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/ChangeLog	2021-07-15 11:48:58 UTC (rev 279940)
@@ -1,3 +1,13 @@
+2021-07-15  Philippe Normand  <[email protected]>
+
+        [GStreamer][Pipewire] Implement getDisplayMedia() backend
+        https://bugs.webkit.org/show_bug.cgi?id=210926
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        * Source/cmake/FindLIBPORTAL.cmake: Added.
+        * Source/cmake/GStreamerChecks.cmake: Enable Pipewire support if libportal was found.
+
 2021-07-13  Michael Catanzaro  <[email protected]>
 
         Remove USE_64KB_PAGE_BLOCK

Modified: trunk/LayoutTests/ChangeLog (279939 => 279940)


--- trunk/LayoutTests/ChangeLog	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/LayoutTests/ChangeLog	2021-07-15 11:48:58 UTC (rev 279940)
@@ -1,3 +1,12 @@
+2021-07-15  Philippe Normand  <[email protected]>
+
+        [GStreamer][Pipewire] Implement getDisplayMedia() backend
+        https://bugs.webkit.org/show_bug.cgi?id=210926
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        * platform/glib/TestExpectations: Unskip now-passing getDisplayMedia tests.
+
 2021-07-15  Ziran Sun  <[email protected]>
 
         Resync web-platform-tests/css/css-grid tests from upstream

Modified: trunk/LayoutTests/platform/glib/TestExpectations (279939 => 279940)


--- trunk/LayoutTests/platform/glib/TestExpectations	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/LayoutTests/platform/glib/TestExpectations	2021-07-15 11:48:58 UTC (rev 279940)
@@ -1140,6 +1140,8 @@
 webkit.org/b/206656 imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-preload-none-manual.https.html [ Crash Failure Pass ]
 webkit.org/b/210385 imported/w3c/web-platform-tests/mediacapture-streams/MediaStream-MediaElement-firstframe.https.html [ Failure Crash Pass ]
 
+webkit.org/b/227987 http/tests/media/media-stream/get-display-media-prompt.html [ Timeout Pass ]
+
 #////////////////////////////////////////////////////////////////////////////////////////
 # End of WebRTC-related bugs
 #////////////////////////////////////////////////////////////////////////////////////////
@@ -1522,15 +1524,6 @@
 webkit.org/b/109570 media/track/regions-webvtt [ Skip ]
 webkit.org/b/109570 media/track/w3c [ Skip ]
 
-# No support for screen capture
-fast/mediastream/screencapture-user-gesture.html [ Skip ]
-webkit.org/b/210926 fast/mediastream/getDisplayMedia-max-constraints1.html [ Skip ]
-webkit.org/b/210926 fast/mediastream/getDisplayMedia-max-constraints2.html [ Skip ]
-webkit.org/b/210926 fast/mediastream/getDisplayMedia-max-constraints3.html [ Skip ]
-webkit.org/b/210926 fast/mediastream/get-display-media-muted.html [ Skip ]
-webkit.org/b/210926 fast/mediastream/constraint-intrinsic-size.html [ Skip ]
-webkit.org/b/191007 http/tests/media/media-stream/get-display-media-prompt.html [ Skip ]
-
 # Modern-media-controls support
 webkit.org/b/182502 http/tests/media/modern-media-controls [ Skip ]
 webkit.org/b/182502 media/modern-media-controls [ Skip ]

Modified: trunk/Source/WTF/ChangeLog (279939 => 279940)


--- trunk/Source/WTF/ChangeLog	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WTF/ChangeLog	2021-07-15 11:48:58 UTC (rev 279940)
@@ -1,3 +1,13 @@
+2021-07-15  Philippe Normand  <[email protected]>
+
+        [GStreamer][Pipewire] Implement getDisplayMedia() backend
+        https://bugs.webkit.org/show_bug.cgi?id=210926
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        * Scripts/Preferences/WebPreferencesExperimental.yaml: Enable screen-capture if libportal
+        was found.
+
 2021-07-14  Jer Noble  <[email protected]>
 
         Unreviewed build fix after r279912 (239661@main); Add a HAVE(SYSTEM_STATUS) macro.

Modified: trunk/Source/WTF/Scripts/Preferences/WebPreferencesExperimental.yaml (279939 => 279940)


--- trunk/Source/WTF/Scripts/Preferences/WebPreferencesExperimental.yaml	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WTF/Scripts/Preferences/WebPreferencesExperimental.yaml	2021-07-15 11:48:58 UTC (rev 279940)
@@ -927,7 +927,7 @@
     WebKitLegacy:
       default: false
     WebKit:
-      "PLATFORM(MAC)": true
+      "PLATFORM(MAC) || USE(PIPEWIRE)": true
       default: false
     WebCore:
       default: false

Modified: trunk/Source/WebCore/ChangeLog (279939 => 279940)


--- trunk/Source/WebCore/ChangeLog	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/ChangeLog	2021-07-15 11:48:58 UTC (rev 279940)
@@ -1,3 +1,87 @@
+2021-07-15  Philippe Normand  <[email protected]>
+
+        [GStreamer][Pipewire] Implement getDisplayMedia() backend
+        https://bugs.webkit.org/show_bug.cgi?id=210926
+
+        Reviewed by Xabier Rodriguez-Calvar.
+
+        Display capture is now supported by GStreamer ports. When requested, the capture device
+        manager queries the host through the libportal API. The user is presented with a prompt
+        allowing to select either a monitor or an application window. Once selected, the manager
+        creates the associated CaptureDevice, passing the pipewire file descriptor along. The
+        existing GStreamer video capture source infrastructure is reused, leveraging the GStreamer
+        pipewiresrc element to generate a live video stream of the display device.
+
+        As display capture devices need to respect the aspect-ratio of the video frame, taking into
+        account max-{widther,height}, some code was reused from the Cocoa display capture code and
+        placed in RealtimeVideoCaptureSource as ensureIntrinsicSizeMaintainsAspectRatio().
+
+        Unfortunately this aspect-ratio enforcing feature is currently used only for mock capture
+        sources. The GStreamer pipewiresrc doesn't support caps renegotiation well yet, so we can't
+        comply with the aspect-ratio enforcing, for the time being.
+
+        This patch is covered by newly unskipped layout tests and API tests.
+
+        * platform/GStreamer.cmake:
+        * platform/mediastream/RealtimeMediaSource.cpp:
+        (WebCore::RealtimeMediaSource::setIntrinsicSize):
+        (WebCore::RealtimeMediaSource::hashedId const):
+        * platform/mediastream/RealtimeMediaSource.h:
+        * platform/mediastream/RealtimeVideoCaptureSource.cpp:
+        (WebCore::RealtimeVideoCaptureSource::ensureIntrinsicSizeMaintainsAspectRatio):
+        * platform/mediastream/RealtimeVideoCaptureSource.h:
+        * platform/mediastream/RealtimeVideoSource.cpp:
+        (WebCore::RealtimeVideoSource::sourceSettingsChanged):
+        * platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp:
+        (WebCore::GStreamerAudioCapturer::GStreamerAudioCapturer):
+        * platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp:
+        (WebCore::GStreamerDisplayCaptureDeviceManager::GStreamerDisplayCaptureDeviceManager):
+        (WebCore::GStreamerDisplayCaptureDeviceManager::~GStreamerDisplayCaptureDeviceManager):
+        (WebCore::GStreamerDisplayCaptureDeviceManager::computeCaptureDevices):
+        (WebCore::GStreamerDisplayCaptureDeviceManager::setSession):
+        (WebCore::GStreamerDisplayCaptureDeviceManager::sessionStarted):
+        (WebCore::GStreamerDisplayCaptureDeviceManager::notifyClient):
+        (WebCore::GStreamerDisplayCaptureDeviceManager::sessionWasClosed):
+        * platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h:
+        * platform/mediastream/gstreamer/GStreamerCapturer.cpp:
+        (WebCore::GStreamerCapturer::GStreamerCapturer):
+        (WebCore::GStreamerCapturer::Observer::~Observer):
+        (WebCore::GStreamerCapturer::addObserver):
+        (WebCore::GStreamerCapturer::removeObserver):
+        (WebCore::GStreamerCapturer::forEachObserver):
+        (WebCore::GStreamerCapturer::createSource):
+        * platform/mediastream/gstreamer/GStreamerCapturer.h:
+        (WebCore::GStreamerCapturer::Observer::sourceCapsChanged):
+        (WebCore::GStreamerCapturer::deviceType const):
+        * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp:
+        (WebCore::GStreamerVideoCaptureSource::create):
+        (WebCore::GStreamerVideoCaptureSource::createPipewireSource):
+        (WebCore::GStreamerVideoCaptureSource::GStreamerVideoCaptureSource):
+        (WebCore::m_deviceType):
+        (WebCore::GStreamerVideoCaptureSource::~GStreamerVideoCaptureSource):
+        (WebCore::GStreamerVideoCaptureSource::sourceCapsChanged):
+        (WebCore::GStreamerVideoCaptureSource::startProducingData):
+        (WebCore::GStreamerVideoCaptureSource::stopProducingData):
+        (WebCore::m_capturer): Deleted.
+        * platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h:
+        * platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp:
+        (WebCore::initializeDebugCategory):
+        (WebCore::GStreamerVideoCapturer::GStreamerVideoCapturer):
+        (WebCore::GStreamerVideoCapturer::createSource):
+        (WebCore::GStreamerVideoCapturer::setPipewireFD):
+        (WebCore::GStreamerVideoCapturer::setSize):
+        (WebCore::GStreamerVideoCapturer::setFrameRate):
+        * platform/mediastream/gstreamer/GStreamerVideoCapturer.h:
+        * platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp:
+        (WebCore::MockRealtimeVideoSource::create):
+        (WebCore::MockRealtimeVideoSourceGStreamer::createMockDisplayCaptureSource):
+        * platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.h:
+        * platform/mock/MockRealtimeMediaSourceCenter.cpp:
+        * platform/mock/MockRealtimeVideoSource.cpp:
+        (WebCore::MockRealtimeVideoSource::capabilities):
+        (WebCore::MockRealtimeVideoSource::generateFrame):
+        * platform/mock/MockRealtimeVideoSource.h:
+
 2021-07-15  Philippe Normand  <[email protected]> and Miguel Gomez  <[email protected]>
 
         [GStreamer][GL] A420 compositing support

Modified: trunk/Source/WebCore/platform/GStreamer.cmake (279939 => 279940)


--- trunk/Source/WebCore/platform/GStreamer.cmake	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/GStreamer.cmake	2021-07-15 11:48:58 UTC (rev 279940)
@@ -151,6 +151,10 @@
                 ${GSTREAMER_CODECPARSERS_LIBRARIES}
             )
         endif ()
+
+        if (USE_PIPEWIRE)
+            list(APPEND WebCore_LIBRARIES LIBPORTAL::LIBPORTAL)
+        endif ()
     endif ()
 endif ()
 

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -978,7 +978,7 @@
     return size;
 }
 
-void RealtimeMediaSource::setIntrinsicSize(const IntSize& size)
+void RealtimeMediaSource::setIntrinsicSize(const IntSize& size, bool notifyObservers)
 {
     if (m_intrinsicSize == size)
         return;
@@ -987,6 +987,8 @@
     
     auto currentSize = this->size();
     m_intrinsicSize = size;
+    if (!notifyObservers)
+        return;
 
     if (currentSize != this->size()) {
         scheduleDeferredTask([this] {
@@ -1097,7 +1099,11 @@
 
 const String& RealtimeMediaSource::hashedId() const
 {
-    ASSERT(!m_hashedID.isEmpty());
+#ifndef NDEBUG
+    auto deviceType = this->deviceType();
+    if (deviceType != CaptureDevice::DeviceType::Screen && deviceType != CaptureDevice::DeviceType::Window)
+        ASSERT(!m_hashedID.isEmpty());
+#endif
     return m_hashedID;
 }
 

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeMediaSource.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -150,7 +150,7 @@
     void setSize(const IntSize&);
 
     const IntSize intrinsicSize() const;
-    void setIntrinsicSize(const IntSize&);
+    void setIntrinsicSize(const IntSize&, bool notifyObservers = true);
 
     double frameRate() const { return m_frameRate; }
     void setFrameRate(double);

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -427,6 +427,33 @@
     setFrameRate(match->requestedFrameRate);
 }
 
+void RealtimeVideoCaptureSource::ensureIntrinsicSizeMaintainsAspectRatio()
+{
+    auto intrinsicSize = this->intrinsicSize();
+    auto frameSize = size();
+    if (!frameSize.height())
+        frameSize.setHeight(intrinsicSize.height());
+    if (!frameSize.width())
+        frameSize.setWidth(intrinsicSize.width());
+
+    auto maxHeight = std::min(frameSize.height(), intrinsicSize.height());
+    auto maxWidth = std::min(frameSize.width(), intrinsicSize.width());
+
+    auto heightForMaxWidth = maxWidth * intrinsicSize.height() / intrinsicSize.width();
+    auto widthForMaxHeight = maxHeight * intrinsicSize.width() / intrinsicSize.height();
+
+    if (heightForMaxWidth <= maxHeight) {
+        setSize({ maxWidth, heightForMaxWidth });
+        return;
+    }
+    if (widthForMaxHeight <= maxWidth) {
+        setSize({ widthForMaxHeight, maxHeight });
+        return;
+    }
+
+    setSize(intrinsicSize);
+}
+
 #if !RELEASE_LOG_DISABLED
 Ref<JSON::Object> SizeAndFrameRate::toJSONObject() const
 {

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeVideoCaptureSource.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -73,6 +73,8 @@
     void dispatchMediaSampleToObservers(MediaSample&);
     const Vector<IntSize>& standardVideoSizes();
 
+    void ensureIntrinsicSizeMaintainsAspectRatio();
+
 private:
     struct CaptureSizeAndFrameRate {
         RefPtr<VideoPreset> encodingPreset;

Modified: trunk/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/RealtimeVideoSource.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -112,9 +112,7 @@
 void RealtimeVideoSource::sourceSettingsChanged()
 {
     auto rotation = m_source->sampleRotation();
-    auto size = this->size();
-    if (size.isEmpty())
-        size = m_source->size();
+    auto size = m_source->size();
     if (rotation == MediaSample::VideoRotation::Left || rotation == MediaSample::VideoRotation::Right)
         size = size.transposedSize();
 

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerAudioCapturer.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -45,7 +45,7 @@
 }
 
 GStreamerAudioCapturer::GStreamerAudioCapturer()
-    : GStreamerCapturer("appsrc", adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, AudioCaptureSampleRate, nullptr)))
+    : GStreamerCapturer("appsrc", adoptGRef(gst_caps_new_simple("audio/x-raw", "rate", G_TYPE_INT, AudioCaptureSampleRate, nullptr)), CaptureDevice::DeviceType::Microphone)
 {
 }
 

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -25,6 +25,7 @@
 #include "GStreamerCaptureDeviceManager.h"
 
 #include "GStreamerCommon.h"
+#include <wtf/text/IntegerToStringConversion.h>
 
 namespace WebCore {
 
@@ -202,6 +203,97 @@
     }
 }
 
+GStreamerDisplayCaptureDeviceManager::GStreamerDisplayCaptureDeviceManager()
+{
+#if USE(PIPEWIRE)
+    m_portal = adoptGRef(xdp_portal_new());
+#endif
+}
+
+GStreamerDisplayCaptureDeviceManager::~GStreamerDisplayCaptureDeviceManager()
+{
+#if USE(PIPEWIRE)
+    if (m_session) {
+        xdp_session_close(m_session.get());
+        m_session = nullptr;
+    }
+#endif
+}
+
+void GStreamerDisplayCaptureDeviceManager::computeCaptureDevices(CompletionHandler<void()>&& callback)
+{
+#if USE(PIPEWIRE)
+    m_callback = WTFMove(callback);
+    xdp_portal_create_screencast_session(m_portal.get(), static_cast<XdpOutputType>(XDP_OUTPUT_MONITOR | XDP_OUTPUT_WINDOW), XDP_SCREENCAST_FLAG_NONE, nullptr, [](GObject* source, GAsyncResult* result, gpointer userData) {
+        GUniqueOutPtr<GError> error;
+        auto* session = xdp_portal_create_screencast_session_finish(XDP_PORTAL(source), result, &error.outPtr());
+        if (!session) {
+            WTFLogAlways("Failed to create screencast session: %s", error->message);
+            return;
+        }
+        auto& manager = *reinterpret_cast<GStreamerDisplayCaptureDeviceManager*>(userData);
+        manager.setSession(session);
+    }, this);
+#else
+    ASSERT_NOT_REACHED();
+    callback();
+#endif
+}
+
+#if USE(PIPEWIRE)
+void GStreamerDisplayCaptureDeviceManager::setSession(XdpSession* session)
+{
+    ASSERT(session);
+    m_session = adoptGRef(session);
+
+    g_signal_connect_swapped(m_session.get(), "closed", G_CALLBACK(+[](GStreamerDisplayCaptureDeviceManager* manager, XdpSession*) {
+        manager->sessionWasClosed();
+    }), this);
+    xdp_session_start(m_session.get(), nullptr, nullptr, [](GObject* source, GAsyncResult* result, gpointer userData) {
+        auto* session = XDP_SESSION(source);
+        GUniqueOutPtr<GError> error;
+        auto& manager = *reinterpret_cast<GStreamerDisplayCaptureDeviceManager*>(userData);
+        if (!xdp_session_start_finish(session, result, &error.outPtr())) {
+            WTFLogAlways("Failed to start screencast session: %s", error->message);
+            manager.notifyClient();
+            return;
+        }
+
+        manager.sessionStarted();
+    }, this);
+}
+#endif
+
+void GStreamerDisplayCaptureDeviceManager::sessionStarted()
+{
+    m_devices.clear();
+#if USE(PIPEWIRE)
+    int fd = xdp_session_open_pipewire_remote(m_session.get());
+
+    // FIXME: The libportal API doesn't seem to expose which XdpOutputType was selected by the
+    // user, so hardcode Screen here. 🤷
+    CaptureDevice captureDevice(WTF::numberToStringUnsigned<String>(fd), CaptureDevice::DeviceType::Screen, makeString("Capture Display"));
+    captureDevice.setEnabled(true);
+    m_devices.append(WTFMove(captureDevice));
+#endif
+    notifyClient();
+}
+
+void GStreamerDisplayCaptureDeviceManager::notifyClient()
+{
+    if (m_callback)
+        m_callback();
+}
+
+void GStreamerDisplayCaptureDeviceManager::sessionWasClosed()
+{
+#if USE(PIPEWIRE)
+    m_session = nullptr;
+#endif
+    m_devices.clear();
+    deviceChanged();
+}
+
 } // namespace WebCore
 
 #endif // ENABLE(MEDIA_STREAM) && USE(GSTREAMER)

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCaptureDeviceManager.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -28,6 +28,10 @@
 #include "GStreamerCaptureDevice.h"
 #include "RealtimeMediaSourceFactory.h"
 
+#if USE(PIPEWIRE)
+#include <libportal/portal.h>
+#endif
+
 namespace WebCore {
 
 class GStreamerCaptureDeviceManager : public CaptureDeviceManager {
@@ -66,15 +70,32 @@
     GStreamerVideoCaptureDeviceManager() = default;
 };
 
-class GStreamerDisplayCaptureDeviceManager final : public GStreamerCaptureDeviceManager {
+class GStreamerDisplayCaptureDeviceManager final : public CaptureDeviceManager {
     friend class NeverDestroyed<GStreamerDisplayCaptureDeviceManager>;
 public:
     static GStreamerDisplayCaptureDeviceManager& singleton();
-    CaptureDevice::DeviceType deviceType() final { return CaptureDevice::DeviceType::Screen; }
+    const Vector<CaptureDevice>& captureDevices() final { return m_devices; };
+    void computeCaptureDevices(CompletionHandler<void()>&&) final;
+
+protected:
+#if USE(PIPEWIRE)
+    void setSession(XdpSession*);
+#endif
+    void sessionStarted();
+    void notifyClient();
+    void sessionWasClosed();
+
 private:
-    GStreamerDisplayCaptureDeviceManager() = default;
+    GStreamerDisplayCaptureDeviceManager();
+    ~GStreamerDisplayCaptureDeviceManager();
+
+    CompletionHandler<void()> m_callback;
+    Vector<CaptureDevice> m_devices;
+#if USE(PIPEWIRE)
+    GRefPtr<XdpPortal> m_portal;
+    GRefPtr<XdpSession> m_session;
+#endif
 };
-
 }
 
 #endif // ENABLE(MEDIA_STREAM)  && USE(GSTREAMER)

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -48,14 +48,16 @@
     : m_device(device.device())
     , m_caps(caps)
     , m_sourceFactory(nullptr)
+    , m_deviceType(device.type())
 {
     initializeDebugCategory();
 }
 
-GStreamerCapturer::GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps> caps)
+GStreamerCapturer::GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps> caps, CaptureDevice::DeviceType deviceType)
     : m_device(nullptr)
     , m_caps(caps)
     , m_sourceFactory(sourceFactory)
+    , m_deviceType(deviceType)
 {
     initializeDebugCategory();
 }
@@ -66,14 +68,54 @@
         disconnectSimpleBusMessageCallback(pipeline());
 }
 
+GStreamerCapturer::Observer::~Observer()
+{
+}
+
+void GStreamerCapturer::addObserver(Observer& observer)
+{
+    ASSERT(isMainThread());
+    m_observers.add(observer);
+}
+
+void GStreamerCapturer::removeObserver(Observer& observer)
+{
+    ASSERT(isMainThread());
+    m_observers.remove(observer);
+}
+
+void GStreamerCapturer::forEachObserver(const Function<void(Observer&)>& apply)
+{
+    ASSERT(isMainThread());
+    auto protectedThis = makeRef(*this);
+    m_observers.forEach(apply);
+}
+
 GstElement* GStreamerCapturer::createSource()
 {
     if (m_sourceFactory) {
         m_src = makeElement(m_sourceFactory);
+        ASSERT(m_src);
         if (GST_IS_APP_SRC(m_src.get()))
             g_object_set(m_src.get(), "is-live", true, "format", GST_FORMAT_TIME, nullptr);
 
-        ASSERT(m_src);
+        if (m_deviceType == CaptureDevice::DeviceType::Screen) {
+            auto pad = adoptGRef(gst_element_get_static_pad(m_src.get(), "src"));
+            gst_pad_add_probe(pad.get(), GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM, [](GstPad*, GstPadProbeInfo* info, void* userData) -> GstPadProbeReturn {
+                auto* event = gst_pad_probe_info_get_event(info);
+                if (GST_EVENT_TYPE(event) != GST_EVENT_CAPS)
+                    return GST_PAD_PROBE_OK;
+
+                callOnMainThread([event, capturer = reinterpret_cast<GStreamerCapturer*>(userData)] {
+                    GstCaps* caps;
+                    gst_event_parse_caps(event, &caps);
+                    capturer->forEachObserver([caps](Observer& observer) {
+                        observer.sourceCapsChanged(caps);
+                    });
+                });
+                return GST_PAD_PROBE_OK;
+            }, this, nullptr);
+        }
         return m_src.get();
     }
 

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerCapturer.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -28,16 +28,30 @@
 #include "GStreamerCommon.h"
 
 #include <gst/gst.h>
+#include <wtf/ThreadSafeRefCounted.h>
+#include <wtf/WeakHashSet.h>
 
 namespace WebCore {
 
-class GStreamerCapturer {
-    WTF_MAKE_FAST_ALLOCATED;
+class GStreamerCapturer
+    : public ThreadSafeRefCounted<GStreamerCapturer, WTF::DestructionThread::MainRunLoop> {
+
 public:
+    class Observer : public CanMakeWeakPtr<Observer> {
+    public:
+        virtual ~Observer();
+
+        virtual void sourceCapsChanged(const GstCaps*) { }
+    };
+
     GStreamerCapturer(GStreamerCaptureDevice, GRefPtr<GstCaps>);
-    GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps>);
-    ~GStreamerCapturer();
+    GStreamerCapturer(const char* sourceFactory, GRefPtr<GstCaps>, CaptureDevice::DeviceType);
+    virtual ~GStreamerCapturer();
 
+    void addObserver(Observer&);
+    void removeObserver(Observer&);
+    void forEachObserver(const Function<void(Observer&)>&);
+
     void setupPipeline();
     void play();
     void stop();
@@ -57,6 +71,8 @@
     bool isInterrupted() const;
     void setInterrupted(bool);
 
+    CaptureDevice::DeviceType deviceType() const { return m_deviceType; }
+
 protected:
     GRefPtr<GstElement> m_sink;
     GRefPtr<GstElement> m_src;
@@ -66,10 +82,11 @@
     GRefPtr<GstDevice> m_device;
     GRefPtr<GstCaps> m_caps;
     GRefPtr<GstElement> m_pipeline;
+    const char* m_sourceFactory;
 
 private:
-    const char* m_sourceFactory;
-
+    CaptureDevice::DeviceType m_deviceType;
+    WeakHashSet<Observer> m_observers;
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -29,6 +29,7 @@
 #include "MediaSampleGStreamer.h"
 
 #include <gst/app/gstappsink.h>
+#include <wtf/text/StringToIntegerConversion.h>
 
 namespace WebCore {
 
@@ -77,10 +78,15 @@
 
 class GStreamerDisplayCaptureSourceFactory final : public DisplayCaptureFactory {
 public:
-    CaptureSourceOrError createDisplayCaptureSource(const CaptureDevice&, const MediaConstraints*) final
+    CaptureSourceOrError createDisplayCaptureSource(const CaptureDevice& device, const MediaConstraints* constraints) final
     {
-        // FIXME: Implement this.
+#if USE(PIPEWIRE)
+        return GStreamerVideoCaptureSource::createPipewireSource(device.persistentId().isolatedCopy(), { }, constraints, device.type());
+#else
+        UNUSED_PARAM(device);
+        UNUSED_PARAM(constraints);
         return { };
+#endif
     }
 private:
     CaptureDeviceManager& displayCaptureDeviceManager() final { return GStreamerDisplayCaptureDeviceManager::singleton(); }
@@ -101,7 +107,16 @@
     }
 
     auto source = adoptRef(*new GStreamerVideoCaptureSource(device.value(), WTFMove(hashSalt)));
+    if (constraints) {
+        if (auto result = source->applyConstraints(*constraints))
+            return WTFMove(result->badConstraint);
+    }
+    return CaptureSourceOrError(WTFMove(source));
+}
 
+CaptureSourceOrError GStreamerVideoCaptureSource::createPipewireSource(String&& deviceID, String&& hashSalt, const MediaConstraints* constraints, CaptureDevice::DeviceType deviceType)
+{
+    auto source = adoptRef(*new GStreamerVideoCaptureSource(WTFMove(deviceID), { }, WTFMove(hashSalt), "pipewiresrc", deviceType));
     if (constraints) {
         if (auto result = source->applyConstraints(*constraints))
             return WTFMove(result->badConstraint);
@@ -119,22 +134,36 @@
     return libWebRTCDisplayCaptureSourceFactory();
 }
 
-GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const gchar *source_factory)
-    : RealtimeVideoCaptureSource(WTFMove(deviceID), WTFMove(name), WTFMove(hashSalt))
-    , m_capturer(makeUnique<GStreamerVideoCapturer>(source_factory))
+GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const gchar* sourceFactory, CaptureDevice::DeviceType deviceType)
+    : RealtimeVideoCaptureSource(WTFMove(name), WTFMove(deviceID), WTFMove(hashSalt))
+    , m_capturer(makeUnique<GStreamerVideoCapturer>(sourceFactory, deviceType))
+    , m_deviceType(deviceType)
 {
     initializeDebugCategory();
+
+    if (g_str_equal(sourceFactory, "pipewiresrc")) {
+        if (auto fd = parseInteger<int>(persistentID()))
+            m_capturer->setPipewireFD(*fd);
+    }
+    m_capturer->addObserver(*this);
 }
 
 GStreamerVideoCaptureSource::GStreamerVideoCaptureSource(GStreamerCaptureDevice device, String&& hashSalt)
     : RealtimeVideoCaptureSource(String { device.persistentId() }, String { device.label() }, WTFMove(hashSalt))
     , m_capturer(makeUnique<GStreamerVideoCapturer>(device))
+    , m_deviceType(CaptureDevice::DeviceType::Camera)
 {
     initializeDebugCategory();
+    m_capturer->addObserver(*this);
 }
 
 GStreamerVideoCaptureSource::~GStreamerVideoCaptureSource()
 {
+    m_capturer->removeObserver(*this);
+    if (!m_capturer->pipeline())
+        return;
+    g_signal_handlers_disconnect_by_func(m_capturer->sink(), reinterpret_cast<gpointer>(newSampleCallback), this);
+    m_capturer->stop();
 }
 
 void GStreamerVideoCaptureSource::settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag> settings)
@@ -145,10 +174,27 @@
         m_capturer->setFrameRate(frameRate());
 }
 
+void GStreamerVideoCaptureSource::sourceCapsChanged(const GstCaps* caps)
+{
+    auto videoResolution = getVideoResolutionFromCaps(caps);
+    if (!videoResolution)
+        return;
+
+    setIntrinsicSize(IntSize(*videoResolution), false);
+    if (m_deviceType == CaptureDevice::DeviceType::Screen)
+        ensureIntrinsicSizeMaintainsAspectRatio();
+}
+
 void GStreamerVideoCaptureSource::startProducingData()
 {
+    if (m_capturer->pipeline())
+        return;
+
     m_capturer->setupPipeline();
-    m_capturer->setSize(size().width(), size().height());
+
+    if (m_deviceType == CaptureDevice::DeviceType::Camera)
+        m_capturer->setSize(size().width(), size().height());
+
     m_capturer->setFrameRate(frameRate());
     g_signal_connect(m_capturer->sink(), "new-sample", G_CALLBACK(newSampleCallback), this);
     m_capturer->play();
@@ -176,9 +222,6 @@
 
 void GStreamerVideoCaptureSource::stopProducingData()
 {
-    g_signal_handlers_disconnect_by_func(m_capturer->sink(), reinterpret_cast<gpointer>(newSampleCallback), this);
-    m_capturer->stop();
-
     GST_INFO("Reset height and width after stopping source");
     setSize({ 0, 0 });
 }

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCaptureSource.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -29,13 +29,14 @@
 
 namespace WebCore {
 
-class GStreamerVideoCaptureSource : public RealtimeVideoCaptureSource {
+class GStreamerVideoCaptureSource : public RealtimeVideoCaptureSource, GStreamerCapturer::Observer {
 public:
     static CaptureSourceOrError create(String&& deviceID, String&& hashSalt, const MediaConstraints*);
+    static CaptureSourceOrError createPipewireSource(String&& deviceID, String&& hashSalt, const MediaConstraints*, CaptureDevice::DeviceType);
+
     WEBCORE_EXPORT static VideoCaptureFactory& factory();
 
-    // FIXME: Implement this.
-    WEBCORE_EXPORT static DisplayCaptureFactory& displayFactory(); 
+    WEBCORE_EXPORT static DisplayCaptureFactory& displayFactory();
 
     const RealtimeMediaSourceCapabilities& capabilities() override;
     const RealtimeMediaSourceSettings& settings() override;
@@ -43,8 +44,11 @@
     GStreamerCapturer* capturer() { return m_capturer.get(); }
     void processNewFrame(Ref<MediaSample>&&);
 
+    // GStreamerCapturer::Observer
+    void sourceCapsChanged(const GstCaps*) final;
+
 protected:
-    GStreamerVideoCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const gchar * source_factory);
+    GStreamerVideoCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const gchar* source_factory, CaptureDevice::DeviceType);
     GStreamerVideoCaptureSource(GStreamerCaptureDevice, String&& hashSalt);
     virtual ~GStreamerVideoCaptureSource();
     void startProducingData() override;
@@ -55,7 +59,7 @@
 
     mutable std::optional<RealtimeMediaSourceCapabilities> m_capabilities;
     mutable std::optional<RealtimeMediaSourceSettings> m_currentSettings;
-    CaptureDevice::DeviceType deviceType() const override { return CaptureDevice::DeviceType::Camera; }
+    CaptureDevice::DeviceType deviceType() const override { return m_deviceType; }
 
 private:
     static GstFlowReturn newSampleCallback(GstElement*, GStreamerVideoCaptureSource*);
@@ -64,6 +68,7 @@
     void settingsDidChange(OptionSet<RealtimeMediaSourceSettings::Flag>) final;
 
     std::unique_ptr<GStreamerVideoCapturer> m_capturer;
+    CaptureDevice::DeviceType m_deviceType;
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -25,18 +25,41 @@
 #if ENABLE(MEDIA_STREAM) && USE(GSTREAMER)
 #include "GStreamerVideoCapturer.h"
 
+GST_DEBUG_CATEGORY(webkit_video_capturer_debug);
+#define GST_CAT_DEFAULT webkit_video_capturer_debug
+
 namespace WebCore {
 
+static void initializeDebugCategory()
+{
+    ensureGStreamerInitialized();
+
+    static std::once_flag debugRegisteredFlag;
+    std::call_once(debugRegisteredFlag, [] {
+        GST_DEBUG_CATEGORY_INIT(webkit_video_capturer_debug, "webkitvideocapturer", 0, "WebKit Video Capturer");
+    });
+}
+
 GStreamerVideoCapturer::GStreamerVideoCapturer(GStreamerCaptureDevice device)
     : GStreamerCapturer(device, adoptGRef(gst_caps_new_empty_simple("video/x-raw")))
 {
+    initializeDebugCategory();
 }
 
-GStreamerVideoCapturer::GStreamerVideoCapturer(const char* sourceFactory)
-    : GStreamerCapturer(sourceFactory, adoptGRef(gst_caps_new_empty_simple("video/x-raw")))
+GStreamerVideoCapturer::GStreamerVideoCapturer(const char* sourceFactory, CaptureDevice::DeviceType deviceType)
+    : GStreamerCapturer(sourceFactory, adoptGRef(gst_caps_new_empty_simple("video/x-raw")), deviceType)
 {
+    initializeDebugCategory();
 }
 
+GstElement* GStreamerVideoCapturer::createSource()
+{
+    auto* src = ""
+    if (m_fd)
+        g_object_set(m_src.get(), "fd", *m_fd, nullptr);
+    return src;
+}
+
 GstElement* GStreamerVideoCapturer::createConverter()
 {
     // https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/97#note_56575
@@ -52,11 +75,29 @@
     return info;
 }
 
+void GStreamerVideoCapturer::setPipewireFD(int fd)
+{
+    m_fd = fd;
+}
+
 bool GStreamerVideoCapturer::setSize(int width, int height)
 {
+    if (m_fd.has_value()) {
+        // Pipewiresrc doesn't seem to support caps re-negotiation and framerate configuration properly.
+        GST_FIXME_OBJECT(m_pipeline.get(), "Resizing disabled on display capture source");
+        return true;
+    }
+
     if (!width || !height)
         return false;
 
+    auto videoResolution = getVideoResolutionFromCaps(m_caps.get());
+    if (videoResolution && videoResolution->width() == width && videoResolution->height() == height) {
+        GST_DEBUG_OBJECT(m_pipeline.get(), "Size has not changed");
+        return true;
+    }
+
+    GST_INFO_OBJECT(m_pipeline.get(), "Setting size to %dx%d", width, height);
     m_caps = adoptGRef(gst_caps_copy(m_caps.get()));
     gst_caps_set_simple(m_caps.get(), "width", G_TYPE_INT, width, "height", G_TYPE_INT, height, nullptr);
 
@@ -64,12 +105,17 @@
         return false;
 
     g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr);
-
     return true;
 }
 
 bool GStreamerVideoCapturer::setFrameRate(double frameRate)
 {
+    if (m_fd.has_value()) {
+        // Pipewiresrc doesn't seem to support caps re-negotiation and framerate configuration properly.
+        GST_FIXME_OBJECT(m_pipeline.get(), "Framerate override disabled on display capture source");
+        return true;
+    }
+
     int numerator, denominator;
 
     gst_util_double_to_fraction(frameRate, &numerator, &denominator);
@@ -90,6 +136,7 @@
     if (!m_capsfilter)
         return false;
 
+    GST_INFO_OBJECT(m_pipeline.get(), "Setting framerate to %f fps", frameRate);
     g_object_set(m_capsfilter.get(), "caps", m_caps.get(), nullptr);
 
     return true;

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/GStreamerVideoCapturer.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -33,8 +33,9 @@
 class GStreamerVideoCapturer final : public GStreamerCapturer {
 public:
     GStreamerVideoCapturer(GStreamerCaptureDevice);
-    GStreamerVideoCapturer(const char* source_factory);
+    GStreamerVideoCapturer(const char* source_factory, CaptureDevice::DeviceType);
 
+    GstElement* createSource() final;
     GstElement* createConverter() final;
     const char* name() final { return "Video"; }
 
@@ -41,6 +42,10 @@
     bool setSize(int width, int height);
     bool setFrameRate(double);
     GstVideoInfo getBestFormat();
+
+    void setPipewireFD(int);
+private:
+    std::optional<int> m_fd;
 };
 
 } // namespace WebCore

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -44,12 +44,23 @@
     auto source = adoptRef(*new MockRealtimeVideoSourceGStreamer(WTFMove(deviceID), WTFMove(name), WTFMove(hashSalt)));
     if (constraints) {
         if (auto error = source->applyConstraints(*constraints))
-            return WTFMove(error->message);
+            return WTFMove(error.value().badConstraint);
     }
 
     return CaptureSourceOrError(RealtimeVideoSource::create(WTFMove(source)));
 }
 
+CaptureSourceOrError MockRealtimeVideoSourceGStreamer::createMockDisplayCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const MediaConstraints* constraints)
+{
+    auto source = MockRealtimeVideoSourceGStreamer::createForMockDisplayCapturer(WTFMove(deviceID), WTFMove(name), WTFMove(hashSalt));
+    if (constraints) {
+        if (auto error = source->applyConstraints(*constraints))
+            return WTFMove(error.value().badConstraint);
+    }
+
+    return CaptureSourceOrError(RealtimeVideoSource::create(WTFMove(source)));
+}
+
 Ref<MockRealtimeVideoSource> MockRealtimeVideoSourceGStreamer::createForMockDisplayCapturer(String&& deviceID, String&& name, String&& hashSalt)
 {
     return adoptRef(*new MockRealtimeVideoSourceGStreamer(WTFMove(deviceID), WTFMove(name), WTFMove(hashSalt)));

Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -32,6 +32,8 @@
 public:
     static Ref<MockRealtimeVideoSource> createForMockDisplayCapturer(String&& deviceID, String&& name, String&& hashSalt);
 
+    static CaptureSourceOrError createMockDisplayCaptureSource(String&& deviceID, String&& name, String&& hashSalt, const MediaConstraints*);
+
     ~MockRealtimeVideoSourceGStreamer() = default;
 
 private:

Modified: trunk/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mock/MockRealtimeMediaSourceCenter.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -47,6 +47,10 @@
 #include "MockRealtimeVideoSourceMac.h"
 #endif
 
+#if USE(GSTREAMER)
+#include "MockRealtimeVideoSourceGStreamer.h"
+#endif
+
 namespace WebCore {
 
 static inline Vector<MockMediaDevice> defaultDevices()
@@ -160,6 +164,8 @@
         case CaptureDevice::DeviceType::Window:
 #if PLATFORM(MAC)
             return DisplayCaptureSourceCocoa::create(UniqueRef<DisplayCaptureSourceCocoa::Capturer>(makeUniqueRef<MockDisplayCapturer>(device)), device, constraints);
+#elif USE(GSTREAMER)
+            return MockRealtimeVideoSourceGStreamer::createMockDisplayCaptureSource(String { device.persistentId() }, String { device.label() }, String { }, constraints);
 #else
             return MockRealtimeVideoSource::create(String { device.persistentId() }, String { device.label() }, String { }, constraints);
 #endif

Modified: trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp (279939 => 279940)


--- trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -131,6 +131,10 @@
             capabilities.setDeviceId(hashedId());
             updateCapabilities(capabilities);
             capabilities.setDeviceId(hashedId());
+        } else if (mockDisplay()) {
+            capabilities.setWidth(CapabilityValueOrRange(72, WTF::get<MockDisplayProperties>(m_device.properties).defaultSize.width()));
+            capabilities.setHeight(CapabilityValueOrRange(45, WTF::get<MockDisplayProperties>(m_device.properties).defaultSize.height()));
+            capabilities.setFrameRate(CapabilityValueOrRange(.01, 60.0));
         } else {
             capabilities.setWidth(CapabilityValueOrRange(72, 2880));
             capabilities.setHeight(CapabilityValueOrRange(45, 1800));
@@ -428,6 +432,9 @@
         m_delayUntil = MonotonicTime();
     }
 
+    if (mockDisplay() && !m_frameNumber)
+        ensureIntrinsicSizeMaintainsAspectRatio();
+
     ImageBuffer* buffer = imageBuffer();
     if (!buffer)
         return;
@@ -435,7 +442,7 @@
     GraphicsContext& context = buffer->context();
     GraphicsContextStateSaver stateSaver(context);
 
-    auto size = captureSize();
+    auto size = this->size();
     FloatRect frameRect(FloatPoint(), size);
 
     context.fillRect(FloatRect(FloatPoint(), size), m_fillColor);

Modified: trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h (279939 => 279940)


--- trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h	2021-07-15 11:48:58 UTC (rev 279940)
@@ -71,7 +71,7 @@
     void startProducingData() final;
     void stopProducingData() final;
     bool isCaptureSource() const final { return true; }
-    CaptureDevice::DeviceType deviceType() const final { return CaptureDevice::DeviceType::Camera; }
+    CaptureDevice::DeviceType deviceType() const final { return mockCamera() ? CaptureDevice::DeviceType::Camera : CaptureDevice::DeviceType::Screen; }
     bool supportsSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
     void setSizeAndFrameRate(std::optional<int> width, std::optional<int> height, std::optional<double>) final;
     void setFrameRateWithPreset(double, RefPtr<VideoPreset>) final;

Added: trunk/Source/cmake/FindLIBPORTAL.cmake (0 => 279940)


--- trunk/Source/cmake/FindLIBPORTAL.cmake	                        (rev 0)
+++ trunk/Source/cmake/FindLIBPORTAL.cmake	2021-07-15 11:48:58 UTC (rev 279940)
@@ -0,0 +1,101 @@
+# - Try to find libportal
+# Once done, this will define
+#
+#  LIBPORTAL_FOUND - system has libportal
+#  LIBPORTAL_INCLUDE_DIRS - the libportal include directories
+#  LIBPORTAL_LIBRARIES - link these to use libportal
+#
+# Copyright (C) 2021 Igalia S.L.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1.  Redistributions of source code must retain the above copyright
+#     notice, this list of conditions and the following disclaimer.
+# 2.  Redistributions in binary form must reproduce the above copyright
+#     notice, this list of conditions and the following disclaimer in the
+#     documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND ITS CONTRIBUTORS ``AS
+# IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
+# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR ITS
+# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
+# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
+# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
+# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
+# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+#[=======================================================================[.rst:
+FindLIBPORTAL
+-------------
+
+Find libportal headers and libraries.
+
+Imported Targets
+^^^^^^^^^^^^^^^^
+
+``LIBPORTAL::LIBPORTAL``
+  The LIBPORTAL library, if found.
+
+Result Variables
+^^^^^^^^^^^^^^^^
+
+This will define the following variables in your project:
+
+``LIBPORTAL_FOUND``
+  true if (the requested version of) LIBPORTAL is available.
+``LIBPORTAL_VERSION``
+  the version of LIBPORTAL.
+``LIBPORTAL_LIBRARIES``
+  the libraries to link against to use LIBPORTAL.
+``LIBPORTAL_INCLUDE_DIRS``
+  where to find the LIBPORTAL headers.
+``LIBPORTAL_COMPILE_OPTIONS``
+  this should be passed to target_compile_options(), if the
+  target is not used for linking
+
+#]=======================================================================]
+
+find_package(PkgConfig)
+pkg_check_modules(PC_LIBPORTAL QUIET libportal)
+set(LIBPORTAL_COMPILE_OPTIONS ${PC_LIBPORTAL_CFLAGS_OTHER})
+set(LIBPORTAL_VERSION ${PC_LIBPORTAL_VERSION})
+
+find_path(LIBPORTAL_INCLUDE_DIR
+    NAMES portal.h
+    HINTS ${PC_LIBPORTAL_INCLUDEDIR}
+          ${PC_LIBPORTAL_INCLUDE_DIRS}
+    PATH_SUFFIXES libportal
+)
+
+find_library(LIBPORTAL_LIBRARY
+    NAMES ${LIBPORTAL_NAMES} libportal
+    HINTS ${PC_LIBPORTAL_LIBDIR}
+          ${PC_LIBPORTAL_LIBRARY_DIRS}
+)
+
+include(FindPackageHandleStandardArgs)
+find_package_handle_standard_args(LIBPORTAL
+    FOUND_VAR LIBPORTAL_FOUND
+    REQUIRED_VARS LIBPORTAL_INCLUDE_DIR LIBPORTAL_LIBRARY
+    VERSION_VAR LIBPORTAL_VERSION
+)
+
+if (LIBPORTAL_LIBRARY AND NOT TARGET LIBPORTAL::LIBPORTAL)
+    add_library(LIBPORTAL::LIBPORTAL UNKNOWN IMPORTED GLOBAL)
+    set_target_properties(LIBPORTAL::LIBPORTAL PROPERTIES
+        IMPORTED_LOCATION "${LIBPORTAL_LIBRARY}"
+        INTERFACE_COMPILE_OPTIONS "${LIBPORTAL_COMPILE_OPTIONS}"
+        INTERFACE_INCLUDE_DIRECTORIES "${LIBPORTAL_INCLUDE_DIR}"
+    )
+endif ()
+
+mark_as_advanced(LIBPORTAL_INCLUDE_DIR LIBPORTAL_LIBRARY)
+
+if (LIBPORTAL_FOUND)
+    set(LIBPORTAL_LIBRARIES ${LIBPORTAL_LIBRARY})
+    set(LIBPORTAL_INCLUDE_DIRS ${LIBPORTAL_INCLUDE_DIR})
+endif ()

Modified: trunk/Source/cmake/GStreamerChecks.cmake (279939 => 279940)


--- trunk/Source/cmake/GStreamerChecks.cmake	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Source/cmake/GStreamerChecks.cmake	2021-07-15 11:48:58 UTC (rev 279940)
@@ -57,3 +57,12 @@
 else ()
     SET_AND_EXPOSE_TO_BUILD(USE_LIBWEBRTC FALSE)
 endif ()
+
+if (ENABLE_MEDIA_STREAM)
+    find_package(LIBPORTAL)
+    if (LIBPORTAL_FOUND)
+        SET_AND_EXPOSE_TO_BUILD(USE_PIPEWIRE TRUE)
+    else ()
+        SET_AND_EXPOSE_TO_BUILD(USE_PIPEWIRE FALSE)
+    endif ()
+endif ()

Modified: trunk/Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebsiteData.cpp (279939 => 279940)


--- trunk/Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebsiteData.cpp	2021-07-15 08:42:07 UTC (rev 279939)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebsiteData.cpp	2021-07-15 11:48:58 UTC (rev 279940)
@@ -635,6 +635,7 @@
     WebKitSettings* settings = webkit_web_view_get_settings(test->m_webView);
     gboolean enabled = webkit_settings_get_enable_media_stream(settings);
     webkit_settings_set_enable_media_stream(settings, TRUE);
+    webkit_settings_set_enable_mock_capture_devices(settings, TRUE);
 
     test->clear(WEBKIT_WEBSITE_DATA_DEVICE_ID_HASH_SALT, 0);
 
@@ -681,6 +682,7 @@
     g_assert_null(dataList);
 
     webkit_settings_set_enable_media_stream(settings, enabled);
+    webkit_settings_set_enable_mock_capture_devices(settings, enabled);
 }
 
 static void testWebsiteDataITP(WebsiteDataTest* test, gconstpointer)
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to