Diff
Modified: trunk/ChangeLog (290365 => 290366)
--- trunk/ChangeLog 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/ChangeLog 2022-02-23 12:27:24 UTC (rev 290366)
@@ -1,3 +1,14 @@
+2022-02-23 Zan Dobersek <[email protected]>
+
+ [GStreamer] Add WebKitDMABufVideoSink
+ https://bugs.webkit.org/show_bug.cgi?id=236883
+
+ Reviewed by Philippe Normand.
+
+ * Source/cmake/FindGStreamer.cmake:
+ Also search for gstreamer-allocators-1.0, produce the relevant variables
+ containing include directories and cflags.
+
2022-02-22 Philippe Normand <[email protected]>
[GStreamer] Initial MediaRecorder implementation
Modified: trunk/Source/WebCore/ChangeLog (290365 => 290366)
--- trunk/Source/WebCore/ChangeLog 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Source/WebCore/ChangeLog 2022-02-23 12:27:24 UTC (rev 290366)
@@ -1,3 +1,61 @@
+2022-02-23 Zan Dobersek <[email protected]>
+
+ [GStreamer] Add WebKitDMABufVideoSink
+ https://bugs.webkit.org/show_bug.cgi?id=236883
+
+ Reviewed by Philippe Normand.
+
+ Add custom WebKitDMABufVideoSink sink that is able to accept decoded
+ dmabuf or raw data in a range of RGB-like or YUV formats.
+
+ The sink is pretty straightforward, in large part mirroring the
+ WebKitGLVideoSink implementation. The formats that (at the moment) we
+ can correctly handle inside the TextureMapper-based graphics pipeline
+ are combined with both the dmabuf or raw memory type during the caps
+ construction. Like the GL counterpart, the received samples are passed
+ on to the associated MediaPlayerPrivateGStreamer object. The common
+ integration into the MediaPlayerPrivateGStreamer functionality is
+ encapsulated into the webKitVideoSinkSetMediaPlayerPrivate() function.
+
+ The MediaPlayerPrivateGStreamer implementation will be enhanced later to
+ properly handle samples conforming to either variant of these caps. But
+ the idea is that when using this sink, a dmabuf-containing sample will
+ pass the dmabuf to the graphics subsystem and a raw-data-containing
+ sample will copy the data into a dmabuf object (created or reused) and
+ then pass it on.
+
+ The WEBKIT_GST_DMABUF_SINK_FORCED_FALLBACK_CAPS_FORMAT environment
+ variable can be set to specify the desired exclusive format that this
+ sink should support. This is especially helpful to debug the correct
+ handling of different RGB or YUV formats inside the graphics pipeline.
+ When set, only raw data of the specified format is accepted, meaning the
+ pipeline upstream will perform any necessary conversion of the decoded
+ data into the desired format. Raw data is then packed into dmabufs and
+ sent into the pipeline for display, enabling the developer to examine
+ whether the given format is handled appropriately.
+
+ The sink element is registered in registerWebKitGStreamerElements().
+ This functionality will require The GStreamer allocators library, so the
+ necessary dependency glue is also provided.
+
+ * platform/GStreamer.cmake:
+ * platform/graphics/gstreamer/DMABufVideoSinkGStreamer.cpp: Added.
+ (forcedFallbackCapsFormat):
+ (webKitDMABufVideoSinkConstructed):
+ (webKitDMABufVideoSinkFinalize):
+ (webKitDMABufVideoSinkGetProperty):
+ (webkit_dmabuf_video_sink_class_init):
+ (webKitDMABufVideoSinkSetMediaPlayerPrivate):
+ (webKitDMABufVideoSinkProbePlatform):
+ * platform/graphics/gstreamer/DMABufVideoSinkGStreamer.h: Added.
+ * platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp:
+ (webKitGLVideoSinkSetMediaPlayerPrivate):
+ * platform/graphics/gstreamer/GStreamerCommon.cpp:
+ (WebCore::registerWebKitGStreamerElements):
+ * platform/graphics/gstreamer/GStreamerVideoSinkCommon.cpp: Added.
+ (webKitVideoSinkSetMediaPlayerPrivate):
+ * platform/graphics/gstreamer/GStreamerVideoSinkCommon.h: Added.
+
2022-02-23 Sihui Liu <[email protected]>
REGRESSION(r289474): [iOS] ASSERTION FAILED: isMainThread() under WebCore::IDBServer::UniqueIDBDatabase::abortActiveTransactions()
Modified: trunk/Source/WebCore/platform/GStreamer.cmake (290365 => 290366)
--- trunk/Source/WebCore/platform/GStreamer.cmake 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Source/WebCore/platform/GStreamer.cmake 2022-02-23 12:27:24 UTC (rev 290366)
@@ -9,6 +9,7 @@
list(APPEND WebCore_SOURCES
Modules/webaudio/MediaStreamAudioSourceGStreamer.cpp
platform/graphics/gstreamer/AudioTrackPrivateGStreamer.cpp
+ platform/graphics/gstreamer/DMABufVideoSinkGStreamer.cpp
platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp
platform/graphics/gstreamer/GRefPtrGStreamer.cpp
platform/graphics/gstreamer/GStreamerAudioMixer.cpp
@@ -16,6 +17,7 @@
platform/graphics/gstreamer/GstAllocatorFastMalloc.cpp
platform/graphics/gstreamer/GStreamerRegistryScanner.cpp
platform/graphics/gstreamer/GStreamerVideoFrameHolder.cpp
+ platform/graphics/gstreamer/GStreamerVideoSinkCommon.cpp
platform/graphics/gstreamer/ImageDecoderGStreamer.cpp
platform/graphics/gstreamer/InbandTextTrackPrivateGStreamer.cpp
platform/graphics/gstreamer/MediaEngineConfigurationFactoryGStreamer.cpp
@@ -94,11 +96,13 @@
list(APPEND WebCore_SYSTEM_INCLUDE_DIRECTORIES
${GSTREAMER_INCLUDE_DIRS}
${GSTREAMER_BASE_INCLUDE_DIRS}
+ ${GSTREAMER_ALLOCATORS_INCLUDE_DIRS}
${GSTREAMER_APP_INCLUDE_DIRS}
${GSTREAMER_PBUTILS_INCLUDE_DIRS}
)
list(APPEND WebCore_LIBRARIES
+ ${GSTREAMER_ALLOCATORS_LIBRARIES}
${GSTREAMER_APP_LIBRARIES}
${GSTREAMER_BASE_LIBRARIES}
${GSTREAMER_LIBRARIES}
Added: trunk/Source/WebCore/platform/graphics/gstreamer/DMABufVideoSinkGStreamer.cpp (0 => 290366)
--- trunk/Source/WebCore/platform/graphics/gstreamer/DMABufVideoSinkGStreamer.cpp (rev 0)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/DMABufVideoSinkGStreamer.cpp 2022-02-23 12:27:24 UTC (rev 290366)
@@ -0,0 +1,176 @@
+/*
+ * Copyright (C) 2022 Igalia, S.L
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "config.h"
+#include "DMABufVideoSinkGStreamer.h"
+
+#if ENABLE(VIDEO)
+
+#include "GStreamerCommon.h"
+#include "GStreamerVideoSinkCommon.h"
+#include <gst/allocators/gstdmabuf.h>
+#include <mutex>
+#include <wtf/glib/WTFGType.h>
+
+using namespace WebCore;
+
+enum {
+ PROP_0,
+ PROP_STATS,
+ PROP_LAST
+};
+
+struct _WebKitDMABufVideoSinkPrivate {
+ GRefPtr<GstElement> appSink;
+ MediaPlayerPrivateGStreamer* mediaPlayerPrivate;
+};
+
+GST_DEBUG_CATEGORY_STATIC(webkit_dmabuf_video_sink_debug);
+#define GST_CAT_DEFAULT webkit_dmabuf_video_sink_debug
+
+#define GST_WEBKIT_DMABUF_SINK_CAPS_FORMAT_LIST "{ RGBA, RGBx, BGRA, BGRx, I420, YV12, A420, NV12, NV12, Y444, Y42B, VUYA }"
+static GstStaticPadTemplate sinkTemplate = GST_STATIC_PAD_TEMPLATE("sink", GST_PAD_SINK, GST_PAD_ALWAYS, GST_STATIC_CAPS_ANY);
+
+// TODO: this is a list of remaining YUV formats we want to support, but don't currently work (due to improper handling in TextureMapper):
+// YUY2, YVYU, UYVY, VYUY, AYUV, Y41B
+
+#define webkit_dmabuf_video_sink_parent_class parent_class
+WEBKIT_DEFINE_TYPE_WITH_CODE(WebKitDMABufVideoSink, webkit_dmabuf_video_sink, GST_TYPE_BIN,
+ GST_DEBUG_CATEGORY_INIT(webkit_dmabuf_video_sink_debug, "webkitdmabufvideosink", 0, "DMABuf video sink element"))
+
+// WEBKIT_GST_DMABUF_SINK_FORCED_FALLBACK_CAPS_FORMAT env can be used to force a specific format to be used as the only supported format
+// by this sink. This is most useful for testing and debugging the rendering pipeline behavior for a given format.
+static const char* forcedFallbackCapsFormat()
+{
+ static char s_format[64] { };
+ static std::once_flag s_flag;
+ std::call_once(s_flag,
+ [&] {
+ const char* format = g_getenv("WEBKIT_GST_DMABUF_SINK_FORCED_FALLBACK_CAPS_FORMAT");
+ if (format)
+ g_strlcpy(const_cast<char*>(s_format), format, 64);
+ else
+ s_format[0] = 0;
+ });
+
+ if (!s_format[0])
+ return nullptr;
+ return s_format;
+}
+
+static void webKitDMABufVideoSinkConstructed(GObject* object)
+{
+ GST_CALL_PARENT(G_OBJECT_CLASS, constructed, (object));
+
+ WebKitDMABufVideoSink* sink = WEBKIT_DMABUF_VIDEO_SINK(object);
+
+ sink->priv->appSink = makeGStreamerElement("appsink", "webkit-dmabuf-video-appsink");
+ ASSERT(sink->priv->appSink);
+ g_object_set(sink->priv->appSink.get(), "enable-last-sample", FALSE, "emit-signals", TRUE, "max-buffers", 1, nullptr);
+
+ gst_bin_add(GST_BIN_CAST(sink), sink->priv->appSink.get());
+
+ // This sink handles dmabuf data or raw data of any format in the supported format list.
+ // The dmabuf and raw data types are the two types of data we can handle via this sink (in combination with functionality in
+ // MediaPlayerPrivateGStreamer). The format list corresponds to the formats we are able to then handle in the graphics pipeline.
+ // In case of dmabuf data, that dmabuf is handled most optimally and just relayed to the graphics pipeline.
+ // In case of raw data, dmabuf objects are produced on the spot and filled with that data, and then pushed to the graphics pipeline.
+ static GstStaticCaps s_dmabufCaps = GST_STATIC_CAPS(
+ GST_VIDEO_CAPS_MAKE_WITH_FEATURES(GST_CAPS_FEATURE_MEMORY_DMABUF, GST_WEBKIT_DMABUF_SINK_CAPS_FORMAT_LIST));
+ static GstStaticCaps s_fallbackCaps = GST_STATIC_CAPS(GST_VIDEO_CAPS_MAKE(GST_WEBKIT_DMABUF_SINK_CAPS_FORMAT_LIST));
+
+ GRefPtr<GstCaps> caps = adoptGRef(gst_caps_new_empty());
+ {
+ if (forcedFallbackCapsFormat()) {
+ caps = gst_caps_new_simple("video/x-raw",
+ "format", G_TYPE_STRING, forcedFallbackCapsFormat(), nullptr);
+ } else {
+ gst_caps_append(caps.get(), gst_static_caps_get(&s_dmabufCaps));
+ gst_caps_append(caps.get(), gst_static_caps_get(&s_fallbackCaps));
+ }
+ }
+ g_object_set(sink->priv->appSink.get(), "caps", caps.get(), nullptr);
+
+ GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(sink->priv->appSink.get(), "sink"));
+ gst_element_add_pad(GST_ELEMENT_CAST(sink), gst_ghost_pad_new("sink", pad.get()));
+}
+
+void webKitDMABufVideoSinkFinalize(GObject* object)
+{
+ ASSERT(isMainThread());
+
+ WebKitDMABufVideoSink* sink = WEBKIT_DMABUF_VIDEO_SINK(object);
+ WebKitDMABufVideoSinkPrivate* priv = sink->priv;
+
+ if (priv->mediaPlayerPrivate)
+ g_signal_handlers_disconnect_by_data(priv->appSink.get(), priv->mediaPlayerPrivate);
+
+ GST_DEBUG_OBJECT(object, "WebKitDMABufVideoSink finalized.");
+
+ GST_CALL_PARENT(G_OBJECT_CLASS, finalize, (object));
+}
+
+static void webKitDMABufVideoSinkGetProperty(GObject* object, guint propertyId, GValue* value, GParamSpec* paramSpec)
+{
+ WebKitDMABufVideoSink* sink = WEBKIT_DMABUF_VIDEO_SINK(object);
+
+ switch (propertyId) {
+ case PROP_STATS:
+ if (webkitGstCheckVersion(1, 18, 0)) {
+ GUniqueOutPtr<GstStructure> stats;
+ g_object_get(sink->priv->appSink.get(), "stats", &stats.outPtr(), nullptr);
+ gst_value_set_structure(value, stats.get());
+ }
+ break;
+ default:
+ G_OBJECT_WARN_INVALID_PROPERTY_ID(object, propertyId, paramSpec);
+ RELEASE_ASSERT_NOT_REACHED();
+ break;
+ }
+}
+
+static void webkit_dmabuf_video_sink_class_init(WebKitDMABufVideoSinkClass* klass)
+{
+ GObjectClass* objectClass = G_OBJECT_CLASS(klass);
+ GstElementClass* elementClass = GST_ELEMENT_CLASS(klass);
+
+ objectClass->constructed = webKitDMABufVideoSinkConstructed;
+ objectClass->finalize = webKitDMABufVideoSinkFinalize;
+ objectClass->get_property = webKitDMABufVideoSinkGetProperty;
+
+ gst_element_class_add_pad_template(elementClass, gst_static_pad_template_get(&sinkTemplate));
+ gst_element_class_set_static_metadata(elementClass, "WebKit DMABuf video sink", "Sink/Video", "Renders video", "Zan Dobersek <[email protected]>");
+
+ g_object_class_install_property(objectClass, PROP_STATS, g_param_spec_boxed("stats", "Statistics",
+ "Sink Statistics", GST_TYPE_STRUCTURE, static_cast<GParamFlags>(G_PARAM_READABLE | G_PARAM_STATIC_STRINGS)));
+}
+
+void webKitDMABufVideoSinkSetMediaPlayerPrivate(WebKitDMABufVideoSink* sink, MediaPlayerPrivateGStreamer* player)
+{
+ WebKitDMABufVideoSinkPrivate* priv = sink->priv;
+
+ priv->mediaPlayerPrivate = player;
+ webKitVideoSinkSetMediaPlayerPrivate(priv->appSink.get(), priv->mediaPlayerPrivate);
+}
+
+bool webKitDMABufVideoSinkProbePlatform()
+{
+ return isGStreamerPluginAvailable("app");
+}
+
+#endif // ENABLE(VIDEO)
Added: trunk/Source/WebCore/platform/graphics/gstreamer/DMABufVideoSinkGStreamer.h (0 => 290366)
--- trunk/Source/WebCore/platform/graphics/gstreamer/DMABufVideoSinkGStreamer.h (rev 0)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/DMABufVideoSinkGStreamer.h 2022-02-23 12:27:24 UTC (rev 290366)
@@ -0,0 +1,58 @@
+/*
+ * Copyright (C) 2022 Igalia, S.L
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#pragma once
+
+#if ENABLE(VIDEO)
+
+#include <gst/gst.h>
+
+namespace WebCore {
+class MediaPlayerPrivateGStreamer;
+}
+
+G_BEGIN_DECLS
+
+#define WEBKIT_TYPE_DMABUF_VIDEO_SINK (webkit_dmabuf_video_sink_get_type ())
+#define WEBKIT_DMABUF_VIDEO_SINK(obj) (G_TYPE_CHECK_INSTANCE_CAST ((obj), WEBKIT_TYPE_DMABUF_VIDEO_SINK, WebKitDMABufVideoSink))
+#define WEBKIT_DMABUF_VIDEO_SINK_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST ((klass), WEBKIT_TYPE_DMABUF_VIDEO_SINK, WebKitDMABufVideoSinkClass))
+#define WEBKIT_IS_DMABUF_VIDEO_SINK(obj) (G_TYPE_CHECK_INSTANCE_TYPE ((obj), WEBKIT_TYPE_DMABUF_VIDEO_SINK))
+#define WEBKIT_IS_DMABUF_VIDEO_SINK_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), WEBKIT_TYPE_DMABUF_VIDEO_SINK))
+
+typedef struct _WebKitDMABufVideoSink WebKitDMABufVideoSink;
+typedef struct _WebKitDMABufVideoSinkClass WebKitDMABufVideoSinkClass;
+typedef struct _WebKitDMABufVideoSinkPrivate WebKitDMABufVideoSinkPrivate;
+
+struct _WebKitDMABufVideoSink {
+ GstBin parent;
+
+ WebKitDMABufVideoSinkPrivate *priv;
+};
+
+struct _WebKitDMABufVideoSinkClass {
+ GstBinClass parentClass;
+};
+
+GType webkit_dmabuf_video_sink_get_type(void);
+
+bool webKitDMABufVideoSinkProbePlatform();
+void webKitDMABufVideoSinkSetMediaPlayerPrivate(WebKitDMABufVideoSink*, WebCore::MediaPlayerPrivateGStreamer*);
+
+G_END_DECLS
+
+#endif // ENABLE(VIDEO)
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp (290365 => 290366)
--- trunk/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp 2022-02-23 12:27:24 UTC (rev 290366)
@@ -23,10 +23,9 @@
#if ENABLE(VIDEO) && USE(GSTREAMER_GL)
#include "GStreamerCommon.h"
-#include "MediaPlayerPrivateGStreamer.h"
+#include "GStreamerVideoSinkCommon.h"
#include "PlatformDisplay.h"
-
-#include <gst/app/gstappsink.h>
+#include <gst/gl/gl.h>
#include <wtf/glib/WTFGType.h>
// gstglapi.h may include eglplatform.h and it includes X.h, which
@@ -242,53 +241,7 @@
WebKitGLVideoSinkPrivate* priv = sink->priv;
priv->mediaPlayerPrivate = player;
- g_signal_connect(priv->appSink.get(), "new-sample", G_CALLBACK(+[](GstElement* sink, MediaPlayerPrivateGStreamer* player) -> GstFlowReturn {
- GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink)));
- GstBuffer* buffer = gst_sample_get_buffer(sample.get());
- GST_TRACE_OBJECT(sink, "new-sample with PTS=%" GST_TIME_FORMAT, GST_TIME_ARGS(GST_BUFFER_PTS(buffer)));
- player->triggerRepaint(sample.get());
- return GST_FLOW_OK;
- }), player);
- g_signal_connect(priv->appSink.get(), "new-preroll", G_CALLBACK(+[](GstElement* sink, MediaPlayerPrivateGStreamer* player) -> GstFlowReturn {
- GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_preroll(GST_APP_SINK(sink)));
- GstBuffer* buffer = gst_sample_get_buffer(sample.get());
- GST_DEBUG_OBJECT(sink, "new-preroll with PTS=%" GST_TIME_FORMAT, GST_TIME_ARGS(GST_BUFFER_PTS(buffer)));
- player->triggerRepaint(sample.get());
- return GST_FLOW_OK;
- }), player);
-
- GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(priv->appSink.get(), "sink"));
- gst_pad_add_probe(pad.get(), static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_PUSH | GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM | GST_PAD_PROBE_TYPE_EVENT_FLUSH | GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM), [](GstPad*, GstPadProbeInfo* info, gpointer userData) -> GstPadProbeReturn {
- auto* player = static_cast<MediaPlayerPrivateGStreamer*>(userData);
-
- if (info->type & GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM) {
- if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_EVENT(info)) != GST_EVENT_TAG)
- return GST_PAD_PROBE_OK;
- GstTagList* tagList;
- gst_event_parse_tag(GST_PAD_PROBE_INFO_EVENT(info), &tagList);
- player->updateVideoOrientation(tagList);
- return GST_PAD_PROBE_OK;
- }
-
- // In some platforms (e.g. OpenMAX on the Raspberry Pi) when a resolution change occurs the
- // pipeline has to be drained before a frame with the new resolution can be decoded.
- // In this context, it's important that we don't hold references to any previous frame
- // (e.g. m_sample) so that decoding can continue.
- // We are also not supposed to keep the original frame after a flush.
- if (info->type & GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM) {
- if (GST_QUERY_TYPE(GST_PAD_PROBE_INFO_QUERY(info)) != GST_QUERY_DRAIN)
- return GST_PAD_PROBE_OK;
- GST_DEBUG("Acting upon DRAIN query");
- }
- if (info->type & GST_PAD_PROBE_TYPE_EVENT_FLUSH) {
- if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_EVENT(info)) != GST_EVENT_FLUSH_START)
- return GST_PAD_PROBE_OK;
- GST_DEBUG("Acting upon flush-start event");
- }
-
- player->flushCurrentBuffer();
- return GST_PAD_PROBE_OK;
- }, player, nullptr);
+ webKitVideoSinkSetMediaPlayerPrivate(priv->appSink.get(), priv->mediaPlayerPrivate);
}
bool webKitGLVideoSinkProbePlatform()
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp (290365 => 290366)
--- trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerCommon.cpp 2022-02-23 12:27:24 UTC (rev 290366)
@@ -24,6 +24,7 @@
#if USE(GSTREAMER)
#include "ApplicationGLib.h"
+#include "DMABufVideoSinkGStreamer.h"
#include "GLVideoSinkGStreamer.h"
#include "GStreamerAudioMixer.h"
#include "GUniquePtrGStreamer.h"
@@ -335,6 +336,7 @@
#if ENABLE(VIDEO)
gst_element_register(0, "webkitwebsrc", GST_RANK_PRIMARY + 100, WEBKIT_TYPE_WEB_SRC);
+ gst_element_register(0, "webkitdmabufvideosink", GST_RANK_NONE, WEBKIT_TYPE_DMABUF_VIDEO_SINK);
#if USE(GSTREAMER_GL)
gst_element_register(0, "webkitglvideosink", GST_RANK_NONE, WEBKIT_TYPE_GL_VIDEO_SINK);
#endif
Added: trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerVideoSinkCommon.cpp (0 => 290366)
--- trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerVideoSinkCommon.cpp (rev 0)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerVideoSinkCommon.cpp 2022-02-23 12:27:24 UTC (rev 290366)
@@ -0,0 +1,80 @@
+/*
+ * Copyright (C) 2022 Igalia, S.L
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "config.h"
+#include "GStreamerVideoSinkCommon.h"
+
+#if ENABLE(VIDEO)
+
+#include "MediaPlayerPrivateGStreamer.h"
+#include <gst/app/gstappsink.h>
+
+using namespace WebCore;
+
+void webKitVideoSinkSetMediaPlayerPrivate(GstElement* appSink, MediaPlayerPrivateGStreamer* player)
+{
+ g_signal_connect(appSink, "new-sample", G_CALLBACK(+[](GstElement* sink, MediaPlayerPrivateGStreamer* player) -> GstFlowReturn {
+ GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_sample(GST_APP_SINK(sink)));
+ GstBuffer* buffer = gst_sample_get_buffer(sample.get());
+ GST_TRACE_OBJECT(sink, "new-sample with PTS=%" GST_TIME_FORMAT, GST_TIME_ARGS(GST_BUFFER_PTS(buffer)));
+ player->triggerRepaint(sample.get());
+ return GST_FLOW_OK;
+ }), player);
+ g_signal_connect(appSink, "new-preroll", G_CALLBACK(+[](GstElement* sink, MediaPlayerPrivateGStreamer* player) -> GstFlowReturn {
+ GRefPtr<GstSample> sample = adoptGRef(gst_app_sink_pull_preroll(GST_APP_SINK(sink)));
+ GstBuffer* buffer = gst_sample_get_buffer(sample.get());
+ GST_DEBUG_OBJECT(sink, "new-preroll with PTS=%" GST_TIME_FORMAT, GST_TIME_ARGS(GST_BUFFER_PTS(buffer)));
+ player->triggerRepaint(sample.get());
+ return GST_FLOW_OK;
+ }), player);
+
+ GRefPtr<GstPad> pad = adoptGRef(gst_element_get_static_pad(appSink, "sink"));
+ gst_pad_add_probe(pad.get(), static_cast<GstPadProbeType>(GST_PAD_PROBE_TYPE_PUSH | GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM | GST_PAD_PROBE_TYPE_EVENT_FLUSH | GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM), [](GstPad*, GstPadProbeInfo* info, gpointer userData) -> GstPadProbeReturn {
+ auto* player = static_cast<MediaPlayerPrivateGStreamer*>(userData);
+
+ if (info->type & GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM) {
+ if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_EVENT(info)) != GST_EVENT_TAG)
+ return GST_PAD_PROBE_OK;
+ GstTagList* tagList;
+ gst_event_parse_tag(GST_PAD_PROBE_INFO_EVENT(info), &tagList);
+ player->updateVideoOrientation(tagList);
+ return GST_PAD_PROBE_OK;
+ }
+
+ // In some platforms (e.g. OpenMAX on the Raspberry Pi) when a resolution change occurs the
+ // pipeline has to be drained before a frame with the new resolution can be decoded.
+ // In this context, it's important that we don't hold references to any previous frame
+ // (e.g. m_sample) so that decoding can continue.
+ // We are also not supposed to keep the original frame after a flush.
+ if (info->type & GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM) {
+ if (GST_QUERY_TYPE(GST_PAD_PROBE_INFO_QUERY(info)) != GST_QUERY_DRAIN)
+ return GST_PAD_PROBE_OK;
+ GST_DEBUG("Acting upon DRAIN query");
+ }
+ if (info->type & GST_PAD_PROBE_TYPE_EVENT_FLUSH) {
+ if (GST_EVENT_TYPE(GST_PAD_PROBE_INFO_EVENT(info)) != GST_EVENT_FLUSH_START)
+ return GST_PAD_PROBE_OK;
+ GST_DEBUG("Acting upon flush-start event");
+ }
+
+ player->flushCurrentBuffer();
+ return GST_PAD_PROBE_OK;
+ }, player, nullptr);
+}
+
+#endif // ENABLE(VIDEO)
Added: trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerVideoSinkCommon.h (0 => 290366)
--- trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerVideoSinkCommon.h (rev 0)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/GStreamerVideoSinkCommon.h 2022-02-23 12:27:24 UTC (rev 290366)
@@ -0,0 +1,31 @@
+/*
+ * Copyright (C) 2022 Igalia, S.L
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#pragma once
+
+#if ENABLE(VIDEO)
+
+#include <gst/gst.h>
+
+namespace WebCore {
+class MediaPlayerPrivateGStreamer;
+}
+
+void webKitVideoSinkSetMediaPlayerPrivate(GstElement*, WebCore::MediaPlayerPrivateGStreamer*);
+
+#endif // ENABLE(VIDEO)
Modified: trunk/Source/cmake/FindGStreamer.cmake (290365 => 290366)
--- trunk/Source/cmake/FindGStreamer.cmake 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Source/cmake/FindGStreamer.cmake 2022-02-23 12:27:24 UTC (rev 290366)
@@ -16,6 +16,7 @@
# plugins can be searched, and they define the following variables if
# found:
#
+# gstreamer-allocators: GSTREAMER_ALLOCATORS_INCLUDE_DIRS and GSTREAMER_ALLOCATORS_LIBRARIES
# gstreamer-app: GSTREAMER_APP_INCLUDE_DIRS and GSTREAMER_APP_LIBRARIES
# gstreamer-audio: GSTREAMER_AUDIO_INCLUDE_DIRS and GSTREAMER_AUDIO_LIBRARIES
# gstreamer-fft: GSTREAMER_FFT_INCLUDE_DIRS and GSTREAMER_FFT_LIBRARIES
@@ -98,6 +99,7 @@
# 2. Find GStreamer plugins
# -------------------------
+FIND_GSTREAMER_COMPONENT(GSTREAMER_ALLOCATORS gstreamer-allocators-1.0 gstallocators-1.0)
FIND_GSTREAMER_COMPONENT(GSTREAMER_APP gstreamer-app-1.0 gstapp-1.0)
FIND_GSTREAMER_COMPONENT(GSTREAMER_AUDIO gstreamer-audio-1.0 gstaudio-1.0)
FIND_GSTREAMER_COMPONENT(GSTREAMER_FFT gstreamer-fft-1.0 gstfft-1.0)
@@ -126,6 +128,8 @@
VERSION_VAR GSTREAMER_VERSION)
mark_as_advanced(
+ GSTREAMER_ALLOCATORS_INCLUDE_DIRS
+ GSTREAMER_ALLOCATORS_LIBRARIES
GSTREAMER_APP_INCLUDE_DIRS
GSTREAMER_APP_LIBRARIES
GSTREAMER_AUDIO_INCLUDE_DIRS
Modified: trunk/Tools/ChangeLog (290365 => 290366)
--- trunk/Tools/ChangeLog 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Tools/ChangeLog 2022-02-23 12:27:24 UTC (rev 290366)
@@ -1,3 +1,13 @@
+2022-02-23 Zan Dobersek <[email protected]>
+
+ [GStreamer] Add WebKitDMABufVideoSink
+ https://bugs.webkit.org/show_bug.cgi?id=236883
+
+ Reviewed by Philippe Normand.
+
+ * Scripts/webkitpy/style/checker.py:
+ Add two additional files under the GObject-style exceptions.
+
2022-02-23 Kimmo Kinnunen <[email protected]>
Thread safety analysis macros are confusing for non-Lock use-cases
Modified: trunk/Tools/Scripts/webkitpy/style/checker.py (290365 => 290366)
--- trunk/Tools/Scripts/webkitpy/style/checker.py 2022-02-23 12:06:12 UTC (rev 290365)
+++ trunk/Tools/Scripts/webkitpy/style/checker.py 2022-02-23 12:27:24 UTC (rev 290366)
@@ -230,6 +230,8 @@
# variables and functions containing underscores.
os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'GLVideoSinkGStreamer.cpp'),
os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'GLVideoSinkGStreamer.h'),
+ os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'DMABufVideoSinkGStreamer.cpp'),
+ os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'DMABufVideoSinkGStreamer.h'),
os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'TextCombinerGStreamer.cpp'),
os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'TextCombinerGStreamer.h'),
os.path.join('Source', 'WebCore', 'platform', 'graphics', 'gstreamer', 'TextCombinerPadGStreamer.cpp'),
