Log Message
[WebRTC][GStreamer] fast/mediastream/resize-trim.html is failing https://bugs.webkit.org/show_bug.cgi?id=192888
Patch by Philippe Normand <[email protected]> on 2021-05-04 Reviewed by Xabier Rodriguez-Calvar. Source/WebCore: The test was failing because the GStreamer mock video source was producing garbled frames and the pixel test in the canvas was failing to detect gray. The frame was garbled because the test asked "exact" video width that was inferior to the capture video width and we were not taking this into account when creating a MediaSample from the video frame. MediaSampleGStreamer::createImageSample() can now optionally resize the video frame according to the destinationSize, if it was specified. * html/HTMLCanvasElement.cpp: (WebCore::HTMLCanvasElement::toMediaSample): * platform/graphics/gstreamer/MediaSampleGStreamer.cpp: (WebCore::MediaSampleGStreamer::createImageSample): * platform/graphics/gstreamer/MediaSampleGStreamer.h: (WebCore::MediaSampleGStreamer::createImageSample): * platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp: (WebCore::MockRealtimeVideoSourceGStreamer::updateSampleBuffer): * platform/mock/MockRealtimeVideoSource.h: LayoutTests: * platform/glib/TestExpectations: Unflag test. * resources/platform-helper.js: Remove EFL cruft. (videoCanvasPixelComparisonTolerance): (isEfl): Deleted.
Modified Paths
- trunk/LayoutTests/ChangeLog
- trunk/LayoutTests/platform/glib/TestExpectations
- trunk/LayoutTests/resources/platform-helper.js
- trunk/Source/WebCore/ChangeLog
- trunk/Source/WebCore/html/HTMLCanvasElement.cpp
- trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.cpp
- trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.h
- trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp
- trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h
Diff
Modified: trunk/LayoutTests/ChangeLog (276952 => 276953)
--- trunk/LayoutTests/ChangeLog 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/LayoutTests/ChangeLog 2021-05-04 11:13:37 UTC (rev 276953)
@@ -1,3 +1,15 @@
+2021-05-04 Philippe Normand <[email protected]>
+
+ [WebRTC][GStreamer] fast/mediastream/resize-trim.html is failing
+ https://bugs.webkit.org/show_bug.cgi?id=192888
+
+ Reviewed by Xabier Rodriguez-Calvar.
+
+ * platform/glib/TestExpectations: Unflag test.
+ * resources/platform-helper.js: Remove EFL cruft.
+ (videoCanvasPixelComparisonTolerance):
+ (isEfl): Deleted.
+
2021-05-03 Simon Fraser <[email protected]>
REGRESSION (r274461): [macOS wk1] fast/repaint/canvas-object-fit.html is flakey text failing
Modified: trunk/LayoutTests/platform/glib/TestExpectations (276952 => 276953)
--- trunk/LayoutTests/platform/glib/TestExpectations 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/LayoutTests/platform/glib/TestExpectations 2021-05-04 11:13:37 UTC (rev 276953)
@@ -2044,7 +2044,6 @@
webkit.org/b/191497 http/tests/inspector/network/resource-security-certificate.html [ Skip ]
webkit.org/b/192150 imported/w3c/web-platform-tests/xhr/setrequestheader-content-type.htm [ Failure ]
webkit.org/b/192407 http/tests/inspector/network/resource-security-connection.html [ Skip ]
-webkit.org/b/192888 fast/mediastream/resize-trim.html [ Failure ]
webkit.org/b/192900 imported/w3c/web-platform-tests/eventsource/format-mime-bogus.htm [ Failure ]
webkit.org/b/192903 imported/w3c/web-platform-tests/service-workers/service-worker/fetch-request-redirect.https.html [ Pass Failure ]
webkit.org/b/193323 http/wpt/cache-storage/cache-quota.any.html [ Failure ]
Modified: trunk/LayoutTests/resources/platform-helper.js (276952 => 276953)
--- trunk/LayoutTests/resources/platform-helper.js 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/LayoutTests/resources/platform-helper.js 2021-05-04 11:13:37 UTC (rev 276953)
@@ -3,14 +3,9 @@
return navigator.userAgent.includes("WebKitTestRunnerGTK");
}
-function isEfl()
-{
- return navigator.userAgent.includes("WebKitTestRunnerEFL");
-}
-
function videoCanvasPixelComparisonTolerance()
{
- if (isGtk() || isEfl())
+ if (isGtk())
return 6;
return 2;
Modified: trunk/Source/WebCore/ChangeLog (276952 => 276953)
--- trunk/Source/WebCore/ChangeLog 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/Source/WebCore/ChangeLog 2021-05-04 11:13:37 UTC (rev 276953)
@@ -1,3 +1,28 @@
+2021-05-04 Philippe Normand <[email protected]>
+
+ [WebRTC][GStreamer] fast/mediastream/resize-trim.html is failing
+ https://bugs.webkit.org/show_bug.cgi?id=192888
+
+ Reviewed by Xabier Rodriguez-Calvar.
+
+ The test was failing because the GStreamer mock video source was producing garbled frames
+ and the pixel test in the canvas was failing to detect gray. The frame was garbled because
+ the test asked "exact" video width that was inferior to the capture video width and we were
+ not taking this into account when creating a MediaSample from the video frame.
+
+ MediaSampleGStreamer::createImageSample() can now optionally resize the video frame
+ according to the destinationSize, if it was specified.
+
+ * html/HTMLCanvasElement.cpp:
+ (WebCore::HTMLCanvasElement::toMediaSample):
+ * platform/graphics/gstreamer/MediaSampleGStreamer.cpp:
+ (WebCore::MediaSampleGStreamer::createImageSample):
+ * platform/graphics/gstreamer/MediaSampleGStreamer.h:
+ (WebCore::MediaSampleGStreamer::createImageSample):
+ * platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp:
+ (WebCore::MockRealtimeVideoSourceGStreamer::updateSampleBuffer):
+ * platform/mock/MockRealtimeVideoSource.h:
+
2021-05-04 Youenn Fablet <[email protected]>
Use OptionSet for MediaProducer::MediaStateFlags
Modified: trunk/Source/WebCore/html/HTMLCanvasElement.cpp (276952 => 276953)
--- trunk/Source/WebCore/html/HTMLCanvasElement.cpp 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/Source/WebCore/html/HTMLCanvasElement.cpp 2021-05-04 11:13:37 UTC (rev 276953)
@@ -819,7 +819,7 @@
return MediaSampleAVFObjC::createImageSample(imageBuffer->toBGRAData(), width(), height());
#elif USE(GSTREAMER)
makeRenderingResultsAvailable();
- return MediaSampleGStreamer::createImageSample(imageBuffer->toBGRAData(), width(), height());
+ return MediaSampleGStreamer::createImageSample(imageBuffer->toBGRAData(), size());
#else
return nullptr;
#endif
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.cpp (276952 => 276953)
--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.cpp 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.cpp 2021-05-04 11:13:37 UTC (rev 276953)
@@ -97,15 +97,18 @@
return adoptRef(*gstreamerMediaSample);
}
-Ref<MediaSampleGStreamer> MediaSampleGStreamer::createImageSample(Vector<uint8_t>&& bgraData, unsigned width, unsigned height, double frameRate)
+Ref<MediaSampleGStreamer> MediaSampleGStreamer::createImageSample(Vector<uint8_t>&& bgraData, const IntSize& size, const IntSize& destinationSize, double frameRate)
{
ensureGStreamerInitialized();
- size_t size = bgraData.sizeInBytes();
+ size_t sizeInBytes = bgraData.sizeInBytes();
auto* data = ""
- auto buffer = adoptGRef(gst_buffer_new_wrapped_full(GST_MEMORY_FLAG_READONLY, data, size, 0, size, data, [](gpointer data) {
+ auto buffer = adoptGRef(gst_buffer_new_wrapped_full(GST_MEMORY_FLAG_READONLY, data, sizeInBytes, 0, sizeInBytes, data, [](gpointer data) {
WTF::VectorMalloc::free(data);
}));
+
+ auto width = size.width();
+ auto height = size.height();
gst_buffer_add_video_meta(buffer.get(), GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_BGRA, width, height);
int frameRateNumerator, frameRateDenominator;
@@ -114,6 +117,29 @@
auto caps = adoptGRef(gst_caps_new_simple("video/x-raw", "format", G_TYPE_STRING, "BGRA", "width", G_TYPE_INT, width,
"height", G_TYPE_INT, height, "framerate", GST_TYPE_FRACTION, frameRateNumerator, frameRateDenominator, nullptr));
auto sample = adoptGRef(gst_sample_new(buffer.get(), caps.get(), nullptr, nullptr));
+
+ // Optionally resize the video frame to fit destinationSize. This code path is used mostly by
+ // the mock realtime video source when the gUM constraints specifically required exact width
+ // and/or height values.
+ if (!destinationSize.isZero()) {
+ GstVideoInfo inputInfo;
+ gst_video_info_from_caps(&inputInfo, caps.get());
+
+ width = destinationSize.width();
+ height = destinationSize.height();
+ auto outputCaps = adoptGRef(gst_caps_new_simple("video/x-raw", "format", G_TYPE_STRING, "BGRA", "width", G_TYPE_INT, width,
+ "height", G_TYPE_INT, height, "framerate", GST_TYPE_FRACTION, frameRateNumerator, frameRateDenominator, nullptr));
+ GstVideoInfo outputInfo;
+ gst_video_info_from_caps(&outputInfo, outputCaps.get());
+
+ auto outputBuffer = adoptGRef(gst_buffer_new_allocate(nullptr, GST_VIDEO_INFO_SIZE(&outputInfo), nullptr));
+ gst_buffer_add_video_meta(outputBuffer.get(), GST_VIDEO_FRAME_FLAG_NONE, GST_VIDEO_FORMAT_BGRA, width, height);
+ GUniquePtr<GstVideoConverter> converter(gst_video_converter_new(&inputInfo, &outputInfo, nullptr));
+ GstMappedFrame inputFrame(gst_sample_get_buffer(sample.get()), inputInfo, GST_MAP_READ);
+ GstMappedFrame outputFrame(outputBuffer.get(), outputInfo, GST_MAP_WRITE);
+ gst_video_converter_frame(converter.get(), inputFrame.get(), outputFrame.get());
+ sample = adoptGRef(gst_sample_new(outputBuffer.get(), outputCaps.get(), nullptr, nullptr));
+ }
return create(WTFMove(sample), FloatSize(width, height), { });
}
Modified: trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.h (276952 => 276953)
--- trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.h 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/Source/WebCore/platform/graphics/gstreamer/MediaSampleGStreamer.h 2021-05-04 11:13:37 UTC (rev 276953)
@@ -37,7 +37,7 @@
}
static Ref<MediaSampleGStreamer> createFakeSample(GstCaps*, MediaTime pts, MediaTime dts, MediaTime duration, const FloatSize& presentationSize, const AtomString& trackId);
- static Ref<MediaSampleGStreamer> createImageSample(Vector<uint8_t>&&, unsigned width, unsigned height, double frameRate = 1);
+ static Ref<MediaSampleGStreamer> createImageSample(Vector<uint8_t>&&, const IntSize& originalSize, const IntSize& destinationSize = { }, double frameRate = 1);
void extendToTheBeginning();
MediaTime presentationTime() const override { return m_pts; }
Modified: trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp (276952 => 276953)
--- trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/Source/WebCore/platform/mediastream/gstreamer/MockRealtimeVideoSourceGStreamer.cpp 2021-05-04 11:13:37 UTC (rev 276953)
@@ -65,8 +65,7 @@
if (!imageBuffer)
return;
- auto imageSize = size();
- auto sample = MediaSampleGStreamer::createImageSample(imageBuffer->toBGRAData(), imageSize.width(), imageSize.height(), frameRate());
+ auto sample = MediaSampleGStreamer::createImageSample(imageBuffer->toBGRAData(), captureSize(), size(), frameRate());
sample->offsetTimestampsBy(MediaTime::createWithDouble((elapsedTime() + 100_ms).seconds()));
dispatchMediaSampleToObservers(sample.get());
}
Modified: trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h (276952 => 276953)
--- trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h 2021-05-04 10:04:29 UTC (rev 276952)
+++ trunk/Source/WebCore/platform/mock/MockRealtimeVideoSource.h 2021-05-04 11:13:37 UTC (rev 276953)
@@ -62,6 +62,8 @@
MediaSample::VideoRotation sampleRotation() const final { return m_deviceOrientation; }
void generatePresets() override;
+ IntSize captureSize() const;
+
private:
const RealtimeMediaSourceCapabilities& capabilities() final;
const RealtimeMediaSourceSettings& settings() final;
@@ -73,7 +75,6 @@
bool supportsSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) final;
void setSizeAndFrameRate(Optional<int> width, Optional<int> height, Optional<double>) final;
void setFrameRateWithPreset(double, RefPtr<VideoPreset>) final;
- IntSize captureSize() const;
bool isMockSource() const final { return true; }
_______________________________________________ webkit-changes mailing list [email protected] https://lists.webkit.org/mailman/listinfo/webkit-changes
