Title: [271036] trunk/Source
Revision
271036
Author
[email protected]
Date
2020-12-21 12:17:09 -0800 (Mon, 21 Dec 2020)

Log Message

Many websites crash under toDataURL() when enabling GPU process for DOM
https://bugs.webkit.org/show_bug.cgi?id=220051
<rdar://problem/72412509>

Reviewed by Simon Fraser.

Source/WebCore:

See WebKit ChangeLog for more details.

* platform/graphics/ImageBufferBackend.h:

Add traits for the PreserveResolution enum flag, so that it can be encoded over IPC.

Source/WebKit:

Three methods of canvas readback (`toDataURL`, `toData`, and `toBGRAData`) currently cause the web content
process to crash when enabling GPU process for DOM rendering. This is because we attempt to call into the
respective methods on the image buffer backend following r270342, which unconditionally fire release assertions.

Instead of using the proxy image buffer backend object in the web process, we need to communicate with the
platform `RemoteImageBuffer` in the GPU process. To do this, we add some plumbing through
`RemoteRenderingBackendProxy` and `RemoteRenderingBackend`, and override the `toDataURL`, `toData`, and
`toBGRAData` methods on `RemoteImageBufferProxy` in the web process to request data from the remote rendering
backend instead.

* GPUProcess/graphics/RemoteRenderingBackend.cpp:
(WebKit::RemoteRenderingBackend::getDataURLForImageBuffer):
(WebKit::RemoteRenderingBackend::getDataForImageBuffer):
(WebKit::RemoteRenderingBackend::getBGRADataForImageBuffer):
* GPUProcess/graphics/RemoteRenderingBackend.h:
* GPUProcess/graphics/RemoteRenderingBackend.messages.in:
* Scripts/webkit/messages.py:
* WebProcess/GPU/graphics/RemoteImageBufferProxy.h:
* WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp:
(WebKit::RemoteRenderingBackendProxy::getImageData):
(WebKit::RemoteRenderingBackendProxy::getDataURLForImageBuffer):
(WebKit::RemoteRenderingBackendProxy::getDataForImageBuffer):
(WebKit::RemoteRenderingBackendProxy::getBGRADataForImageBuffer):
* WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h:

Modified Paths

Diff

Modified: trunk/Source/WebCore/ChangeLog (271035 => 271036)


--- trunk/Source/WebCore/ChangeLog	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebCore/ChangeLog	2020-12-21 20:17:09 UTC (rev 271036)
@@ -1,3 +1,17 @@
+2020-12-21  Wenson Hsieh  <[email protected]>
+
+        Many websites crash under toDataURL() when enabling GPU process for DOM
+        https://bugs.webkit.org/show_bug.cgi?id=220051
+        <rdar://problem/72412509>
+
+        Reviewed by Simon Fraser.
+
+        See WebKit ChangeLog for more details.
+
+        * platform/graphics/ImageBufferBackend.h:
+
+        Add traits for the PreserveResolution enum flag, so that it can be encoded over IPC.
+
 2020-12-21  Megan Gardner  <[email protected]>
 
         Copy Localizable.strings file into the correct directory

Modified: trunk/Source/WebCore/platform/graphics/ImageBufferBackend.h (271035 => 271036)


--- trunk/Source/WebCore/platform/graphics/ImageBufferBackend.h	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebCore/platform/graphics/ImageBufferBackend.h	2020-12-21 20:17:09 UTC (rev 271036)
@@ -181,4 +181,12 @@
     >;
 };
 
+template<> struct EnumTraits<WebCore::PreserveResolution> {
+    using values = EnumValues<
+    WebCore::PreserveResolution,
+    WebCore::PreserveResolution::No,
+    WebCore::PreserveResolution::Yes
+    >;
+};
+
 } // namespace WTF

Modified: trunk/Source/WebKit/ChangeLog (271035 => 271036)


--- trunk/Source/WebKit/ChangeLog	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/ChangeLog	2020-12-21 20:17:09 UTC (rev 271036)
@@ -1,3 +1,36 @@
+2020-12-21  Wenson Hsieh  <[email protected]>
+
+        Many websites crash under toDataURL() when enabling GPU process for DOM
+        https://bugs.webkit.org/show_bug.cgi?id=220051
+        <rdar://problem/72412509>
+
+        Reviewed by Simon Fraser.
+
+        Three methods of canvas readback (`toDataURL`, `toData`, and `toBGRAData`) currently cause the web content
+        process to crash when enabling GPU process for DOM rendering. This is because we attempt to call into the
+        respective methods on the image buffer backend following r270342, which unconditionally fire release assertions.
+
+        Instead of using the proxy image buffer backend object in the web process, we need to communicate with the
+        platform `RemoteImageBuffer` in the GPU process. To do this, we add some plumbing through
+        `RemoteRenderingBackendProxy` and `RemoteRenderingBackend`, and override the `toDataURL`, `toData`, and
+        `toBGRAData` methods on `RemoteImageBufferProxy` in the web process to request data from the remote rendering
+        backend instead.
+
+        * GPUProcess/graphics/RemoteRenderingBackend.cpp:
+        (WebKit::RemoteRenderingBackend::getDataURLForImageBuffer):
+        (WebKit::RemoteRenderingBackend::getDataForImageBuffer):
+        (WebKit::RemoteRenderingBackend::getBGRADataForImageBuffer):
+        * GPUProcess/graphics/RemoteRenderingBackend.h:
+        * GPUProcess/graphics/RemoteRenderingBackend.messages.in:
+        * Scripts/webkit/messages.py:
+        * WebProcess/GPU/graphics/RemoteImageBufferProxy.h:
+        * WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp:
+        (WebKit::RemoteRenderingBackendProxy::getImageData):
+        (WebKit::RemoteRenderingBackendProxy::getDataURLForImageBuffer):
+        (WebKit::RemoteRenderingBackendProxy::getDataForImageBuffer):
+        (WebKit::RemoteRenderingBackendProxy::getBGRADataForImageBuffer):
+        * WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h:
+
 2020-12-21  Kate Cheney  <[email protected]>
 
         Missing textInteractionEnabled key in WKPreferences initWithCoder looks like a false value.

Modified: trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp (271035 => 271036)


--- trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.cpp	2020-12-21 20:17:09 UTC (rev 271036)
@@ -278,6 +278,30 @@
     completionHandler(IPC::ImageDataReference(WTFMove(imageData)));
 }
 
+void RemoteRenderingBackend::getDataURLForImageBuffer(const String& mimeType, Optional<double> quality, WebCore::PreserveResolution preserveResolution, WebCore::RenderingResourceIdentifier renderingResourceIdentifier, CompletionHandler<void(String&&)>&& completionHandler)
+{
+    String urlString;
+    if (auto imageBuffer = m_remoteResourceCache.cachedImageBuffer(renderingResourceIdentifier))
+        urlString = imageBuffer->toDataURL(mimeType, quality, preserveResolution);
+    completionHandler(WTFMove(urlString));
+}
+
+void RemoteRenderingBackend::getDataForImageBuffer(const String& mimeType, Optional<double> quality, WebCore::RenderingResourceIdentifier renderingResourceIdentifier, CompletionHandler<void(Vector<uint8_t>&&)>&& completionHandler)
+{
+    Vector<uint8_t> data;
+    if (auto imageBuffer = m_remoteResourceCache.cachedImageBuffer(renderingResourceIdentifier))
+        data = "" quality);
+    completionHandler(WTFMove(data));
+}
+
+void RemoteRenderingBackend::getBGRADataForImageBuffer(WebCore::RenderingResourceIdentifier renderingResourceIdentifier, CompletionHandler<void(Vector<uint8_t>&&)>&& completionHandler)
+{
+    Vector<uint8_t> data;
+    if (auto imageBuffer = m_remoteResourceCache.cachedImageBuffer(renderingResourceIdentifier))
+        data = ""
+    completionHandler(WTFMove(data));
+}
+
 void RemoteRenderingBackend::cacheNativeImage(const ShareableBitmap::Handle& handle, RenderingResourceIdentifier renderingResourceIdentifier)
 {
     auto bitmap = ShareableBitmap::create(handle);

Modified: trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h (271035 => 271036)


--- trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.h	2020-12-21 20:17:09 UTC (rev 271036)
@@ -109,6 +109,9 @@
     void createImageBuffer(const WebCore::FloatSize& logicalSize, WebCore::RenderingMode, float resolutionScale, WebCore::ColorSpace, WebCore::PixelFormat, WebCore::RenderingResourceIdentifier);
     void wakeUpAndApplyDisplayList(const GPUProcessWakeupMessageArguments&);
     void getImageData(WebCore::AlphaPremultiplication outputFormat, WebCore::IntRect srcRect, WebCore::RenderingResourceIdentifier, CompletionHandler<void(IPC::ImageDataReference&&)>&&);
+    void getDataURLForImageBuffer(const String& mimeType, Optional<double> quality, WebCore::PreserveResolution, WebCore::RenderingResourceIdentifier, CompletionHandler<void(String&&)>&&);
+    void getDataForImageBuffer(const String& mimeType, Optional<double> quality, WebCore::RenderingResourceIdentifier, CompletionHandler<void(Vector<uint8_t>&&)>&&);
+    void getBGRADataForImageBuffer(WebCore::RenderingResourceIdentifier, CompletionHandler<void(Vector<uint8_t>&&)>&&);
     void cacheNativeImage(const ShareableBitmap::Handle&, WebCore::RenderingResourceIdentifier);
     void cacheFont(Ref<WebCore::Font>&&);
     void deleteAllFonts();

Modified: trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in (271035 => 271036)


--- trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/GPUProcess/graphics/RemoteRenderingBackend.messages.in	2020-12-21 20:17:09 UTC (rev 271036)
@@ -26,6 +26,9 @@
     CreateImageBuffer(WebCore::FloatSize logicalSize, WebCore::RenderingMode renderingMode, float resolutionScale, WebCore::ColorSpace colorSpace, enum:uint8_t WebCore::PixelFormat pixelFormat, WebCore::RenderingResourceIdentifier renderingResourceIdentifier)
     WakeUpAndApplyDisplayList(struct WebKit::GPUProcessWakeupMessageArguments arguments)
     GetImageData(enum:uint8_t WebCore::AlphaPremultiplication outputFormat, WebCore::IntRect srcRect, WebCore::RenderingResourceIdentifier renderingResourceIdentifier) -> (IPC::ImageDataReference imageData) Synchronous
+    GetDataURLForImageBuffer(String mimeType, Optional<double> quality, enum:uint8_t WebCore::PreserveResolution preserveResolution, WebCore::RenderingResourceIdentifier renderingResourceIdentifier) -> (String urlString) Synchronous
+    GetDataForImageBuffer(String mimeType, Optional<double> quality, WebCore::RenderingResourceIdentifier renderingResourceIdentifier) -> (Vector<uint8_t> data) Synchronous
+    GetBGRADataForImageBuffer(WebCore::RenderingResourceIdentifier renderingResourceIdentifier) -> (Vector<uint8_t> data) Synchronous
     CacheNativeImage(WebKit::ShareableBitmap::Handle handle, WebCore::RenderingResourceIdentifier renderingResourceIdentifier)
     CacheFont(IPC::FontReference font)
     DeleteAllFonts()

Modified: trunk/Source/WebKit/Scripts/webkit/messages.py (271035 => 271036)


--- trunk/Source/WebKit/Scripts/webkit/messages.py	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/Scripts/webkit/messages.py	2020-12-21 20:17:09 UTC (rev 271036)
@@ -638,6 +638,7 @@
         'WebCore::PluginInfo': ['<WebCore/PluginData.h>'],
         'WebCore::PluginLoadClientPolicy': ['<WebCore/PluginData.h>'],
         'WebCore::PolicyAction': ['<WebCore/FrameLoaderTypes.h>'],
+        'WebCore::PreserveResolution': ['<WebCore/ImageBufferBackend.h>'],
         'WebCore::ShouldContinuePolicyCheck': ['<WebCore/FrameLoaderTypes.h>'],
         'WebCore::PolicyCheckIdentifier': ['<WebCore/FrameLoaderTypes.h>'],
         'WebCore::ProcessIdentifier': ['<WebCore/ProcessIdentifier.h>'],

Modified: trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h (271035 => 271036)


--- trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteImageBufferProxy.h	2020-12-21 20:17:09 UTC (rev 271036)
@@ -146,10 +146,33 @@
         if (UNLIKELY(!m_remoteRenderingBackendProxy))
             return nullptr;
 
-        m_remoteRenderingBackendProxy->sendDeferredWakeupMessageIfNeeded();
         return m_remoteRenderingBackendProxy->getImageData(outputFormat, srcRect, m_renderingResourceIdentifier);
     }
 
+    String toDataURL(const String& mimeType, Optional<double> quality, WebCore::PreserveResolution preserveResolution) const override
+    {
+        if (UNLIKELY(!m_remoteRenderingBackendProxy))
+            return { };
+
+        return m_remoteRenderingBackendProxy->getDataURLForImageBuffer(mimeType, quality, preserveResolution, m_renderingResourceIdentifier);
+    }
+
+    Vector<uint8_t> toData(const String& mimeType, Optional<double> quality = WTF::nullopt) const override
+    {
+        if (UNLIKELY(!m_remoteRenderingBackendProxy))
+            return { };
+
+        return m_remoteRenderingBackendProxy->getDataForImageBuffer(mimeType, quality, m_renderingResourceIdentifier);
+    }
+
+    Vector<uint8_t> toBGRAData() const override
+    {
+        if (UNLIKELY(!m_remoteRenderingBackendProxy))
+            return { };
+
+        return m_remoteRenderingBackendProxy->getBGRADataForImageBuffer(m_renderingResourceIdentifier);
+    }
+
     void putImageData(WebCore::AlphaPremultiplication inputFormat, const WebCore::ImageData& imageData, const WebCore::IntRect& srcRect, const WebCore::IntPoint& destPoint = { }, WebCore::AlphaPremultiplication destFormat = WebCore::AlphaPremultiplication::Premultiplied) override
     {
         // The math inside ImageData::create() doesn't agree with the math inside ImageBufferBackend::putImageData() about how m_resolutionScale interacts with the data in the ImageBuffer.

Modified: trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp (271035 => 271036)


--- trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.cpp	2020-12-21 20:17:09 UTC (rev 271036)
@@ -146,11 +146,40 @@
 
 RefPtr<ImageData> RemoteRenderingBackendProxy::getImageData(AlphaPremultiplication outputFormat, const IntRect& srcRect, RenderingResourceIdentifier renderingResourceIdentifier)
 {
+    sendDeferredWakeupMessageIfNeeded();
+
     IPC::ImageDataReference imageDataReference;
     sendSync(Messages::RemoteRenderingBackend::GetImageData(outputFormat, srcRect, renderingResourceIdentifier), Messages::RemoteRenderingBackend::GetImageData::Reply(imageDataReference), m_renderingBackendIdentifier, 1_s);
     return imageDataReference.buffer();
 }
 
+String RemoteRenderingBackendProxy::getDataURLForImageBuffer(const String& mimeType, Optional<double> quality, PreserveResolution preserveResolution, RenderingResourceIdentifier renderingResourceIdentifier)
+{
+    sendDeferredWakeupMessageIfNeeded();
+
+    String urlString;
+    sendSync(Messages::RemoteRenderingBackend::GetDataURLForImageBuffer(mimeType, quality, preserveResolution, renderingResourceIdentifier), Messages::RemoteRenderingBackend::GetDataURLForImageBuffer::Reply(urlString), m_renderingBackendIdentifier, 1_s);
+    return urlString;
+}
+
+Vector<uint8_t> RemoteRenderingBackendProxy::getDataForImageBuffer(const String& mimeType, Optional<double> quality, RenderingResourceIdentifier renderingResourceIdentifier)
+{
+    sendDeferredWakeupMessageIfNeeded();
+
+    Vector<uint8_t> data;
+    sendSync(Messages::RemoteRenderingBackend::GetDataForImageBuffer(mimeType, quality, renderingResourceIdentifier), Messages::RemoteRenderingBackend::GetDataForImageBuffer::Reply(data), m_renderingBackendIdentifier, 1_s);
+    return data;
+}
+
+Vector<uint8_t> RemoteRenderingBackendProxy::getBGRADataForImageBuffer(RenderingResourceIdentifier renderingResourceIdentifier)
+{
+    sendDeferredWakeupMessageIfNeeded();
+
+    Vector<uint8_t> data;
+    sendSync(Messages::RemoteRenderingBackend::GetBGRADataForImageBuffer(renderingResourceIdentifier), Messages::RemoteRenderingBackend::GetBGRADataForImageBuffer::Reply(data), m_renderingBackendIdentifier, 1_s);
+    return data;
+}
+
 void RemoteRenderingBackendProxy::cacheNativeImage(const ShareableBitmap::Handle& handle, RenderingResourceIdentifier renderingResourceIdentifier)
 {
     send(Messages::RemoteRenderingBackend::CacheNativeImage(handle, renderingResourceIdentifier), m_renderingBackendIdentifier);

Modified: trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h (271035 => 271036)


--- trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h	2020-12-21 19:56:50 UTC (rev 271035)
+++ trunk/Source/WebKit/WebProcess/GPU/graphics/RemoteRenderingBackendProxy.h	2020-12-21 20:17:09 UTC (rev 271036)
@@ -81,6 +81,9 @@
     // Messages to be sent.
     RefPtr<WebCore::ImageBuffer> createImageBuffer(const WebCore::FloatSize&, WebCore::RenderingMode, float resolutionScale, WebCore::ColorSpace, WebCore::PixelFormat);
     RefPtr<WebCore::ImageData> getImageData(WebCore::AlphaPremultiplication outputFormat, const WebCore::IntRect& srcRect, WebCore::RenderingResourceIdentifier);
+    String getDataURLForImageBuffer(const String& mimeType, Optional<double> quality, WebCore::PreserveResolution, WebCore::RenderingResourceIdentifier);
+    Vector<uint8_t> getDataForImageBuffer(const String& mimeType, Optional<double> quality, WebCore::RenderingResourceIdentifier);
+    Vector<uint8_t> getBGRADataForImageBuffer(WebCore::RenderingResourceIdentifier);
     WebCore::DisplayList::FlushIdentifier flushDisplayListAndCommit(const WebCore::DisplayList::DisplayList&, WebCore::RenderingResourceIdentifier);
     void cacheNativeImage(const ShareableBitmap::Handle&, WebCore::RenderingResourceIdentifier);
     void cacheFont(Ref<WebCore::Font>&&);
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to