Title: [284834] branches/safari-612-branch/Source/WebKit
- Revision
- 284834
- Author
- [email protected]
- Date
- 2021-10-25 14:51:10 -0700 (Mon, 25 Oct 2021)
Log Message
Cherry-pick r284102. rdar://problem/84630200
WebGL video texture upload is very slow due to excessive transfer of the video pixel buffer
https://bugs.webkit.org/show_bug.cgi?id=231425
Patch by Kimmo Kinnunen <[email protected]> on 2021-10-13
Reviewed by Youenn Fablet.
Typical WebGL content requests the videos to be uploaded to a texture
once per render loop update, even though the video has not changed.
The video pixel buffer is slow to transfer across IPC. It should be transferred
only when it has changed.
MediaPlayerPrivateAVFoundationObjC and MediaPlayerPrivateMediaSourceAVFObjC hold
the last requested pixel buffer ref. They will update it or discard it only
when pixelBufferForCurrentTime() is called.
Cache the pixel buffer to MediaPlayerPrivateRemote (GPUP side proxy)
Cache the pixel buffer to RemoteMediaPlayerProxy (WebP side proxy)
The caches increase the total memory use only during the duration
of the call, since after the ref has been updated, it refers
to the one always held in the original objects (MediaPlayerPrivateAVFoundationObjC
and MediaPlayerPrivateMediaSourceAVFObjC)
No new tests, fixes a perf regression wrt GPUP media.
* GPUProcess/media/RemoteMediaPlayerProxy.cpp:
(WebKit::RemoteMediaPlayerProxy::invalidate):
* GPUProcess/media/RemoteMediaPlayerProxy.h:
* GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
* GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
(WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
(WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged):
(WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTime): Deleted.
* WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
* WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
(WebKit::MediaPlayerPrivateRemote::pixelBufferForCurrentTime):
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@284102 268f45cc-cd09-0410-ab3c-d52691b4dbfc
Modified Paths
Diff
Modified: branches/safari-612-branch/Source/WebKit/ChangeLog (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/ChangeLog 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/ChangeLog 2021-10-25 21:51:10 UTC (rev 284834)
@@ -1,5 +1,86 @@
2021-10-25 Null <[email protected]>
+ Cherry-pick r284102. rdar://problem/84630200
+
+ WebGL video texture upload is very slow due to excessive transfer of the video pixel buffer
+ https://bugs.webkit.org/show_bug.cgi?id=231425
+
+ Patch by Kimmo Kinnunen <[email protected]> on 2021-10-13
+ Reviewed by Youenn Fablet.
+
+ Typical WebGL content requests the videos to be uploaded to a texture
+ once per render loop update, even though the video has not changed.
+ The video pixel buffer is slow to transfer across IPC. It should be transferred
+ only when it has changed.
+
+ MediaPlayerPrivateAVFoundationObjC and MediaPlayerPrivateMediaSourceAVFObjC hold
+ the last requested pixel buffer ref. They will update it or discard it only
+ when pixelBufferForCurrentTime() is called.
+
+ Cache the pixel buffer to MediaPlayerPrivateRemote (GPUP side proxy)
+ Cache the pixel buffer to RemoteMediaPlayerProxy (WebP side proxy)
+
+ The caches increase the total memory use only during the duration
+ of the call, since after the ref has been updated, it refers
+ to the one always held in the original objects (MediaPlayerPrivateAVFoundationObjC
+ and MediaPlayerPrivateMediaSourceAVFObjC)
+
+ No new tests, fixes a perf regression wrt GPUP media.
+
+ * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
+ (WebKit::RemoteMediaPlayerProxy::invalidate):
+ * GPUProcess/media/RemoteMediaPlayerProxy.h:
+ * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
+ * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
+ (WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
+ (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged):
+ (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTime): Deleted.
+ * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
+ * WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
+ (WebKit::MediaPlayerPrivateRemote::pixelBufferForCurrentTime):
+
+ git-svn-id: https://svn.webkit.org/repository/webkit/trunk@284102 268f45cc-cd09-0410-ab3c-d52691b4dbfc
+
+ 2021-10-13 Kimmo Kinnunen <[email protected]>
+
+ WebGL video texture upload is very slow due to excessive transfer of the video pixel buffer
+ https://bugs.webkit.org/show_bug.cgi?id=231425
+
+ Reviewed by Youenn Fablet.
+
+ Typical WebGL content requests the videos to be uploaded to a texture
+ once per render loop update, even though the video has not changed.
+ The video pixel buffer is slow to transfer across IPC. It should be transferred
+ only when it has changed.
+
+ MediaPlayerPrivateAVFoundationObjC and MediaPlayerPrivateMediaSourceAVFObjC hold
+ the last requested pixel buffer ref. They will update it or discard it only
+ when pixelBufferForCurrentTime() is called.
+
+ Cache the pixel buffer to MediaPlayerPrivateRemote (GPUP side proxy)
+ Cache the pixel buffer to RemoteMediaPlayerProxy (WebP side proxy)
+
+ The caches increase the total memory use only during the duration
+ of the call, since after the ref has been updated, it refers
+ to the one always held in the original objects (MediaPlayerPrivateAVFoundationObjC
+ and MediaPlayerPrivateMediaSourceAVFObjC)
+
+ No new tests, fixes a perf regression wrt GPUP media.
+
+ * GPUProcess/media/RemoteMediaPlayerProxy.cpp:
+ (WebKit::RemoteMediaPlayerProxy::invalidate):
+ * GPUProcess/media/RemoteMediaPlayerProxy.h:
+ * GPUProcess/media/RemoteMediaPlayerProxy.messages.in:
+ * GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm:
+ (WebKit::RemoteMediaPlayerProxy::nativeImageForCurrentTime):
+ (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged):
+ (WebKit::RemoteMediaPlayerProxy::pixelBufferForCurrentTime): Deleted.
+ * WebProcess/GPU/media/MediaPlayerPrivateRemote.h:
+ * WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm:
+ (WebKit::MediaPlayerPrivateRemote::pixelBufferForCurrentTime):
+
+2021-10-25 Null <[email protected]>
+
Cherry-pick r283975. rdar://problem/84630618
Error the encoder in case we cannot create a RemoteVideoSample
Modified: branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.cpp 2021-10-25 21:51:10 UTC (rev 284834)
@@ -104,6 +104,9 @@
m_sandboxExtension = nullptr;
}
m_renderingResourcesRequest = { };
+#if USE(AVFOUNDATION)
+ m_pixelBufferForCurrentTime = nullptr;
+#endif
}
void RemoteMediaPlayerProxy::getConfiguration(RemoteMediaPlayerConfiguration& configuration)
Modified: branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.h 2021-10-25 21:51:10 UTC (rev 284834)
@@ -311,7 +311,7 @@
void nativeImageForCurrentTime(CompletionHandler<void(std::optional<WTF::MachSendRight>&&)>&&);
#endif
#if USE(AVFOUNDATION)
- void pixelBufferForCurrentTime(CompletionHandler<void(RetainPtr<CVPixelBufferRef>&&)>&&);
+ void pixelBufferForCurrentTimeIfChanged(CompletionHandler<void(std::optional<RetainPtr<CVPixelBufferRef>>&&)>&&);
#endif
#if !RELEASE_LOG_DISABLED
@@ -366,7 +366,9 @@
ScopedRenderingResourcesRequest m_renderingResourcesRequest;
bool m_observingTimeChanges { false };
-
+#if USE(AVFOUNDATION)
+ RetainPtr<CVPixelBufferRef> m_pixelBufferForCurrentTime;
+#endif
#if !RELEASE_LOG_DISABLED
const Logger& m_logger;
#endif
Modified: branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/GPUProcess/media/RemoteMediaPlayerProxy.messages.in 2021-10-25 21:51:10 UTC (rev 284834)
@@ -126,7 +126,7 @@
NativeImageForCurrentTime() -> (std::optional<MachSendRight> sendRight) Synchronous
#endif
#if USE(AVFOUNDATION)
- PixelBufferForCurrentTime() -> (RetainPtr<CVPixelBufferRef> pixelBuffer) Synchronous
+ PixelBufferForCurrentTimeIfChanged() -> (std::optional<RetainPtr<CVPixelBufferRef>> pixelBuffer) Synchronous
#endif
PlayAtHostTime(MonotonicTime time)
Modified: branches/safari-612-branch/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/GPUProcess/media/cocoa/RemoteMediaPlayerProxyCocoa.mm 2021-10-25 21:51:10 UTC (rev 284834)
@@ -111,13 +111,20 @@
completionHandler(surface->createSendRight());
}
-void RemoteMediaPlayerProxy::pixelBufferForCurrentTime(CompletionHandler<void(RetainPtr<CVPixelBufferRef>&&)>&& completionHandler)
+#if USE(AVFOUNDATION)
+void RemoteMediaPlayerProxy::pixelBufferForCurrentTimeIfChanged(CompletionHandler<void(std::optional<RetainPtr<CVPixelBufferRef>>&&)>&& completionHandler)
{
- RetainPtr<CVPixelBufferRef> result;
+ std::optional<RetainPtr<CVPixelBufferRef>> result;
+ RetainPtr<CVPixelBufferRef> pixelBuffer;
if (m_player)
- result = m_player->pixelBufferForCurrentTime();
+ pixelBuffer = m_player->pixelBufferForCurrentTime();
+ if (m_pixelBufferForCurrentTime != pixelBuffer) {
+ result = pixelBuffer;
+ m_pixelBufferForCurrentTime = WTFMove(pixelBuffer);
+ }
completionHandler(WTFMove(result));
}
+#endif
} // namespace WebKit
Modified: branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/MediaPlayerPrivateRemote.h 2021-10-25 21:51:10 UTC (rev 284834)
@@ -445,6 +445,9 @@
bool m_waitingForKey { false };
bool m_timeIsProgressing { false };
bool m_renderingCanBeAccelerated { false };
+#if USE(AVFOUNDATION)
+ RetainPtr<CVPixelBufferRef> m_pixelBufferForCurrentTime;
+#endif
};
} // namespace WebKit
Modified: branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm (284833 => 284834)
--- branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm 2021-10-25 21:51:06 UTC (rev 284833)
+++ branches/safari-612-branch/Source/WebKit/WebProcess/GPU/media/cocoa/MediaPlayerPrivateRemoteCocoa.mm 2021-10-25 21:51:10 UTC (rev 284834)
@@ -66,14 +66,17 @@
return NativeImage::create(WTFMove(platformImage));
}
+#if USE(AVFOUNDATION)
RetainPtr<CVPixelBufferRef> MediaPlayerPrivateRemote::pixelBufferForCurrentTime()
{
-
- RetainPtr<CVPixelBufferRef> result;
- if (!connection().sendSync(Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTime(), Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTime::Reply(result), m_id))
+ std::optional<RetainPtr<CVPixelBufferRef>> result;
+ if (!connection().sendSync(Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTimeIfChanged(), Messages::RemoteMediaPlayerProxy::PixelBufferForCurrentTimeIfChanged::Reply(result), m_id))
return nullptr;
- return result;
+ if (result)
+ m_pixelBufferForCurrentTime = WTFMove(*result);
+ return m_pixelBufferForCurrentTime;
}
+#endif
} // namespace WebKit
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
