Title: [294088] trunk
- Revision
- 294088
- Author
- wenson_hs...@apple.com
- Date
- 2022-05-11 21:05:58 -0700 (Wed, 11 May 2022)
Log Message
ImageAnalysisQueue should extract and analyze images inside of subframes
https://bugs.webkit.org/show_bug.cgi?id=240328
rdar://93134975
Reviewed by Tim Horton.
Teach `ImageAnalysisQueue` to recursively find all images on the page (including images in of subframe content)
and queue them for analysis. To do this, we refactor `enqueueAllImages` to call into a new recursive helper
method, `enqueueAllImagesRecursive`, to look for more candidate image elements that exist inside frame owner
elements (e.g. `iframe`).
Test: ImageAnalysisTests.AnalyzeImagesInSubframes
* page/ImageAnalysisQueue.cpp:
(WebCore::ImageAnalysisQueue::enqueueAllImages):
(WebCore::ImageAnalysisQueue::enqueueAllImagesRecursive):
* page/ImageAnalysisQueue.h:
ImageAnalysisQueue should extract and analyze images inside of subframes
https://bugs.webkit.org/show_bug.cgi?id=240328
rdar://93134975
Reviewed by Tim Horton.
Add an API test to verify that we extract and analyze images inside of subframes, in addition to images in the
main frame.
* TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm:
(TestWebKitAPI::TEST):
* TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html:
Canonical link: https://commits.webkit.org/250478@main
Modified Paths
Diff
Modified: trunk/Source/WebCore/ChangeLog (294087 => 294088)
--- trunk/Source/WebCore/ChangeLog 2022-05-12 02:06:37 UTC (rev 294087)
+++ trunk/Source/WebCore/ChangeLog 2022-05-12 04:05:58 UTC (rev 294088)
@@ -1,3 +1,23 @@
+2022-05-11 Wenson Hsieh <wenson_hs...@apple.com>
+
+ ImageAnalysisQueue should extract and analyze images inside of subframes
+ https://bugs.webkit.org/show_bug.cgi?id=240328
+ rdar://93134975
+
+ Reviewed by Tim Horton.
+
+ Teach `ImageAnalysisQueue` to recursively find all images on the page (including images in of subframe content)
+ and queue them for analysis. To do this, we refactor `enqueueAllImages` to call into a new recursive helper
+ method, `enqueueAllImagesRecursive`, to look for more candidate image elements that exist inside frame owner
+ elements (e.g. `iframe`).
+
+ Test: ImageAnalysisTests.AnalyzeImagesInSubframes
+
+ * page/ImageAnalysisQueue.cpp:
+ (WebCore::ImageAnalysisQueue::enqueueAllImages):
+ (WebCore::ImageAnalysisQueue::enqueueAllImagesRecursive):
+ * page/ImageAnalysisQueue.h:
+
2022-05-11 Patrick Angle <pan...@apple.com>
Web Inspector: Parse InjectedScriptSource as a built-in to get guaranteed non-user-overriden JSC built-ins
Modified: trunk/Source/WebCore/page/ImageAnalysisQueue.cpp (294087 => 294088)
--- trunk/Source/WebCore/page/ImageAnalysisQueue.cpp 2022-05-12 02:06:37 UTC (rev 294087)
+++ trunk/Source/WebCore/page/ImageAnalysisQueue.cpp 2022-05-12 04:05:58 UTC (rev 294088)
@@ -97,8 +97,18 @@
m_identifier = identifier;
}
+ enqueueAllImagesRecursive(document);
+}
+
+void ImageAnalysisQueue::enqueueAllImagesRecursive(Document& document)
+{
for (auto& image : descendantsOfType<HTMLImageElement>(document))
enqueueIfNeeded(image);
+
+ for (auto& frameOwner : descendantsOfType<HTMLFrameOwnerElement>(document)) {
+ if (RefPtr contentDocument = frameOwner.contentDocument())
+ enqueueAllImagesRecursive(*contentDocument);
+ }
}
void ImageAnalysisQueue::resumeProcessing()
Modified: trunk/Source/WebCore/page/ImageAnalysisQueue.h (294087 => 294088)
--- trunk/Source/WebCore/page/ImageAnalysisQueue.h 2022-05-12 02:06:37 UTC (rev 294087)
+++ trunk/Source/WebCore/page/ImageAnalysisQueue.h 2022-05-12 04:05:58 UTC (rev 294088)
@@ -54,6 +54,8 @@
void resumeProcessingSoon();
void resumeProcessing();
+ void enqueueAllImagesRecursive(Document&);
+
enum class Priority : bool { Low, High };
struct Task {
WeakPtr<HTMLImageElement> element;
Modified: trunk/Tools/ChangeLog (294087 => 294088)
--- trunk/Tools/ChangeLog 2022-05-12 02:06:37 UTC (rev 294087)
+++ trunk/Tools/ChangeLog 2022-05-12 04:05:58 UTC (rev 294088)
@@ -1,3 +1,18 @@
+2022-05-11 Wenson Hsieh <wenson_hs...@apple.com>
+
+ ImageAnalysisQueue should extract and analyze images inside of subframes
+ https://bugs.webkit.org/show_bug.cgi?id=240328
+ rdar://93134975
+
+ Reviewed by Tim Horton.
+
+ Add an API test to verify that we extract and analyze images inside of subframes, in addition to images in the
+ main frame.
+
+ * TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm:
+ (TestWebKitAPI::TEST):
+ * TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html:
+
2022-05-10 Ben Nham <n...@apple.com>
Re-send connection configuration if webpushd dies
Modified: trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm (294087 => 294088)
--- trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm 2022-05-12 02:06:37 UTC (rev 294087)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/ImageAnalysisTests.mm 2022-05-12 04:05:58 UTC (rev 294088)
@@ -255,6 +255,23 @@
EXPECT_WK_STREQ(overlayText, @"Foo bar");
}
+TEST(ImageAnalysisTests, AnalyzeImagesInSubframes)
+{
+ auto requestSwizzler = makeImageAnalysisRequestSwizzler(processRequestWithResults);
+
+ auto webView = createWebViewWithTextRecognitionEnhancements();
+ [webView synchronouslyLoadTestPageNamed:@"multiple-images"];
+ __block bool doneInsertingFrame = false;
+ [webView callAsyncJavaScript:@"appendAndLoadSubframe(source)" arguments:@{ @"source" : @"multiple-images.html" } inFrame:nil inContentWorld:WKContentWorld.pageWorld completionHandler:^(id, NSError *error) {
+ EXPECT_NULL(error);
+ doneInsertingFrame = true;
+ }];
+ Util::run(&doneInsertingFrame);
+
+ [webView _startImageAnalysis:nil];
+ [webView waitForImageAnalysisRequests:10];
+}
+
TEST(ImageAnalysisTests, AnalyzeDynamicallyLoadedImages)
{
auto requestSwizzler = makeImageAnalysisRequestSwizzler(processRequestWithResults);
Modified: trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html (294087 => 294088)
--- trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html 2022-05-12 02:06:37 UTC (rev 294087)
+++ trunk/Tools/TestWebKitAPI/Tests/WebKitCocoa/multiple-images.html 2022-05-12 04:05:58 UTC (rev 294088)
@@ -27,6 +27,16 @@
document.body.appendChild(image);
}
+ async function appendAndLoadSubframe(source)
+ {
+ const frame = document.createElement("iframe");
+ frame.src = ""
+ await new Promise(resolve => {
+ frame.addEventListener("load", resolve);
+ document.body.appendChild(frame);
+ });
+ }
+
function hideAllImages()
{
Array.from(document.images).forEach(image => image.style.setProperty("display", "none"));
_______________________________________________
webkit-changes mailing list
webkit-changes@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-changes
