Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 48c8c316241e1519cd2deb1dcdda00788fa629a6
https://github.com/WebKit/WebKit/commit/48c8c316241e1519cd2deb1dcdda00788fa629a6
Author: Wenson Hsieh <[email protected]>
Date: 2023-11-29 (Wed, 29 Nov 2023)
Changed paths:
A
LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items-expected.txt
A
LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items.html
M LayoutTests/resources/ui-helper.js
M Source/WebKit/UIProcess/ios/WKContentViewInteraction.h
M Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm
M Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl
M Tools/TestRunnerShared/UIScriptContext/UIScriptController.h
M Tools/WebKitTestRunner/TestController.h
M Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm
M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h
M Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm
Log Message:
-----------
[iOS] Image analysis menu items for images with machine-readable codes
sometimes fail to appear
https://bugs.webkit.org/show_bug.cgi?id=265544
rdar://118540193
Reviewed by Megan Gardner.
Our current logic for dynamically inserting image analysis menu items (such as
"Look Up" and any
menu items for machine-readable codes) works like this:
1. We kick off image analysis for machine-readable codes and visual search
when the context menu is
about to be presented. The menu is presented with a hidden placeholder menu
item, which
represents the set of dynamically-inserted image analysis items we might
insert in the future.
2. When the image analysis started by (1) is done, we use
`-updateVisibleMenuWithBlock:` on the
context menu interaction to replace the placeholder item above with the
final set of image
analysis items (only if the WebKit client didn't remove the placeholder
menu item).
However, there's a subtle race here, where image analysis may complete before
the context menu has a
visible menu. In this scenario, the call to `-updateVisibleMenuWithBlock:`
returns early without
calling the update block, and we end up missing our only chance to swap out the
placeholder item for
the actual image analysis items.
To fix this, we detect the case where our image analysis completes before the
menu is visible, and
try again in
`-contextMenuInteraction:willDisplayMenuForConfiguration:animator:`, after the
menu
is guaranteed to be visible.
Test:
fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items.html
*
LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items-expected.txt:
Added.
*
LayoutTests/fast/events/touch/ios/long-press-on-image-with-machine-readable-code-menu-items.html:
Added.
Add a new layout test, along with some additional test infrastructure to
exercise this fix.
* LayoutTests/resources/ui-helper.js:
Add a way to simulate MRC menu item results when long-pressing on an image in
layout tests, on iOS.
(window.UIHelper.contextMenuItems):
(window.UIHelper.installFakeMachineReadableCodeResultsForImageAnalysis):
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.h:
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _setUpImageAnalysis]):
(-[WKContentView _tearDownImageAnalysis]):
(-[WKContentView imageAnalysisGestureDidBegin:]):
Replace the extant boolean flag
`_waitingForDynamicImageAnalysisContextMenuActions` with a tri-state
enum type, `_dynamicImageAnalysisContextMenuState`, which allows us to
appropriately deal with the
corner case where image analysis completes before the context menu has any
visible items. See above
for more details.
(-[WKContentView
_completeImageAnalysisRequestForContextMenu:requestIdentifier:hasTextResults:]):
(-[WKContentView _insertDynamicImageAnalysisContextMenuItemsIfPossible]):
Pull the existing logic for updating visible context menu items to include the
dynamically-inserted
image analysis items into a separate helper method, so that we can invoke it
after the context menu
is done presenting.
If the update block runs successfully (i.e. the menu is visible), then we
transition directly to the
`NotWaiting` state and we're done. However, if the update block never runs when
calling into
`-updateVisibleMenuWithBlock:`, we instead transition to
`WaitingForVisibleMenu` and wait until the
menu is done presenting, before trying to perform the update again.
(-[WKContentView placeholderForDynamicallyInsertedImageAnalysisActions]):
(-[WKContentView
contextMenuInteraction:willDisplayMenuForConfiguration:animator:]):
Update the menu to include MRC menu items (and other dynamically-inserted image
analysis items) if
needed — note that we'll only perform the update here, in the case where we
failed to run the update
block at all, in
`-_completeImageAnalysisRequestForContextMenu:requestIdentifier:hasTextResults:`.
* Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
* Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:
(WTR::UIScriptController::installFakeMachineReadableCodeResultsForImageAnalysis):
* Tools/WebKitTestRunner/TestController.h:
* Tools/WebKitTestRunner/cocoa/TestControllerCocoa.mm:
(fakeMachineReadableCodeMenuForTesting):
(-[FakeMachineReadableCodeImageAnalysis allLines]):
(-[FakeMachineReadableCodeImageAnalysis hasResultsForAnalysisTypes:]):
(-[FakeMachineReadableCodeImageAnalysis mrcMenu]):
Add a helper class that represents a fake VisionKit image analysis result that
contains MRC menu
items (for simplicity, there's only one hard-coded item titled "QR code").
(swizzledProcessImageAnalysisRequest):
(WTR::TestController::installFakeMachineReadableCodeResultsForImageAnalysis):
(WTR::TestController::shouldUseFakeMachineReadableCodeResultsForImageAnalysis
const):
(WTR::TestController::cocoaResetStateToConsistentValues):
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.h:
* Tools/WebKitTestRunner/cocoa/UIScriptControllerCocoa.mm:
(WTR::UIScriptControllerCocoa::installFakeMachineReadableCodeResultsForImageAnalysis):
Canonical link: https://commits.webkit.org/271317@main
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
