[webkit-changes] [295315] branches/safari-613-branch

Mon, 06 Jun 2022 19:30:45 -0700

Title: [295315] branches/safari-613-branch
Revision
295315
Author
[email protected]
Date
2022-06-06 19:30:06 -0700 (Mon, 06 Jun 2022)

Log Message

Cherry-pick 2c80da108c11. rdar://problem/84667910

    [iOS] Keyboard obscures file upload context menu when trying to attach a file in gmail.com
    https://bugs.webkit.org/show_bug.cgi?id=241320
    rdar://84667910

    Reviewed by Tim Horton.

    When composing a message on gmail.com, if a user taps the "Attach file" button in the bottom
    toolbar, Gmail's script programmatically clicks a hidden file input and then immediately focuses the
    editable body field. The context menu presents at the interaction location near the bottom of the
    viewport; however, if no hardware keyboard is attached, the software keyboard appears right
    afterwards, obscuring the context menu completely.

    To address this, teach `WKFileUploadPanel` to reposition its context menu when the keyboard is
    shown, if it might overlap with the bounds of the input view.

    * LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt: Added.
    * LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html: Added.

    Add a layout test to verify that focusing a text field with the software keyboard while presenting a
    file picker doesn't result in the file picker's context menu being obscured behind the keyboard.

    * LayoutTests/resources/ui-helper.js:
    (window.UIHelper.dismissMenu):
    (window.UIHelper.contextMenuRect):

    Add a couple of new UIHelper methods.

    * Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
    (-[WKContentView _keyboardDidShow]):

    If the keyboard appears while the file upload panel is active, reposition the context menu if it
    overlaps with the new keyboard bounds.

    (-[WKContentView _zoomToFocusRectAfterShowingKeyboardIfNeeded]):

    Factor this existing logic out into a helper method.

    * Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h:
    * Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm:
    (-[WKFileUploadPanel contextMenuInteraction:willEndForConfiguration:animator:]):

    If the `_isRepositioningContextMenu` flag is set, avoid removing and destroying the context menu
    interaction.

    (-[WKFileUploadPanel ensureContextMenuInteraction]):

    Drive-by fix: adjust `-ensureContextMenuInteraction` so that it returns the either the existing or
    newly created context menu interaction.

    (-[WKFileUploadPanel repositionContextMenuIfNeeded]):

    Add a helper method to reposition the context menu by removing and re-presenting the context menu
    interaction without animation, only if it overlaps the input view bounds. While removing and
    presenting the context menu again, set a `_isRepositioningContextMenu` flag, such that

    (-[WKFileUploadPanel showDocumentPickerMenu]):
    * Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
    * Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:
    (WTR::UIScriptController::contextMenuRect const):
    * Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm:
    (-[TestRunnerWKWebView _dismissAllContextMenuInteractions]):

    Adjust this helper to immediately cancel all context menu interactions without animation, so that
    context menu removal delegate methods don't bleed into subsequent tests.

    * Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h:
    * Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm:
    (WTR::UIScriptControllerIOS::menuRect const):

    Use the new `toObject` helper method.

    (WTR::UIScriptControllerIOS::contextMenuRect const):

    Add support for a new script controller method to return the context menu container view's bounds
    in window coordinates. Note that I'm using window coordinates here as opposed to root view
    coordinates, since the new test that uses this method needs to compare the context menu's position
    against the input view bounds in window coordinates.

    (WTR::UIScriptControllerIOS::toObject const):

    Add a helper method to convert the given `CGRect` into a _javascript_ object reference.

    Canonical link: https://commits.webkit.org/251344@main
    git-svn-id: https://svn.webkit.org/repository/webkit/trunk@295299 268f45cc-cd09-0410-ab3c-d52691b4dbfc

Modified Paths

Added Paths

Diff

Added: branches/safari-613-branch/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt (0 => 295315)


--- branches/safari-613-branch/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt	                        (rev 0)
+++ branches/safari-613-branch/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt	2022-06-07 02:30:06 UTC (rev 295315)
@@ -0,0 +1,10 @@
+This test verifies that when programmatically clicking a file input while focusing an editable control with a software keyboard attached, the file upload context menu is presented above the software keyboard. To manually test, click the 'Show file picker' button without a hardware keyboard attached, and verify that the context menu is visible.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Context menu was presented above the keyboard
+PASS successfullyParsed is true
+
+TEST COMPLETE
+Show file picker

Added: branches/safari-613-branch/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html (0 => 295315)


--- branches/safari-613-branch/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html	                        (rev 0)
+++ branches/safari-613-branch/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html	2022-06-07 02:30:06 UTC (rev 295315)
@@ -0,0 +1,72 @@
+<!DOCTYPE html> <!-- webkit-test-runner [ useFlexibleViewport=true shouldHandleRunOpenPanel=false ] -->
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<head>
+<script src=""
+<script src=""
+<script>
+jsTestIsAsync = true;
+
+async function waitForContextMenuToBePresentedAboveInputView()
+{
+    while (true) {
+        await UIHelper.delayFor(50);
+
+        let contextMenuRect = await UIHelper.contextMenuRect();
+        let inputViewBounds = await UIHelper.inputViewBounds();
+        if (!contextMenuRect || !inputViewBounds)
+            continue;
+
+        if (contextMenuRect.top + contextMenuRect.height < inputViewBounds.top)
+            return;
+    }
+}
+
+addEventListener("load", async () => {
+    description("This test verifies that when programmatically clicking a file input while focusing an editable control with a software keyboard attached, the file upload context menu is presented above the software keyboard. To manually test, click the 'Show file picker' button without a hardware keyboard attached, and verify that the context menu is visible.");
+
+    let fileInput = document.querySelector("input[type=file]");
+    let textInput = document.querySelector("input[type=text]");
+    let chooseFileButton = document.querySelector("button");
+    chooseFileButton.addEventListener("click", async () => {
+        fileInput.click();
+        textInput.focus();
+    });
+
+    await UIHelper.setHardwareKeyboardAttached(false);
+    await UIHelper.activateElementAndWaitForInputSession(chooseFileButton);
+    await waitForContextMenuToBePresentedAboveInputView();
+    testPassed("Context menu was presented above the keyboard");
+    await UIHelper.dismissMenu();
+    textInput.blur();
+    await UIHelper.waitForKeyboardToHide();
+    finishJSTest();
+});
+</script>
+<style>
+input[type="text"] {
+    font-size: 16px;
+}
+
+input[type="file"] {
+    width: 0;
+    height: 0;
+    position: absolute;
+}
+
+button {
+    border: 1px solid black;
+    position: fixed;
+    bottom: 100px;
+    left: 1em;
+    width: 180px;
+    height: 44px;
+    font-size: 16px;
+}
+</style>
+</head>
+<body>
+<input type="text" /><input type="file" />
+<button>Show file picker</button>
+</body>
+</html>
\ No newline at end of file

Modified: branches/safari-613-branch/LayoutTests/resources/ui-helper.js (295314 => 295315)


--- branches/safari-613-branch/LayoutTests/resources/ui-helper.js	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/LayoutTests/resources/ui-helper.js	2022-06-07 02:30:06 UTC (rev 295315)
@@ -765,6 +765,16 @@
         });
     }
 
+    static dismissMenu()
+    {
+        if (!this.isWebKit2())
+            return Promise.resolve();
+
+        return new Promise(resolve => {
+            testRunner.runUIScript("uiController.dismissMenu()", resolve);
+        });
+    }
+
     static waitForKeyboardToHide()
     {
         if (!this.isWebKit2() || !this.isIOSFamily())
@@ -1342,6 +1352,13 @@
         });
     }
 
+    static contextMenuRect()
+    {
+        return new Promise(resolve => {
+            testRunner.runUIScript("JSON.stringify(uiController.contextMenuRect)", result => resolve(JSON.parse(result)));
+        });
+    }
+
     static setHardwareKeyboardAttached(attached)
     {
         return new Promise(resolve => testRunner.runUIScript(`uiController.setHardwareKeyboardAttached(${attached ? "true" : "false"})`, resolve));

Modified: branches/safari-613-branch/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm (295314 => 295315)


--- branches/safari-613-branch/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm	2022-06-07 02:30:06 UTC (rev 295315)
@@ -2348,6 +2348,15 @@
 
 - (void)_keyboardDidShow
 {
+    [self _zoomToFocusRectAfterShowingKeyboardIfNeeded];
+
+#if USE(UICONTEXTMENU)
+    [_fileUploadPanel repositionContextMenuIfNeeded];
+#endif
+}
+
+- (void)_zoomToFocusRectAfterShowingKeyboardIfNeeded
+{
     if (!_shouldZoomToFocusRectAfterShowingKeyboard)
         return;

Modified: branches/safari-613-branch/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h (295314 => 295315)


--- branches/safari-613-branch/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h	2022-06-07 02:30:06 UTC (rev 295315)
@@ -44,6 +44,10 @@
 - (void)presentWithParameters:(API::OpenPanelParameters*)parameters resultListener:(WebKit::WebOpenPanelResultListenerProxy*)listener;
 - (void)dismiss;
 
+#if USE(UICONTEXTMENU)
+- (void)repositionContextMenuIfNeeded;
+#endif
+
 - (NSArray<NSString *> *)currentAvailableActionTitles;
 - (NSArray<NSString *> *)acceptedTypeIdentifiers;
 @end

Modified: branches/safari-613-branch/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm (295314 => 295315)


--- branches/safari-613-branch/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm	2022-06-07 02:30:06 UTC (rev 295315)
@@ -39,6 +39,7 @@
 #import "WKData.h"
 #import "WKStringCF.h"
 #import "WKURLCF.h"
+#import "WKWebViewInternal.h"
 #import "WebIconUtilities.h"
 #import "WebOpenPanelResultListenerProxy.h"
 #import "WebPageProxy.h"
@@ -49,6 +50,7 @@
 #import <wtf/MainThread.h>
 #import <wtf/OptionSet.h>
 #import <wtf/RetainPtr.h>
+#import <wtf/SetForScope.h>
 #import <wtf/WeakObjCPtr.h>
 #import <wtf/text/StringView.h>
 
@@ -357,6 +359,7 @@
     BOOL _isPresentingSubMenu;
     ALLOW_DEPRECATED_DECLARATIONS_END
 #if USE(UICONTEXTMENU)
+    BOOL _isRepositioningContextMenu;
     RetainPtr<UIContextMenuInteraction> _documentContextMenuInteraction;
 #endif
     RetainPtr<UIDocumentPickerViewController> _documentPickerController;
@@ -688,6 +691,9 @@
 
 - (void)contextMenuInteraction:(UIContextMenuInteraction *)interaction willEndForConfiguration:(UIContextMenuConfiguration *)configuration animator:(id<UIContextMenuInteractionAnimating>)animator
 {
+    if (_isRepositioningContextMenu)
+        return;
+
     [animator addCompletion:^{
         [self removeContextMenuInteraction];
         if (!self->_isPresentingSubMenu)
@@ -704,7 +710,7 @@
     }
 }
 
-- (void)ensureContextMenuInteraction
+- (UIContextMenuInteraction *)ensureContextMenuInteraction
 {
     if (!_documentContextMenuInteraction) {
         _documentContextMenuInteraction = adoptNS([[UIContextMenuInteraction alloc] initWithDelegate:self]);
@@ -711,10 +717,37 @@
         [_view addInteraction:_documentContextMenuInteraction.get()];
         self->_isPresentingSubMenu = NO;
     }
+    return _documentContextMenuInteraction.get();
 }
 
-#endif
+- (void)repositionContextMenuIfNeeded
+{
+    if (!_documentContextMenuInteraction)
+        return;
 
+    auto *webView = [_view webView];
+    if (!webView)
+        return;
+
+    auto inputViewBoundsInWindow = webView->_inputViewBoundsInWindow;
+    if (CGRectIsEmpty(inputViewBoundsInWindow))
+        return;
+
+    // The exact bounds of the context menu container itself isn't exposed through any UIKit API or SPI,
+    // and would require traversing the view hierarchy in search of internal UIKit views. For now, just
+    // reposition the context menu if its presentation location is covered by the input view.
+    if (!CGRectContainsPoint(inputViewBoundsInWindow, [_view convertPoint:_interactionPoint toView:webView.window]))
+        return;
+
+    SetForScope repositioningContextMenuScope { _isRepositioningContextMenu, YES };
+    [UIView performWithoutAnimation:^{
+        [_documentContextMenuInteraction dismissMenu];
+        [_view presentContextMenu:_documentContextMenuInteraction.get() atLocation:_interactionPoint];
+    }];
+}
+
+#endif // USE(UICONTEXTMENU)
+
 - (void)showFilePickerMenu
 {
     NSArray *mediaTypes = [_acceptedUTIs allObjects];
@@ -733,10 +766,9 @@
 {
     // FIXME 49961589: Support picking media with UIImagePickerController
 #if HAVE(UICONTEXTMENU_LOCATION)
-    if (_allowedImagePickerTypes.containsAny({ WKFileUploadPanelImagePickerType::Image, WKFileUploadPanelImagePickerType::Video })) {
-        [self ensureContextMenuInteraction];
-        [_view presentContextMenu:_documentContextMenuInteraction.get() atLocation:_interactionPoint];
-    } else // Image and Video types are not accepted so bypass the menu and open the file picker directly.
+    if (_allowedImagePickerTypes.containsAny({ WKFileUploadPanelImagePickerType::Image, WKFileUploadPanelImagePickerType::Video }))
+        [_view presentContextMenu:self.ensureContextMenuInteraction atLocation:_interactionPoint];
+    else // Image and Video types are not accepted so bypass the menu and open the file picker directly.
 #endif
         [self showFilePickerMenu];

Modified: branches/safari-613-branch/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl (295314 => 295315)


--- branches/safari-613-branch/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl	2022-06-07 02:30:06 UTC (rev 295315)
@@ -266,6 +266,7 @@
     readonly attribute boolean isDismissingMenu;
     readonly attribute boolean isShowingMenu;
     readonly attribute object menuRect;
+    readonly attribute object contextMenuRect;
     object rectForMenuAction(DOMString action);
     undefined chooseMenuAction(DOMString action, object callback);
     undefined dismissMenu();

Modified: branches/safari-613-branch/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h (295314 => 295315)


--- branches/safari-613-branch/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h	2022-06-07 02:30:06 UTC (rev 295315)
@@ -283,6 +283,7 @@
     virtual bool isShowingMenu() const { notImplemented(); return false; }
     virtual JSObjectRef rectForMenuAction(JSStringRef) const { notImplemented(); return nullptr; }
     virtual JSObjectRef menuRect() const { notImplemented(); return nullptr; }
+    virtual JSObjectRef contextMenuRect() const { notImplemented(); return nullptr; }
     virtual bool isShowingContextMenu() const { notImplemented(); return false; }
 
     // Selection

Modified: branches/safari-613-branch/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm (295314 => 295315)


--- branches/safari-613-branch/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm	2022-06-07 02:30:06 UTC (rev 295315)
@@ -259,10 +259,13 @@
 
 - (void)_dismissAllContextMenuInteractions
 {
-#if PLATFORM(IOS)
+#if USE(UICONTEXTMENU)
     for (id <UIInteraction> interaction in self.contentView.interactions) {
-        if ([interaction isKindOfClass:UIContextMenuInteraction.class])
-            [(UIContextMenuInteraction *)interaction dismissMenu];
+        if (auto contextMenuInteraction = dynamic_objc_cast<UIContextMenuInteraction>(interaction)) {
+            [UIView performWithoutAnimation:^{
+                [contextMenuInteraction dismissMenu];
+            }];
+        }
     }
 #endif
 }

Modified: branches/safari-613-branch/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h (295314 => 295315)


--- branches/safari-613-branch/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h	2022-06-07 02:30:06 UTC (rev 295315)
@@ -30,6 +30,8 @@
 #import "UIScriptControllerCocoa.h"
 #import <wtf/BlockPtr.h>
 
+typedef struct CGRect CGRect;
+
 namespace WebCore {
 class FloatPoint;
 class FloatRect;
@@ -123,6 +125,7 @@
     void simulateRotationLikeSafari(DeviceOrientation*, JSValueRef) override;
     bool isShowingPopover() const override;
     JSObjectRef rectForMenuAction(JSStringRef) const override;
+    JSObjectRef contextMenuRect() const override;
     JSObjectRef menuRect() const override;
     bool isDismissingMenu() const override;
     void chooseMenuAction(JSStringRef, JSValueRef) override;
@@ -170,6 +173,8 @@
     WebCore::FloatRect rectForMenuAction(CFStringRef) const;
     void singleTapAtPointWithModifiers(WebCore::FloatPoint location, Vector<String>&& modifierFlags, BlockPtr<void()>&&);
 
+    JSObjectRef toObject(CGRect) const;
+
     bool isWebContentFirstResponder() const override;
 };

Modified: branches/safari-613-branch/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm (295314 => 295315)


--- branches/safari-613-branch/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm	2022-06-07 02:30:00 UTC (rev 295314)
+++ branches/safari-613-branch/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm	2022-06-07 02:30:06 UTC (rev 295315)
@@ -1105,10 +1105,24 @@
     if (!calloutBar.window)
         return nullptr;
 
-    CGRect rectInRootViewCoordinates = [calloutBar convertRect:calloutBar.bounds toView:platformContentView()];
-    return m_context->objectFromRect(WebCore::FloatRect(rectInRootViewCoordinates.origin.x, rectInRootViewCoordinates.origin.y, rectInRootViewCoordinates.size.width, rectInRootViewCoordinates.size.height));
+    return toObject([calloutBar convertRect:calloutBar.bounds toView:platformContentView()]);
 }
 
+JSObjectRef UIScriptControllerIOS::contextMenuRect() const
+{
+    auto *window = webView().window;
+    auto *contextMenuView = [findAllViewsInHierarchyOfType(window, NSClassFromString(@"_UIContextMenuView")) firstObject];
+    if (!contextMenuView)
+        return nullptr;
+
+    return toObject([contextMenuView convertRect:contextMenuView.bounds toView:nil]);
+}
+
+JSObjectRef UIScriptControllerIOS::toObject(CGRect rect) const
+{
+    return m_context->objectFromRect(WebCore::FloatRect(rect.origin.x, rect.origin.y, rect.size.width, rect.size.height));
+}
+
 bool UIScriptControllerIOS::isDismissingMenu() const
 {
     return webView().dismissingMenu;

_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to