- Revision
- 295299
- Author
- wenson_hs...@apple.com
- Date
- 2022-06-06 15:08:33 -0700 (Mon, 06 Jun 2022)
Log Message
[iOS] Keyboard obscures file upload context menu when trying to attach a file in gmail.com
https://bugs.webkit.org/show_bug.cgi?id=241320
rdar://84667910
Reviewed by Tim Horton.
When composing a message on gmail.com, if a user taps the "Attach file" button in the bottom
toolbar, Gmail's script programmatically clicks a hidden file input and then immediately focuses the
editable body field. The context menu presents at the interaction location near the bottom of the
viewport; however, if no hardware keyboard is attached, the software keyboard appears right
afterwards, obscuring the context menu completely.
To address this, teach `WKFileUploadPanel` to reposition its context menu when the keyboard is
shown, if it might overlap with the bounds of the input view.
* LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt: Added.
* LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html: Added.
Add a layout test to verify that focusing a text field with the software keyboard while presenting a
file picker doesn't result in the file picker's context menu being obscured behind the keyboard.
* LayoutTests/resources/ui-helper.js:
(window.UIHelper.dismissMenu):
(window.UIHelper.contextMenuRect):
Add a couple of new UIHelper methods.
* Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView _keyboardDidShow]):
If the keyboard appears while the file upload panel is active, reposition the context menu if it
overlaps with the new keyboard bounds.
(-[WKContentView _zoomToFocusRectAfterShowingKeyboardIfNeeded]):
Factor this existing logic out into a helper method.
* Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h:
* Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm:
(-[WKFileUploadPanel contextMenuInteraction:willEndForConfiguration:animator:]):
If the `_isRepositioningContextMenu` flag is set, avoid removing and destroying the context menu
interaction.
(-[WKFileUploadPanel ensureContextMenuInteraction]):
Drive-by fix: adjust `-ensureContextMenuInteraction` so that it returns the either the existing or
newly created context menu interaction.
(-[WKFileUploadPanel repositionContextMenuIfNeeded]):
Add a helper method to reposition the context menu by removing and re-presenting the context menu
interaction without animation, only if it overlaps the input view bounds. While removing and
presenting the context menu again, set a `_isRepositioningContextMenu` flag, such that
(-[WKFileUploadPanel showDocumentPickerMenu]):
* Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl:
* Tools/TestRunnerShared/UIScriptContext/UIScriptController.h:
(WTR::UIScriptController::contextMenuRect const):
* Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm:
(-[TestRunnerWKWebView _dismissAllContextMenuInteractions]):
Adjust this helper to immediately cancel all context menu interactions without animation, so that
context menu removal delegate methods don't bleed into subsequent tests.
* Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h:
* Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm:
(WTR::UIScriptControllerIOS::menuRect const):
Use the new `toObject` helper method.
(WTR::UIScriptControllerIOS::contextMenuRect const):
Add support for a new script controller method to return the context menu container view's bounds
in window coordinates. Note that I'm using window coordinates here as opposed to root view
coordinates, since the new test that uses this method needs to compare the context menu's position
against the input view bounds in window coordinates.
(WTR::UIScriptControllerIOS::toObject const):
Add a helper method to convert the given `CGRect` into a _javascript_ object reference.
Canonical link: https://commits.webkit.org/251344@main
Modified Paths
Added Paths
Diff
Added: trunk/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt (0 => 295299)
--- trunk/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt (rev 0)
+++ trunk/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard-expected.txt 2022-06-06 22:08:33 UTC (rev 295299)
@@ -0,0 +1,10 @@
+This test verifies that when programmatically clicking a file input while focusing an editable control with a software keyboard attached, the file upload context menu is presented above the software keyboard. To manually test, click the 'Show file picker' button without a hardware keyboard attached, and verify that the context menu is visible.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS Context menu was presented above the keyboard
+PASS successfullyParsed is true
+
+TEST COMPLETE
+Show file picker
Added: trunk/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html (0 => 295299)
--- trunk/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html (rev 0)
+++ trunk/LayoutTests/fast/forms/ios/show-file-upload-context-menu-above-keyboard.html 2022-06-06 22:08:33 UTC (rev 295299)
@@ -0,0 +1,72 @@
+<!DOCTYPE html> <!-- webkit-test-runner [ useFlexibleViewport=true shouldHandleRunOpenPanel=false ] -->
+<html>
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<head>
+<script src=""
+<script src=""
+<script>
+jsTestIsAsync = true;
+
+async function waitForContextMenuToBePresentedAboveInputView()
+{
+ while (true) {
+ await UIHelper.delayFor(50);
+
+ let contextMenuRect = await UIHelper.contextMenuRect();
+ let inputViewBounds = await UIHelper.inputViewBounds();
+ if (!contextMenuRect || !inputViewBounds)
+ continue;
+
+ if (contextMenuRect.top + contextMenuRect.height < inputViewBounds.top)
+ return;
+ }
+}
+
+addEventListener("load", async () => {
+ description("This test verifies that when programmatically clicking a file input while focusing an editable control with a software keyboard attached, the file upload context menu is presented above the software keyboard. To manually test, click the 'Show file picker' button without a hardware keyboard attached, and verify that the context menu is visible.");
+
+ let fileInput = document.querySelector("input[type=file]");
+ let textInput = document.querySelector("input[type=text]");
+ let chooseFileButton = document.querySelector("button");
+ chooseFileButton.addEventListener("click", async () => {
+ fileInput.click();
+ textInput.focus();
+ });
+
+ await UIHelper.setHardwareKeyboardAttached(false);
+ await UIHelper.activateElementAndWaitForInputSession(chooseFileButton);
+ await waitForContextMenuToBePresentedAboveInputView();
+ testPassed("Context menu was presented above the keyboard");
+ await UIHelper.dismissMenu();
+ textInput.blur();
+ await UIHelper.waitForKeyboardToHide();
+ finishJSTest();
+});
+</script>
+<style>
+input[type="text"] {
+ font-size: 16px;
+}
+
+input[type="file"] {
+ width: 0;
+ height: 0;
+ position: absolute;
+}
+
+button {
+ border: 1px solid black;
+ position: fixed;
+ bottom: 100px;
+ left: 1em;
+ width: 180px;
+ height: 44px;
+ font-size: 16px;
+}
+</style>
+</head>
+<body>
+<input type="text" /><input type="file" />
+<button>Show file picker</button>
+</body>
+</html>
\ No newline at end of file
Modified: trunk/LayoutTests/resources/ui-helper.js (295298 => 295299)
--- trunk/LayoutTests/resources/ui-helper.js 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/LayoutTests/resources/ui-helper.js 2022-06-06 22:08:33 UTC (rev 295299)
@@ -778,6 +778,16 @@
});
}
+ static dismissMenu()
+ {
+ if (!this.isWebKit2())
+ return Promise.resolve();
+
+ return new Promise(resolve => {
+ testRunner.runUIScript("uiController.dismissMenu()", resolve);
+ });
+ }
+
static waitForKeyboardToHide()
{
if (!this.isWebKit2() || !this.isIOSFamily())
@@ -1355,6 +1365,13 @@
});
}
+ static contextMenuRect()
+ {
+ return new Promise(resolve => {
+ testRunner.runUIScript("JSON.stringify(uiController.contextMenuRect)", result => resolve(JSON.parse(result)));
+ });
+ }
+
static setHardwareKeyboardAttached(attached)
{
return new Promise(resolve => testRunner.runUIScript(`uiController.setHardwareKeyboardAttached(${attached ? "true" : "false"})`, resolve));
Modified: trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm (295298 => 295299)
--- trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm 2022-06-06 22:08:33 UTC (rev 295299)
@@ -2377,6 +2377,15 @@
- (void)_keyboardDidShow
{
+ [self _zoomToFocusRectAfterShowingKeyboardIfNeeded];
+
+#if USE(UICONTEXTMENU)
+ [_fileUploadPanel repositionContextMenuIfNeeded];
+#endif
+}
+
+- (void)_zoomToFocusRectAfterShowingKeyboardIfNeeded
+{
if (!_shouldZoomToFocusRectAfterShowingKeyboard)
return;
Modified: trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h (295298 => 295299)
--- trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.h 2022-06-06 22:08:33 UTC (rev 295299)
@@ -44,6 +44,10 @@
- (void)presentWithParameters:(API::OpenPanelParameters*)parameters resultListener:(WebKit::WebOpenPanelResultListenerProxy*)listener;
- (void)dismiss;
+#if USE(UICONTEXTMENU)
+- (void)repositionContextMenuIfNeeded;
+#endif
+
- (NSArray<NSString *> *)currentAvailableActionTitles;
- (NSArray<NSString *> *)acceptedTypeIdentifiers;
@end
Modified: trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm (295298 => 295299)
--- trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Source/WebKit/UIProcess/ios/forms/WKFileUploadPanel.mm 2022-06-06 22:08:33 UTC (rev 295299)
@@ -39,6 +39,7 @@
#import "WKData.h"
#import "WKStringCF.h"
#import "WKURLCF.h"
+#import "WKWebViewInternal.h"
#import "WebIconUtilities.h"
#import "WebOpenPanelResultListenerProxy.h"
#import "WebPageProxy.h"
@@ -49,6 +50,7 @@
#import <wtf/MainThread.h>
#import <wtf/OptionSet.h>
#import <wtf/RetainPtr.h>
+#import <wtf/SetForScope.h>
#import <wtf/WeakObjCPtr.h>
#import <wtf/text/StringView.h>
@@ -356,6 +358,7 @@
BOOL _isPresentingSubMenu;
ALLOW_DEPRECATED_DECLARATIONS_END
#if USE(UICONTEXTMENU)
+ BOOL _isRepositioningContextMenu;
RetainPtr<UIContextMenuInteraction> _documentContextMenuInteraction;
#endif
RetainPtr<UIDocumentPickerViewController> _documentPickerController;
@@ -682,6 +685,9 @@
- (void)contextMenuInteraction:(UIContextMenuInteraction *)interaction willEndForConfiguration:(UIContextMenuConfiguration *)configuration animator:(id<UIContextMenuInteractionAnimating>)animator
{
+ if (_isRepositioningContextMenu)
+ return;
+
[animator addCompletion:^{
[self removeContextMenuInteraction];
if (!self->_isPresentingSubMenu)
@@ -698,7 +704,7 @@
}
}
-- (void)ensureContextMenuInteraction
+- (UIContextMenuInteraction *)ensureContextMenuInteraction
{
if (!_documentContextMenuInteraction) {
_documentContextMenuInteraction = adoptNS([[UIContextMenuInteraction alloc] initWithDelegate:self]);
@@ -705,10 +711,37 @@
[_view addInteraction:_documentContextMenuInteraction.get()];
self->_isPresentingSubMenu = NO;
}
+ return _documentContextMenuInteraction.get();
}
-#endif
+- (void)repositionContextMenuIfNeeded
+{
+ if (!_documentContextMenuInteraction)
+ return;
+ auto *webView = [_view webView];
+ if (!webView)
+ return;
+
+ auto inputViewBoundsInWindow = webView->_inputViewBoundsInWindow;
+ if (CGRectIsEmpty(inputViewBoundsInWindow))
+ return;
+
+ // The exact bounds of the context menu container itself isn't exposed through any UIKit API or SPI,
+ // and would require traversing the view hierarchy in search of internal UIKit views. For now, just
+ // reposition the context menu if its presentation location is covered by the input view.
+ if (!CGRectContainsPoint(inputViewBoundsInWindow, [_view convertPoint:_interactionPoint toView:webView.window]))
+ return;
+
+ SetForScope repositioningContextMenuScope { _isRepositioningContextMenu, YES };
+ [UIView performWithoutAnimation:^{
+ [_documentContextMenuInteraction dismissMenu];
+ [_view presentContextMenu:_documentContextMenuInteraction.get() atLocation:_interactionPoint];
+ }];
+}
+
+#endif // USE(UICONTEXTMENU)
+
- (void)showFilePickerMenu
{
NSArray *mediaTypes = [_acceptedUTIs allObjects];
@@ -727,10 +760,9 @@
{
// FIXME 49961589: Support picking media with UIImagePickerController
#if HAVE(UICONTEXTMENU_LOCATION)
- if (_allowedImagePickerTypes.containsAny({ WKFileUploadPanelImagePickerType::Image, WKFileUploadPanelImagePickerType::Video })) {
- [self ensureContextMenuInteraction];
- [_view presentContextMenu:_documentContextMenuInteraction.get() atLocation:_interactionPoint];
- } else // Image and Video types are not accepted so bypass the menu and open the file picker directly.
+ if (_allowedImagePickerTypes.containsAny({ WKFileUploadPanelImagePickerType::Image, WKFileUploadPanelImagePickerType::Video }))
+ [_view presentContextMenu:self.ensureContextMenuInteraction atLocation:_interactionPoint];
+ else // Image and Video types are not accepted so bypass the menu and open the file picker directly.
#endif
[self showFilePickerMenu];
Modified: trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl (295298 => 295299)
--- trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Tools/TestRunnerShared/UIScriptContext/Bindings/UIScriptController.idl 2022-06-06 22:08:33 UTC (rev 295299)
@@ -266,6 +266,7 @@
readonly attribute boolean isDismissingMenu;
readonly attribute boolean isShowingMenu;
readonly attribute object menuRect;
+ readonly attribute object contextMenuRect;
object rectForMenuAction(DOMString action);
undefined chooseMenuAction(DOMString action, object callback);
undefined dismissMenu();
Modified: trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h (295298 => 295299)
--- trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Tools/TestRunnerShared/UIScriptContext/UIScriptController.h 2022-06-06 22:08:33 UTC (rev 295299)
@@ -285,6 +285,7 @@
virtual bool isShowingMenu() const { notImplemented(); return false; }
virtual JSObjectRef rectForMenuAction(JSStringRef) const { notImplemented(); return nullptr; }
virtual JSObjectRef menuRect() const { notImplemented(); return nullptr; }
+ virtual JSObjectRef contextMenuRect() const { notImplemented(); return nullptr; }
virtual bool isShowingContextMenu() const { notImplemented(); return false; }
// Selection
Modified: trunk/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm (295298 => 295299)
--- trunk/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Tools/WebKitTestRunner/cocoa/TestRunnerWKWebView.mm 2022-06-06 22:08:33 UTC (rev 295299)
@@ -261,10 +261,13 @@
- (void)_dismissAllContextMenuInteractions
{
-#if PLATFORM(IOS)
+#if USE(UICONTEXTMENU)
for (id <UIInteraction> interaction in self.contentView.interactions) {
- if ([interaction isKindOfClass:UIContextMenuInteraction.class])
- [(UIContextMenuInteraction *)interaction dismissMenu];
+ if (auto contextMenuInteraction = dynamic_objc_cast<UIContextMenuInteraction>(interaction)) {
+ [UIView performWithoutAnimation:^{
+ [contextMenuInteraction dismissMenu];
+ }];
+ }
}
#endif
}
Modified: trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h (295298 => 295299)
--- trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.h 2022-06-06 22:08:33 UTC (rev 295299)
@@ -30,6 +30,8 @@
#import "UIScriptControllerCocoa.h"
#import <wtf/BlockPtr.h>
+typedef struct CGRect CGRect;
+
namespace WebCore {
class FloatPoint;
class FloatRect;
@@ -123,6 +125,7 @@
void simulateRotationLikeSafari(DeviceOrientation*, JSValueRef) override;
bool isShowingPopover() const override;
JSObjectRef rectForMenuAction(JSStringRef) const override;
+ JSObjectRef contextMenuRect() const override;
JSObjectRef menuRect() const override;
bool isDismissingMenu() const override;
void chooseMenuAction(JSStringRef, JSValueRef) override;
@@ -170,6 +173,8 @@
WebCore::FloatRect rectForMenuAction(CFStringRef) const;
void singleTapAtPointWithModifiers(WebCore::FloatPoint location, Vector<String>&& modifierFlags, BlockPtr<void()>&&);
+ JSObjectRef toObject(CGRect) const;
+
bool isWebContentFirstResponder() const override;
};
Modified: trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm (295298 => 295299)
--- trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm 2022-06-06 22:04:29 UTC (rev 295298)
+++ trunk/Tools/WebKitTestRunner/ios/UIScriptControllerIOS.mm 2022-06-06 22:08:33 UTC (rev 295299)
@@ -1105,10 +1105,24 @@
if (!calloutBar.window)
return nullptr;
- CGRect rectInRootViewCoordinates = [calloutBar convertRect:calloutBar.bounds toView:platformContentView()];
- return m_context->objectFromRect(WebCore::FloatRect(rectInRootViewCoordinates.origin.x, rectInRootViewCoordinates.origin.y, rectInRootViewCoordinates.size.width, rectInRootViewCoordinates.size.height));
+ return toObject([calloutBar convertRect:calloutBar.bounds toView:platformContentView()]);
}
+JSObjectRef UIScriptControllerIOS::contextMenuRect() const
+{
+ auto *window = webView().window;
+ auto *contextMenuView = [findAllViewsInHierarchyOfType(window, NSClassFromString(@"_UIContextMenuView")) firstObject];
+ if (!contextMenuView)
+ return nullptr;
+
+ return toObject([contextMenuView convertRect:contextMenuView.bounds toView:nil]);
+}
+
+JSObjectRef UIScriptControllerIOS::toObject(CGRect rect) const
+{
+ return m_context->objectFromRect(WebCore::FloatRect(rect.origin.x, rect.origin.y, rect.size.width, rect.size.height));
+}
+
bool UIScriptControllerIOS::isDismissingMenu() const
{
return webView().dismissingMenu;
