Title: [248338] trunk
Revision
248338
Author
[email protected]
Date
2019-08-06 20:15:19 -0700 (Tue, 06 Aug 2019)

Log Message

[iPadOS] slides.google.com: tapping near cursor in a slide title focuses the speaker notes
https://bugs.webkit.org/show_bug.cgi?id=200216

Reviewed by Wenson Hsieh.

Source/WebKit:

The bug was caused by a race condition between Google slides removing inputmode="none" from the hidden
content editable and updating the focused region upon receiving a pointerup event, which happens after
the Google slides had already updated its page layout & coordinates based on new visual viewport with
the software keyboard's boudning rect taken into account.

Delay bringing up the software keyboard for a inputmode change until all touches are released.

In the future, we could consider also delaying the software keyboard to be brought in general until
touchend / pointerup events are dispatched but this is rather risky since that could affact random
other websites while Google suites is the only major site to make use of inputmode="none".

This patch also reverts r243044, which was added for Google slides, since it's no longer needed and
interferes with this patch by adding another way to bring up the software keyboard.

Note: Adjusting touchend / pointerup coordinates while the keyboard is being brought up doesn't work
because the page had already updated the layout by then based on new visual viewport size.

Test: fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html

* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::handleTouchEventSynchronously): Call didReleaseAllTouchPoints when all
touches are released.
(WebKit::WebPageProxy::handleTouchEventAsynchronously): Ditto.
(WebKit::WebPageProxy::handleTouchEvent): Ditto.
* UIProcess/WebPageProxy.h:
(WebKit::WebPageProxy::didReleaseAllTouchPoints): Added for non-iOS platforms.
(WebKit::WebPageProxy::m_pendingInputModeChange): Added. Used when inputmode is changed while
there is an on-going touch interaction.
* UIProcess/ios/WebPageProxyIOS.mm:
(WebKit::WebPageProxy::elementDidFocus): Clear m_pendingInputModeChange when a new element is focused.
(WebKit::WebPageProxy::elementDidBlur): Ditto for bluring.
(WebKit::WebPageProxy::focusedElementDidChangeInputMode): Don't bring up the software keyboard now if
there are on-going touches by exiting early after setting m_pendingInputModeChange.
(WebKit::WebPageProxy::didReleaseAllTouchPoints): Bring up the software keyboard if inputmode
had changed from "none" to something else.
* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::dispatchTouchEvent): Removed the code added by r243044.

LayoutTests:

Added a new regression test and removed the one added for r243044.

* fast/events/touch/ios/show-keyboard-after-preventing-touchstart-expected.txt: Removed.
* fast/events/touch/ios/show-keyboard-after-preventing-touchstart.html: Removed.
* fast/forms/ios/inputmode-change-update-keyboard-after-pointerup-expected.txt: Added.
* fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html: Added.
* fast/forms/ios/inputmode-change-update-keyboard.html: Fixed the test for manual testing.

Modified Paths

Added Paths

Removed Paths

Diff

Modified: trunk/LayoutTests/ChangeLog (248337 => 248338)


--- trunk/LayoutTests/ChangeLog	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/LayoutTests/ChangeLog	2019-08-07 03:15:19 UTC (rev 248338)
@@ -1,3 +1,18 @@
+2019-08-06  Ryosuke Niwa  <[email protected]>
+
+        [iPadOS] slides.google.com: tapping near cursor in a slide title focuses the speaker notes
+        https://bugs.webkit.org/show_bug.cgi?id=200216
+
+        Reviewed by Wenson Hsieh.
+
+        Added a new regression test and removed the one added for r243044.
+
+        * fast/events/touch/ios/show-keyboard-after-preventing-touchstart-expected.txt: Removed.
+        * fast/events/touch/ios/show-keyboard-after-preventing-touchstart.html: Removed.
+        * fast/forms/ios/inputmode-change-update-keyboard-after-pointerup-expected.txt: Added.
+        * fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html: Added.
+        * fast/forms/ios/inputmode-change-update-keyboard.html: Fixed the test for manual testing.
+
 2019-08-06  Commit Queue  <[email protected]>
 
         Unreviewed, rolling out r248289.

Deleted: trunk/LayoutTests/fast/events/touch/ios/show-keyboard-after-preventing-touchstart-expected.txt (248337 => 248338)


--- trunk/LayoutTests/fast/events/touch/ios/show-keyboard-after-preventing-touchstart-expected.txt	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/LayoutTests/fast/events/touch/ios/show-keyboard-after-preventing-touchstart-expected.txt	2019-08-07 03:15:19 UTC (rev 248338)
@@ -1,11 +0,0 @@
-
-Verifies that the keyboard shows up even after preventing default on touchstart when focusing a hidden editable area. To manually test, tap the red box; the textarea should remain focused, and the keyboard should appear.
-
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-PASS keyboard was shown.
-PASS document.activeElement is textarea
-PASS successfullyParsed is true
-
-TEST COMPLETE
-

Deleted: trunk/LayoutTests/fast/events/touch/ios/show-keyboard-after-preventing-touchstart.html (248337 => 248338)


--- trunk/LayoutTests/fast/events/touch/ios/show-keyboard-after-preventing-touchstart.html	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/LayoutTests/fast/events/touch/ios/show-keyboard-after-preventing-touchstart.html	2019-08-07 03:15:19 UTC (rev 248338)
@@ -1,59 +0,0 @@
-<!DOCTYPE html>
-<html>
-<head>
-<script src=""
-<script src=""
-<style>
-html, body {
-    width: 100%;
-    height: 100%;
-    margin: 0;
-}
-
-textarea, #target {
-    width: 100%;
-    height: 100px;
-    position: absolute;
-    top: 0;
-}
-
-#target {
-    background-color: tomato;
-    z-index: 1;
-    opacity: 0.25;
-}
-
-textarea {
-    opacity: 0;
-}
-
-#description {
-    margin-top: 100px;
-}
-</style>
-</head>
-<body>
-    <textarea></textarea>
-    <div id="target"></div>
-    <pre id="description"></pre>
-    <pre id="console"></pre>
-</body>
-<script>
-    jsTestIsAsync = true;
-    textarea = document.querySelector("textarea");
-    target = document.getElementById("target");
-    target.addEventListener("touchstart", event => event.preventDefault());
-
-    description("Verifies that the keyboard shows up even after preventing default on touchstart when focusing a hidden editable area. To manually test, tap the red box; the textarea should remain focused, and the keyboard should appear.");
-
-    addEventListener("load", async () => {
-        textarea.focus();
-        await UIHelper.activateElementAndWaitForInputSession(target);
-        testPassed("keyboard was shown.");
-        shouldBe("document.activeElement", "textarea");
-        textarea.blur();
-        await UIHelper.waitForKeyboardToHide();
-        finishJSTest();
-    });
-</script>
-</html>

Added: trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard-after-pointerup-expected.txt (0 => 248338)


--- trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard-after-pointerup-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard-after-pointerup-expected.txt	2019-08-07 03:15:19 UTC (rev 248338)
@@ -0,0 +1,17 @@
+This tests updating inputmode of an input element from "none" to "text". The software keyboard should be brought up after pointerup.
+To manually test, tap on the input element below. The software keyboard should be brought up
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS textHeight = keyboardRect.height; keyboardRect.height > 0 is true
+PASS logs.length is 2
+PASS logs[0].event.type is "pointerdown"
+PASS logs[0].visualViewportHeight is originalVisualViewportHeight
+PASS logs[1].event.type is "pointerup"
+PASS logs[1].visualViewportHeight is originalVisualViewportHeight
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+

Added: trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html (0 => 248338)


--- trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html	                        (rev 0)
+++ trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html	2019-08-07 03:15:19 UTC (rev 248338)
@@ -0,0 +1,60 @@
+<!DOCTYPE html>
+<html>
+<head>
+<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
+<script src=""
+<script src=""
+</head>
+<body>
+<input inputmode="none">
+<div id="countdown"></div>
+<script>
+jsTestIsAsync = true;
+
+description('This tests updating inputmode of an input element from "none" to "text". The software keyboard should be brought up after pointerup.<br>'
+    + 'To manually test, tap on the input element below. The software keyboard should be brought up');
+
+const input = document.querySelector("input");
+input.focus();
+
+let originalVisualViewportHeight;
+let logs = [];
+async function runTest() {
+    if (window.testRunner)
+        await UIHelper.setHardwareKeyboardAttached(false);
+
+    originalVisualViewportHeight = window.visualViewport.height;
+
+    let didResize = () => { };
+    window.visualViewport.addEventListener('resize', () => didResize());
+    input.addEventListener('pointerdown', (event) => {
+        input.inputMode = 'text';
+        logs.push({event, visualViewportHeight: visualViewport.height});
+    }, {once: true});
+    input.addEventListener('pointerup', (event) => {
+        logs.push({event, visualViewportHeight: visualViewport.height});
+    }, {once: true});
+
+    if (window.testRunner) {
+        await UIHelper.activateFormControl(input);
+        window.keyboardRect = await UIHelper.inputViewBounds();
+        shouldBeTrue('textHeight = keyboardRect.height; keyboardRect.height > 0');
+    } else {
+        await new Promise((resolve) => { didResize = resolve; });
+        shouldBeTrue('textHeight = document.documentElement.clientHeight - visualViewport.height; document.documentElement.clientHeight - visualViewport.height < 100');
+    }
+
+    shouldBe('logs.length', '2');
+    shouldBeEqualToString('logs[0].event.type', 'pointerdown');
+    shouldBe('logs[0].visualViewportHeight', 'originalVisualViewportHeight');
+    shouldBeEqualToString('logs[1].event.type', 'pointerup');
+    shouldBe('logs[1].visualViewportHeight', 'originalVisualViewportHeight');
+
+    finishJSTest();
+}
+
+window._onload_ = () => setTimeout(runTest, 0);
+
+</script>
+</body>
+</html>

Modified: trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard.html (248337 => 248338)


--- trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard.html	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/LayoutTests/fast/forms/ios/inputmode-change-update-keyboard.html	2019-08-07 03:15:19 UTC (rev 248338)
@@ -30,7 +30,8 @@
 }
 
 async function runTest() {
-    await UIHelper.setHardwareKeyboardAttached(false);
+    if (window.testRunner)
+        await UIHelper.setHardwareKeyboardAttached(false);
 
     debug('inputmode="text"');
 

Modified: trunk/Source/WebKit/ChangeLog (248337 => 248338)


--- trunk/Source/WebKit/ChangeLog	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/Source/WebKit/ChangeLog	2019-08-07 03:15:19 UTC (rev 248338)
@@ -1,3 +1,48 @@
+2019-08-06  Ryosuke Niwa  <[email protected]>
+
+        [iPadOS] slides.google.com: tapping near cursor in a slide title focuses the speaker notes
+        https://bugs.webkit.org/show_bug.cgi?id=200216
+
+        Reviewed by Wenson Hsieh.
+
+        The bug was caused by a race condition between Google slides removing inputmode="none" from the hidden
+        content editable and updating the focused region upon receiving a pointerup event, which happens after
+        the Google slides had already updated its page layout & coordinates based on new visual viewport with
+        the software keyboard's boudning rect taken into account.
+
+        Delay bringing up the software keyboard for a inputmode change until all touches are released.
+
+        In the future, we could consider also delaying the software keyboard to be brought in general until
+        touchend / pointerup events are dispatched but this is rather risky since that could affact random
+        other websites while Google suites is the only major site to make use of inputmode="none".
+
+        This patch also reverts r243044, which was added for Google slides, since it's no longer needed and
+        interferes with this patch by adding another way to bring up the software keyboard.
+
+        Note: Adjusting touchend / pointerup coordinates while the keyboard is being brought up doesn't work
+        because the page had already updated the layout by then based on new visual viewport size.
+
+        Test: fast/forms/ios/inputmode-change-update-keyboard-after-pointerup.html
+
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::handleTouchEventSynchronously): Call didReleaseAllTouchPoints when all
+        touches are released.
+        (WebKit::WebPageProxy::handleTouchEventAsynchronously): Ditto.
+        (WebKit::WebPageProxy::handleTouchEvent): Ditto.
+        * UIProcess/WebPageProxy.h:
+        (WebKit::WebPageProxy::didReleaseAllTouchPoints): Added for non-iOS platforms.
+        (WebKit::WebPageProxy::m_pendingInputModeChange): Added. Used when inputmode is changed while
+        there is an on-going touch interaction.
+        * UIProcess/ios/WebPageProxyIOS.mm:
+        (WebKit::WebPageProxy::elementDidFocus): Clear m_pendingInputModeChange when a new element is focused.
+        (WebKit::WebPageProxy::elementDidBlur): Ditto for bluring.
+        (WebKit::WebPageProxy::focusedElementDidChangeInputMode): Don't bring up the software keyboard now if
+        there are on-going touches by exiting early after setting m_pendingInputModeChange.
+        (WebKit::WebPageProxy::didReleaseAllTouchPoints): Bring up the software keyboard if inputmode
+        had changed from "none" to something else.
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::dispatchTouchEvent): Removed the code added by r243044.
+
 2019-08-06  Fujii Hironori  <[email protected]>
 
         ASSERTION FAILED: m_observers.isEmpty() if WKPageSetPageStateClient is used

Modified: trunk/Source/WebKit/UIProcess/WebPageProxy.cpp (248337 => 248338)


--- trunk/Source/WebKit/UIProcess/WebPageProxy.cpp	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/Source/WebKit/UIProcess/WebPageProxy.cpp	2019-08-07 03:15:19 UTC (rev 248338)
@@ -2688,8 +2688,10 @@
     pageClient().doneWithTouchEvent(event, handled);
     m_process->responsivenessTimer().stop();
 
-    if (event.allTouchPointsAreReleased())
+    if (event.allTouchPointsAreReleased()) {
         m_touchAndPointerEventTracking.reset();
+        didReleaseAllTouchPoints();
+    }
 }
 
 void WebPageProxy::resetPotentialTapSecurityOrigin()
@@ -2711,8 +2713,10 @@
 
     m_process->send(Messages::EventDispatcher::TouchEvent(m_pageID, event), 0);
 
-    if (event.allTouchPointsAreReleased())
+    if (event.allTouchPointsAreReleased()) {
         m_touchAndPointerEventTracking.reset();
+        didReleaseAllTouchPoints();
+    }
 }
 
 #elif ENABLE(TOUCH_EVENTS)
@@ -2745,8 +2749,10 @@
         }
     }
 
-    if (event.allTouchPointsAreReleased())
+    if (event.allTouchPointsAreReleased()) {
         m_touchAndPointerEventTracking.reset();
+        didReleaseAllTouchPoints();
+    }
 }
 #endif // ENABLE(TOUCH_EVENTS)
 

Modified: trunk/Source/WebKit/UIProcess/WebPageProxy.h (248337 => 248338)


--- trunk/Source/WebKit/UIProcess/WebPageProxy.h	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/Source/WebKit/UIProcess/WebPageProxy.h	2019-08-07 03:15:19 UTC (rev 248338)
@@ -1946,6 +1946,7 @@
     void elementDidFocus(const FocusedElementInformation&, bool userIsInteracting, bool blurPreviousNode, OptionSet<WebCore::ActivityState::Flag> activityStateChanges, const UserData&);
     void elementDidBlur();
     void focusedElementDidChangeInputMode(WebCore::InputMode);
+    void didReleaseAllTouchPoints();
     void didReceiveEditorStateUpdateAfterFocus();
 
     void showInspectorHighlight(const WebCore::Highlight&);
@@ -1954,6 +1955,8 @@
     void enableInspectorNodeSearch();
     void disableInspectorNodeSearch();
     void focusedElementInformationCallback(const FocusedElementInformation&, CallbackID);
+#else
+    void didReleaseAllTouchPoints() { }
 #endif // PLATFORM(IOS_FAMILY)
 
 #if ENABLE(DATA_DETECTION)
@@ -2174,6 +2177,7 @@
 #endif
 
 #if PLATFORM(IOS_FAMILY)
+    Optional<WebCore::InputMode> m_pendingInputModeChange;
     Optional<WebCore::ViewportArguments> m_overrideViewportArguments;
     VisibleContentRectUpdateInfo m_lastVisibleContentRectUpdate;
     uint64_t m_firstLayerTreeTransactionIdAfterDidCommitLoad { 0 };

Modified: trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm (248337 => 248338)


--- trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/Source/WebKit/UIProcess/ios/WebPageProxyIOS.mm	2019-08-07 03:15:19 UTC (rev 248338)
@@ -905,6 +905,7 @@
 
 void WebPageProxy::elementDidFocus(const FocusedElementInformation& information, bool userIsInteracting, bool blurPreviousNode, OptionSet<WebCore::ActivityState::Flag> activityStateChanges, const UserData& userData)
 {
+    m_pendingInputModeChange = WTF::nullopt;
     m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement = true;
 
     API::Object* userDataObject = process().transformHandlesToObjects(userData.object()).get();
@@ -919,6 +920,7 @@
 
 void WebPageProxy::elementDidBlur()
 {
+    m_pendingInputModeChange = WTF::nullopt;
     m_waitingForPostLayoutEditorStateUpdateAfterFocusingElement = false;
     m_deferredElementDidFocusArguments = nullptr;
     pageClient().elementDidBlur();
@@ -926,9 +928,25 @@
 
 void WebPageProxy::focusedElementDidChangeInputMode(WebCore::InputMode mode)
 {
+#if ENABLE(TOUCH_EVENTS)
+    if (m_touchAndPointerEventTracking.isTrackingAnything()) {
+        m_pendingInputModeChange = mode;
+        return;
+    }
+#endif
+
     pageClient().focusedElementDidChangeInputMode(mode);
 }
 
+void WebPageProxy::didReleaseAllTouchPoints()
+{
+    if (!m_pendingInputModeChange)
+        return;
+
+    pageClient().focusedElementDidChangeInputMode(*m_pendingInputModeChange);
+    m_pendingInputModeChange = WTF::nullopt;
+}
+
 void WebPageProxy::autofillLoginCredentials(const String& username, const String& password)
 {
     m_process->send(Messages::WebPage::AutofillLoginCredentials(username, password), m_pageID);

Modified: trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp (248337 => 248338)


--- trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2019-08-07 02:48:07 UTC (rev 248337)
+++ trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2019-08-07 03:15:19 UTC (rev 248338)
@@ -2897,21 +2897,10 @@
 void WebPage::dispatchTouchEvent(const WebTouchEvent& touchEvent, bool& handled)
 {
     SetForScope<bool> userIsInteractingChange { m_userIsInteracting, true };
-
-    auto oldFocusedFrame = makeRefPtr(m_page->focusController().focusedFrame());
-    auto oldFocusedElement = makeRefPtr(oldFocusedFrame ? oldFocusedFrame->document()->focusedElement() : nullptr);
-
     m_lastInteractionLocation = touchEvent.position();
     CurrentEvent currentEvent(touchEvent);
     handled = handleTouchEvent(touchEvent, m_page.get());
     updatePotentialTapSecurityOrigin(touchEvent, handled);
-
-    if (handled && oldFocusedElement) {
-        auto newFocusedFrame = makeRefPtr(m_page->focusController().focusedFrame());
-        auto newFocusedElement = makeRefPtr(newFocusedFrame ? newFocusedFrame->document()->focusedElement() : nullptr);
-        if (oldFocusedElement == newFocusedElement && isTransparentOrFullyClipped(*newFocusedElement))
-            elementDidRefocus(*newFocusedElement);
-    }
 }
 
 void WebPage::touchEventSync(const WebTouchEvent& touchEvent, CompletionHandler<void(bool)>&& reply)
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to