Title: [239881] trunk
Revision
239881
Author
[email protected]
Date
2019-01-11 16:21:49 -0800 (Fri, 11 Jan 2019)

Log Message

[iOS] Precision drop state thrashes when dragging near the top edge of an editable element
https://bugs.webkit.org/show_bug.cgi?id=193364
<rdar://problem/47214117>

Reviewed by Tim Horton.

Source/WebCore:

Add a new helper method on DragCaretController to compute the bounds of the editable element around the drop
caret position. This is either the enclosing form control (in the case of text fields and text areas), or the
highest editable root. See WebKit ChangeLog for more details.

Test: DragAndDropTests.AvoidPreciseDropNearTopOfTextArea

* editing/FrameSelection.cpp:
(WebCore::DragCaretController::editableElementRectInRootViewCoordinates const):
* editing/FrameSelection.h:

Source/WebKit:

On iOS, marking a UIDropProposal as precise offsets the hit-testing location of the drop by a small distance
either upwards or downwards from the actual location of the user's finger. When dragging over an editable
element, WebKit currently marks the drop proposal as precise; however, when dragging over the top edge of an
editable element, what happens is that the hit-testing location is offset to a location outside of the editable
element, which causes us to turn off precision drop mode; subsequently, turning off precision drop mode removes
the offset, which causes us to hit-test within the editable element once again and re-enable precision mode, and
the cycle continues.

In order to mitigate this, bail out of precision drop mode when dragging near the top or bottom edges of the
highest editable root that contains the current drop caret position (or, if the drop caret is inside of a text
form control, use the form control as the editable element instead).

* UIProcess/WebPageProxy.cpp:
(WebKit::WebPageProxy::didPerformDragControllerAction):
* UIProcess/WebPageProxy.h:
(WebKit::WebPageProxy::currentDragCaretEditableElementRect const):
* UIProcess/WebPageProxy.messages.in:
* UIProcess/ios/WKContentViewInteraction.mm:
(-[WKContentView dropInteraction:sessionDidUpdate:]):

Avoid precise mode when we're less than 25pt away from the top and bottom edge of the editable element rect.
Since the drag location offset amount is a fixed offset in window coordinates, we first convert this minimum
distance to the content view's coordinate space by dividing by the content scale factor.

* WebProcess/WebPage/WebPage.cpp:
(WebKit::WebPage::performDragControllerAction):

Tools:

Add a test to verify that dragging near the top of a textarea element does not flag the drop proposal as
precise, whereas dragging near the middle of the textarea does.

* TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm:
(TestWebKitAPI::TEST):

Modified Paths

Diff

Modified: trunk/Source/WebCore/ChangeLog (239880 => 239881)


--- trunk/Source/WebCore/ChangeLog	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebCore/ChangeLog	2019-01-12 00:21:49 UTC (rev 239881)
@@ -1,3 +1,21 @@
+2019-01-11  Wenson Hsieh  <[email protected]>
+
+        [iOS] Precision drop state thrashes when dragging near the top edge of an editable element
+        https://bugs.webkit.org/show_bug.cgi?id=193364
+        <rdar://problem/47214117>
+
+        Reviewed by Tim Horton.
+
+        Add a new helper method on DragCaretController to compute the bounds of the editable element around the drop
+        caret position. This is either the enclosing form control (in the case of text fields and text areas), or the
+        highest editable root. See WebKit ChangeLog for more details.
+
+        Test: DragAndDropTests.AvoidPreciseDropNearTopOfTextArea
+
+        * editing/FrameSelection.cpp:
+        (WebCore::DragCaretController::editableElementRectInRootViewCoordinates const):
+        * editing/FrameSelection.h:
+
 2019-01-11  Tim Horton  <[email protected]>
 
         REGRESSION (PSON): Firefox app lacks Open in New Tab in menu

Modified: trunk/Source/WebCore/editing/FrameSelection.cpp (239880 => 239881)


--- trunk/Source/WebCore/editing/FrameSelection.cpp	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebCore/editing/FrameSelection.cpp	2019-01-12 00:21:49 UTC (rev 239881)
@@ -114,6 +114,30 @@
     return { };
 }
 
+IntRect DragCaretController::editableElementRectInRootViewCoordinates() const
+{
+    if (!hasCaret())
+        return { };
+
+    RefPtr<ContainerNode> editableContainer;
+    if (auto* formControl = enclosingTextFormControl(m_position.deepEquivalent()))
+        editableContainer = formControl;
+    else
+        editableContainer = highestEditableRoot(m_position.deepEquivalent());
+
+    if (!editableContainer)
+        return { };
+
+    auto* renderer = editableContainer->renderer();
+    if (!renderer)
+        return { };
+
+    if (auto* view = editableContainer->document().view())
+        return view->contentsToRootView(renderer->absoluteBoundingBoxRect());
+
+    return { };
+}
+
 static inline bool shouldAlwaysUseDirectionalSelection(Frame* frame)
 {
     return !frame || frame->editor().behavior().shouldConsiderSelectionAsDirectional();

Modified: trunk/Source/WebCore/editing/FrameSelection.h (239880 => 239881)


--- trunk/Source/WebCore/editing/FrameSelection.h	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebCore/editing/FrameSelection.h	2019-01-12 00:21:49 UTC (rev 239881)
@@ -104,6 +104,7 @@
     void setCaretPosition(const VisiblePosition&);
     void clear() { setCaretPosition(VisiblePosition()); }
     WEBCORE_EXPORT IntRect caretRectInRootViewCoordinates() const;
+    WEBCORE_EXPORT IntRect editableElementRectInRootViewCoordinates() const;
 
     void nodeWillBeRemoved(Node&);
 

Modified: trunk/Source/WebKit/ChangeLog (239880 => 239881)


--- trunk/Source/WebKit/ChangeLog	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebKit/ChangeLog	2019-01-12 00:21:49 UTC (rev 239881)
@@ -1,3 +1,38 @@
+2019-01-11  Wenson Hsieh  <[email protected]>
+
+        [iOS] Precision drop state thrashes when dragging near the top edge of an editable element
+        https://bugs.webkit.org/show_bug.cgi?id=193364
+        <rdar://problem/47214117>
+
+        Reviewed by Tim Horton.
+
+        On iOS, marking a UIDropProposal as precise offsets the hit-testing location of the drop by a small distance
+        either upwards or downwards from the actual location of the user's finger. When dragging over an editable
+        element, WebKit currently marks the drop proposal as precise; however, when dragging over the top edge of an
+        editable element, what happens is that the hit-testing location is offset to a location outside of the editable
+        element, which causes us to turn off precision drop mode; subsequently, turning off precision drop mode removes
+        the offset, which causes us to hit-test within the editable element once again and re-enable precision mode, and
+        the cycle continues.
+
+        In order to mitigate this, bail out of precision drop mode when dragging near the top or bottom edges of the
+        highest editable root that contains the current drop caret position (or, if the drop caret is inside of a text
+        form control, use the form control as the editable element instead).
+
+        * UIProcess/WebPageProxy.cpp:
+        (WebKit::WebPageProxy::didPerformDragControllerAction):
+        * UIProcess/WebPageProxy.h:
+        (WebKit::WebPageProxy::currentDragCaretEditableElementRect const):
+        * UIProcess/WebPageProxy.messages.in:
+        * UIProcess/ios/WKContentViewInteraction.mm:
+        (-[WKContentView dropInteraction:sessionDidUpdate:]):
+
+        Avoid precise mode when we're less than 25pt away from the top and bottom edge of the editable element rect.
+        Since the drag location offset amount is a fixed offset in window coordinates, we first convert this minimum
+        distance to the content view's coordinate space by dividing by the content scale factor.
+
+        * WebProcess/WebPage/WebPage.cpp:
+        (WebKit::WebPage::performDragControllerAction):
+
 2019-01-11  Tim Horton  <[email protected]>
 
         REGRESSION (PSON): Firefox app lacks Open in New Tab in menu

Modified: trunk/Source/WebKit/UIProcess/WebPageProxy.cpp (239880 => 239881)


--- trunk/Source/WebKit/UIProcess/WebPageProxy.cpp	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebKit/UIProcess/WebPageProxy.cpp	2019-01-12 00:21:49 UTC (rev 239881)
@@ -2090,7 +2090,7 @@
 #endif
 }
 
-void WebPageProxy::didPerformDragControllerAction(uint64_t dragOperation, WebCore::DragHandlingMethod dragHandlingMethod, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, const IntRect& insertionRect)
+void WebPageProxy::didPerformDragControllerAction(uint64_t dragOperation, WebCore::DragHandlingMethod dragHandlingMethod, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, const IntRect& insertionRect, const IntRect& editableElementRect)
 {
     MESSAGE_CHECK(dragOperation <= DragOperationDelete);
 
@@ -2098,6 +2098,7 @@
     m_currentDragHandlingMethod = dragHandlingMethod;
     m_currentDragIsOverFileInput = mouseIsOverFileInput;
     m_currentDragNumberOfFilesToBeAccepted = numberOfItemsToBeAccepted;
+    m_currentDragCaretEditableElementRect = editableElementRect;
     setDragCaretRect(insertionRect);
 }
 

Modified: trunk/Source/WebKit/UIProcess/WebPageProxy.h (239880 => 239881)


--- trunk/Source/WebKit/UIProcess/WebPageProxy.h	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebKit/UIProcess/WebPageProxy.h	2019-01-12 00:21:49 UTC (rev 239881)
@@ -979,7 +979,7 @@
     void performDragOperation(WebCore::DragData&, const String& dragStorageName, SandboxExtension::Handle&&, SandboxExtension::HandleArray&&);
     void didPerformDragOperation(bool handled);
 
-    void didPerformDragControllerAction(uint64_t dragOperation, WebCore::DragHandlingMethod, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, const WebCore::IntRect& insertionRect);
+    void didPerformDragControllerAction(uint64_t dragOperation, WebCore::DragHandlingMethod, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, const WebCore::IntRect& insertionRect, const WebCore::IntRect& editableElementRect);
     void dragEnded(const WebCore::IntPoint& clientPosition, const WebCore::IntPoint& globalPosition, uint64_t operation);
     void didStartDrag();
     void dragCancelled();
@@ -1038,6 +1038,7 @@
     bool currentDragIsOverFileInput() const { return m_currentDragIsOverFileInput; }
     unsigned currentDragNumberOfFilesToBeAccepted() const { return m_currentDragNumberOfFilesToBeAccepted; }
     WebCore::IntRect currentDragCaretRect() const { return m_currentDragCaretRect; }
+    WebCore::IntRect currentDragCaretEditableElementRect() const { return m_currentDragCaretEditableElementRect; }
     void resetCurrentDragInformation();
     void didEndDragging();
 #endif
@@ -2179,6 +2180,7 @@
     bool m_currentDragIsOverFileInput { false };
     unsigned m_currentDragNumberOfFilesToBeAccepted { 0 };
     WebCore::IntRect m_currentDragCaretRect;
+    WebCore::IntRect m_currentDragCaretEditableElementRect;
 #endif
 
     PageLoadState m_pageLoadState;

Modified: trunk/Source/WebKit/UIProcess/WebPageProxy.messages.in (239880 => 239881)


--- trunk/Source/WebKit/UIProcess/WebPageProxy.messages.in	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebKit/UIProcess/WebPageProxy.messages.in	2019-01-12 00:21:49 UTC (rev 239881)
@@ -311,7 +311,7 @@
 
     # Drag and drop messages
 #if ENABLE(DRAG_SUPPORT)
-    DidPerformDragControllerAction(uint64_t dragOperation, enum:uint8_t WebCore::DragHandlingMethod dragHandlingMethod, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, WebCore::IntRect insertionRect)
+    DidPerformDragControllerAction(uint64_t dragOperation, enum:uint8_t WebCore::DragHandlingMethod dragHandlingMethod, bool mouseIsOverFileInput, unsigned numberOfItemsToBeAccepted, WebCore::IntRect insertionRect, WebCore::IntRect editableElementRect)
     DidEndDragging();
 #endif
 #if PLATFORM(COCOA) && ENABLE(DRAG_SUPPORT)

Modified: trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm (239880 => 239881)


--- trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebKit/UIProcess/ios/WKContentViewInteraction.mm	2019-01-12 00:21:49 UTC (rev 239881)
@@ -5989,7 +5989,16 @@
 
     auto proposal = adoptNS([[UIDropProposal alloc] initWithDropOperation:static_cast<UIDropOperation>(operation)]);
     auto dragHandlingMethod = _page->currentDragHandlingMethod();
-    [proposal setPrecise:dragHandlingMethod == WebCore::DragHandlingMethod::EditPlainText || dragHandlingMethod == WebCore::DragHandlingMethod::EditRichText];
+    if (dragHandlingMethod == WebCore::DragHandlingMethod::EditPlainText || dragHandlingMethod == WebCore::DragHandlingMethod::EditRichText) {
+        // When dragging near the top or bottom edges of an editable element, enabling precision drop mode may result in the drag session hit-testing outside of the editable
+        // element, causing the drag to no longer be accepted. This in turn disables precision drop mode, which causes the drag session to hit-test inside of the editable
+        // element again, which enables precision mode, thus continuing the cycle. To avoid precision mode thrashing, we forbid precision mode when dragging near the top or
+        // bottom of the editable element.
+        auto minimumDistanceFromVerticalEdgeForPreciseDrop = 25 / _webView.scrollView.zoomScale;
+        [proposal setPrecise:CGRectContainsPoint(CGRectInset(_page->currentDragCaretEditableElementRect(), 0, minimumDistanceFromVerticalEdgeForPreciseDrop), [session locationInView:self])];
+    } else
+        [proposal setPrecise:NO];
+
     if ([delegate respondsToSelector:@selector(_webView:willUpdateDropProposalToProposal:forSession:)])
         proposal = [delegate _webView:_webView willUpdateDropProposalToProposal:proposal.get() forSession:session];
 

Modified: trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp (239880 => 239881)


--- trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Source/WebKit/WebProcess/WebPage/WebPage.cpp	2019-01-12 00:21:49 UTC (rev 239881)
@@ -3671,7 +3671,7 @@
 void WebPage::performDragControllerAction(DragControllerAction action, const IntPoint& clientPosition, const IntPoint& globalPosition, uint64_t draggingSourceOperationMask, WebSelectionData&& selection, uint32_t flags)
 {
     if (!m_page) {
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(DragOperationNone, DragHandlingMethod::None, false, 0, { }));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(DragOperationNone, DragHandlingMethod::None, false, 0, { }, { }));
         return;
     }
 
@@ -3679,12 +3679,12 @@
     switch (action) {
     case DragControllerAction::Entered: {
         DragOperation resolvedDragOperation = m_page->dragController().dragEntered(dragData);
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), { }));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), { }, { }));
         return;
     }
     case DragControllerAction::Updated: {
         DragOperation resolvedDragOperation = m_page->dragController().dragEntered(dragData);
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), { }));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), { }, { }));
         return;
     }
     case DragControllerAction::Exited:
@@ -3702,7 +3702,7 @@
 void WebPage::performDragControllerAction(DragControllerAction action, const WebCore::DragData& dragData, SandboxExtension::Handle&& sandboxExtensionHandle, SandboxExtension::HandleArray&& sandboxExtensionsHandleArray)
 {
     if (!m_page) {
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(DragOperationNone, DragHandlingMethod::None, false, 0, { }));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(DragOperationNone, DragHandlingMethod::None, false, 0, { }, { }));
         return;
     }
 
@@ -3709,17 +3709,17 @@
     switch (action) {
     case DragControllerAction::Entered: {
         DragOperation resolvedDragOperation = m_page->dragController().dragEntered(dragData);
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), m_page->dragCaretController().caretRectInRootViewCoordinates()));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), m_page->dragCaretController().caretRectInRootViewCoordinates(), m_page->dragCaretController().editableElementRectInRootViewCoordinates()));
         return;
     }
     case DragControllerAction::Updated: {
         DragOperation resolvedDragOperation = m_page->dragController().dragUpdated(dragData);
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), m_page->dragCaretController().caretRectInRootViewCoordinates()));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(resolvedDragOperation, m_page->dragController().dragHandlingMethod(), m_page->dragController().mouseIsOverFileInput(), m_page->dragController().numberOfItemsToBeAccepted(), m_page->dragCaretController().caretRectInRootViewCoordinates(), m_page->dragCaretController().editableElementRectInRootViewCoordinates()));
         return;
     }
     case DragControllerAction::Exited:
         m_page->dragController().dragExited(dragData);
-        send(Messages::WebPageProxy::DidPerformDragControllerAction(DragOperationNone, DragHandlingMethod::None, false, 0, { }));
+        send(Messages::WebPageProxy::DidPerformDragControllerAction(DragOperationNone, DragHandlingMethod::None, false, 0, { }, { }));
         return;
         
     case DragControllerAction::PerformDragOperation: {

Modified: trunk/Tools/ChangeLog (239880 => 239881)


--- trunk/Tools/ChangeLog	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Tools/ChangeLog	2019-01-12 00:21:49 UTC (rev 239881)
@@ -1,3 +1,17 @@
+2019-01-11  Wenson Hsieh  <[email protected]>
+
+        [iOS] Precision drop state thrashes when dragging near the top edge of an editable element
+        https://bugs.webkit.org/show_bug.cgi?id=193364
+        <rdar://problem/47214117>
+
+        Reviewed by Tim Horton.
+
+        Add a test to verify that dragging near the top of a textarea element does not flag the drop proposal as
+        precise, whereas dragging near the middle of the textarea does.
+
+        * TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm:
+        (TestWebKitAPI::TEST):
+
 2019-01-11  Jonathan Bedard  <[email protected]>
 
         webkitpy: Support alternate simctl device list output

Modified: trunk/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm (239880 => 239881)


--- trunk/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm	2019-01-11 23:30:07 UTC (rev 239880)
+++ trunk/Tools/TestWebKitAPI/Tests/ios/DragAndDropTestsIOS.mm	2019-01-12 00:21:49 UTC (rev 239881)
@@ -326,6 +326,27 @@
     EXPECT_TRUE([simulator lastKnownDropProposal].precise);
 }
 
+TEST(DragAndDropTests, AvoidPreciseDropNearTopOfTextArea)
+{
+    auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
+    [webView synchronouslyLoadHTMLString:@"<meta name='viewport' content='width=device-width, initial-scale=1'><body style='margin: 0'><textarea style='height: 100px'></textarea></body>"];
+
+    auto simulator = adoptNS([[DragAndDropSimulator alloc] initWithWebView:webView.get()]);
+    auto firstItem = adoptNS([[NSItemProvider alloc] initWithObject:[NSURL URLWithString:@"https://webkit.org"]]);
+    [simulator setExternalItemProviders:@[ firstItem.get() ]];
+    [simulator runFrom:CGPointMake(320, 10) to:CGPointMake(20, 10)];
+
+    EXPECT_WK_STREQ("https://webkit.org/", [webView stringByEvaluatingJavaScript:@"document.querySelector('textarea').value"]);
+    EXPECT_FALSE([simulator lastKnownDropProposal].precise);
+    [webView evaluateJavaScript:@"document.querySelector('textarea').value = ''" completionHandler:nil];
+
+    auto secondItem = adoptNS([[NSItemProvider alloc] initWithObject:[NSURL URLWithString:@"https://apple.com"]]);
+    [simulator setExternalItemProviders:@[ secondItem.get() ]];
+    [simulator runFrom:CGPointMake(320, 50) to:CGPointMake(20, 50)];
+    EXPECT_WK_STREQ("https://apple.com/", [webView stringByEvaluatingJavaScript:@"document.querySelector('textarea').value"]);
+    EXPECT_TRUE([simulator lastKnownDropProposal].precise);
+}
+
 TEST(DragAndDropTests, ImageInLinkWithoutHREFToInput)
 {
     auto webView = adoptNS([[TestWKWebView alloc] initWithFrame:CGRectMake(0, 0, 320, 500)]);
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to