Title: [123889] trunk
Revision
123889
Author
[email protected]
Date
2012-07-27 10:59:36 -0700 (Fri, 27 Jul 2012)

Log Message

Improve touch adjustment for targetting small controls.
https://bugs.webkit.org/show_bug.cgi?id=92293

Patch by Kevin Ellis <[email protected]> on 2012-07-27
Reviewed by Antonio Gomes.

Source/WebCore:

Modifies the touch adjustment algorithm to better discriminate small targets without introducing
a bias towards shorter links.  The revised scoring algorihtm uses a normalized distance to
center-line score and a normalized overlap score.  The better of the two scores is used for
determining the best candidate for target adjustment.  The center-line score works well
for discriminating elements with partial overlaps, where relying solely on percent overlap
could bias towards the smaller element.  Conversely, a high percentage in overlap provides a
strong measure of confidence in a candidate target.  An additonal tie-break algorithm is
introduced to minimize the adjustment distance if two equally suitable candidates are found.

Test: touchadjustment/small-target-test.html

* page/TouchAdjustment.cpp:
(TouchAdjustment):
(WebCore::TouchAdjustment::nodeRespondsToTapGesture):
(WebCore::TouchAdjustment::hybridDistanceFunction):
(WebCore::TouchAdjustment::findNodeWithLowestDistanceMetric):
(WebCore::findBestClickableCandidate):

LayoutTests:

Adds a test for cases that were problematic for the touch adjustment algorithm.  Each example
in the test involves a small control immediately adjacent to a larger control.  When the touch
area is significantly larger than the small control, it was possible for the touch element to
completely overlap the control yet select an adjacent element with a partial overlap.

* touchadjustment/small-target-test-expected.txt: Added.
* touchadjustment/small-target-test.html: Added.

Modified Paths

Added Paths

Diff

Modified: trunk/LayoutTests/ChangeLog (123888 => 123889)


--- trunk/LayoutTests/ChangeLog	2012-07-27 17:56:49 UTC (rev 123888)
+++ trunk/LayoutTests/ChangeLog	2012-07-27 17:59:36 UTC (rev 123889)
@@ -1,3 +1,18 @@
+2012-07-27  Kevin Ellis  <[email protected]>
+
+        Improve touch adjustment for targetting small controls.
+        https://bugs.webkit.org/show_bug.cgi?id=92293
+
+        Reviewed by Antonio Gomes.
+
+        Adds a test for cases that were problematic for the touch adjustment algorithm.  Each example
+        in the test involves a small control immediately adjacent to a larger control.  When the touch
+        area is significantly larger than the small control, it was possible for the touch element to
+        completely overlap the control yet select an adjacent element with a partial overlap. 
+
+        * touchadjustment/small-target-test-expected.txt: Added.
+        * touchadjustment/small-target-test.html: Added.
+
 2012-07-27  Sudarsana Nagineni  <[email protected]>
 
         [EFL] Add missing baselines for fast/table/table-row-outline-paint.html

Added: trunk/LayoutTests/touchadjustment/small-target-test-expected.txt (0 => 123889)


--- trunk/LayoutTests/touchadjustment/small-target-test-expected.txt	                        (rev 0)
+++ trunk/LayoutTests/touchadjustment/small-target-test-expected.txt	2012-07-27 17:59:36 UTC (rev 123889)
@@ -0,0 +1,44 @@
+  
+ 
+ 
+Tests if small clickable targets adjacent to larger clickable controls are properly targetted with touch adjustment.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+Testing small direct hits.
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "task"
+PASS adjustedNode.id is "item-label"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "combo-label"
+PASS adjustedNode.id is "drop-down-selector"
+Testing indirect hits.
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "check"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "remove-button"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS adjustedNode.id is "drop-down-selector"
+PASS successfullyParsed is true
+
+TEST COMPLETE
+

Added: trunk/LayoutTests/touchadjustment/small-target-test.html (0 => 123889)


--- trunk/LayoutTests/touchadjustment/small-target-test.html	                        (rev 0)
+++ trunk/LayoutTests/touchadjustment/small-target-test.html	2012-07-27 17:59:36 UTC (rev 123889)
@@ -0,0 +1,214 @@
+<html>
+<head>
+<style>
+  body {
+    -webkit-margin-start: 20px;
+    margin-top: 20px;
+  }
+
+  .control-pair {
+    margin-bottom: 30px;
+  }
+
+  #check {
+    border: 1px solid black;
+    display: inline-block;
+    padding: 5px;
+    vertical-align: middle;
+  }
+
+  #remove-button {
+    border: none;
+    background: #999;
+    display: inline-block;
+    padding: 5px;
+    vertical-align: middle;
+  }
+
+  #combo-button {
+    border: 1px solid #777;
+    padding: 5px;
+    margin-top: 20px;
+  }
+
+  #combo-label {
+    border: 1px solid #777;
+    height: 1em;
+    width: 4em;
+  }
+
+  #combo-button,
+  #combo-label,
+  #drop-down-selector {
+    display: inline-block;
+    vertical-align: middle;
+  }
+
+  #drop-down-selector {
+    -webkit-box-sizing; content-box;
+    width: 0;
+    height: 0;
+    border-left: 5px solid transparent;
+    border-right: 5px solid transparent;
+    border-top: 10px solid #777;
+  }
+</style>
+
+<script src=""
+</head>
+
+<body id="mybody" _onload_="runTests()">
+
+<div class="control-pair">
+  <!-- Small checkbox to the left of another control -->
+  <div id="check" _onclick_="onClick();"></div>
+  <input id="task" type="text" value="Task 1"/>
+</div>
+<div class="control-pair">
+  <!-- Small remove button to the right of another control -->
+  <input id = "item-label" type="text" value = "Option 1"/>
+  <div id = "remove-button" _onclick_="onClick();"/>
+</div>
+<div>
+  <div id="combo-button">
+    <!-- Combo-button controls -->
+    <div id = "combo-label" _onclick_="function() {};"></div>
+    <div id = "drop-down-selector" _onclick_="onClick();"/>
+  </div>
+</div>
+
+<p id='description'></p>
+<div id='console'></div>
+
+<script>
+    var touchRadius = 20;
+    var touchOffset = 8;
+
+    var element;
+    var adjustedNode;
+
+    function onClick()
+    {
+    }
+
+    function findAbsolutePosition(node)
+    {
+        var pos = new Object();
+        pos.left = 0; pos.top = 0;
+        do {
+            pos.left += node.offsetLeft;
+            pos.top += node.offsetTop;
+        } while (node = node.offsetParent);
+        return pos;
+    }
+
+    function testDirectTouch(id)
+    {
+        testTouch(id, 'center');
+    }
+
+    function testIndirectTouch(id)
+    {
+        testTouch(id, 'left');
+        testTouch(id, 'right');
+        testTouch(id, 'top-left');
+        testTouch(id, 'top-right');
+        testTouch(id, 'bottom-left');
+        testTouch(id, 'bottom-right');
+        testTouch(id, 'top');
+        testTouch(id, 'bottom');
+    }
+
+    function testTouch(id, relativePosition)
+    {
+        element = document.getElementById(id);
+        var pos = findAbsolutePosition(element);
+        var x = pos.left; 
+        var y = pos.top;
+        // Set center point for touch.
+        switch (relativePosition) {
+        case 'center':
+          x += element.clientWidth / 2;
+          y += element.clientHeight / 2;
+          break;
+        case 'left':
+          x -= touchOffset;
+          y += element.clientHeight / 2;
+          break;
+        case 'right':
+           x += element.clientWidth + touchOffset;
+           y +=  element.clientHeight / 2;
+           break;
+        case 'top-left':
+          x -= touchOffset;
+          y -= touchOffset;
+          break;
+        case 'top-right':
+           x += element.clientWidth + touchOffset;
+           y -= touchOffset;
+           break;
+        case 'bottom-left':
+          x -= touchOffset;
+          y += element.clientHeight + touchOffset;
+          break;
+        case 'bottom-right':
+           x += element.clientWidth + touchOffset;
+           y += element.clientHeight + touchOffset;
+           break;
+        case 'top':
+           x += element.clientWidth / 2;
+           y -= touchOffset;
+           break;
+        case 'bottom':
+           x += element.clientWidth / 2;
+           y += element.clientHeight + touchOffset;
+        }
+        // Adjust from touch center to top-left corner.
+        x -= touchRadius;
+        y -= touchRadius;
+
+        var width = 2 * touchRadius;
+        var height = 2 * touchRadius;
+
+        adjustedNode = internals.touchNodeAdjustedToBestClickableNode(x, y, width, height, document);
+        while (adjustedNode.nodeType != 1 ) // Not an element.
+            adjustedNode = adjustedNode.parentNode;
+        shouldBeEqualToString('adjustedNode.id', id);
+    }
+
+    function testDirectTouches()
+    {
+        debug('Testing small direct hits.');
+
+        testDirectTouch('check');
+        testDirectTouch('task');
+        testDirectTouch('item-label');
+        testDirectTouch('remove-button');
+        testDirectTouch('combo-label');
+        testDirectTouch('drop-down-selector');
+    }
+
+    function testIndirectTouches()
+    {
+        debug('Testing indirect hits.');
+
+        testIndirectTouch('check');
+        testIndirectTouch('remove-button');
+        testIndirectTouch('drop-down-selector');
+    }
+
+    function runTests()
+    {
+        if (window.testRunner && window.internals && internals.touchNodeAdjustedToBestClickableNode) {
+            description('Tests if small clickable targets adjacent to larger clickable controls are properly targetted with touch adjustment.');
+            testRunner.dumpAsText();
+            testRunner.waitUntilDone();
+            testDirectTouches();
+            testIndirectTouches();
+            isSuccessfullyParsed();
+            testRunner.notifyDone();
+        }
+    }
+</script>
+</body>
+</html>

Modified: trunk/Source/WebCore/ChangeLog (123888 => 123889)


--- trunk/Source/WebCore/ChangeLog	2012-07-27 17:56:49 UTC (rev 123888)
+++ trunk/Source/WebCore/ChangeLog	2012-07-27 17:59:36 UTC (rev 123889)
@@ -1,5 +1,30 @@
 2012-07-27  Kevin Ellis  <[email protected]>
 
+        Improve touch adjustment for targetting small controls.
+        https://bugs.webkit.org/show_bug.cgi?id=92293
+
+        Reviewed by Antonio Gomes.
+
+        Modifies the touch adjustment algorithm to better discriminate small targets without introducing
+        a bias towards shorter links.  The revised scoring algorihtm uses a normalized distance to
+        center-line score and a normalized overlap score.  The better of the two scores is used for
+        determining the best candidate for target adjustment.  The center-line score works well
+        for discriminating elements with partial overlaps, where relying solely on percent overlap
+        could bias towards the smaller element.  Conversely, a high percentage in overlap provides a 
+        strong measure of confidence in a candidate target.  An additonal tie-break algorithm is
+        introduced to minimize the adjustment distance if two equally suitable candidates are found.
+
+        Test: touchadjustment/small-target-test.html
+
+        * page/TouchAdjustment.cpp:
+        (TouchAdjustment):
+        (WebCore::TouchAdjustment::nodeRespondsToTapGesture):
+        (WebCore::TouchAdjustment::hybridDistanceFunction):
+        (WebCore::TouchAdjustment::findNodeWithLowestDistanceMetric):
+        (WebCore::findBestClickableCandidate):
+
+2012-07-27  Kevin Ellis  <[email protected]>
+
         [chromium] Calendar for input type=date should be larger on devices that support touch input.
         https://bugs.webkit.org/show_bug.cgi?id=92424
 

Modified: trunk/Source/WebCore/page/TouchAdjustment.cpp (123888 => 123889)


--- trunk/Source/WebCore/page/TouchAdjustment.cpp	2012-07-27 17:56:49 UTC (rev 123888)
+++ trunk/Source/WebCore/page/TouchAdjustment.cpp	2012-07-27 17:59:36 UTC (rev 123889)
@@ -34,11 +34,14 @@
 #include "RenderBox.h"
 #include "RenderObject.h"
 #include "RenderStyle.h"
+#include "ShadowRoot.h"
 
 namespace WebCore {
 
 namespace TouchAdjustment {
 
+const float zeroTolerance = 1e-6f;
+
 // Class for remembering absolute quads of a target node and what node they represent.
 class SubtargetGeometry {
 public:
@@ -72,6 +75,7 @@
         if (element->hasTagName(HTMLNames::labelTag) && static_cast<HTMLLabelElement*>(element)->control())
             return true;
     }
+
     // FIXME: Implement hasDefaultEventHandler and use that instead of all of the above checks.
     if (node->hasEventListeners()
         && (node->hasEventListeners(eventNames().clickEvent)
@@ -219,7 +223,6 @@
     return rect.distanceSquaredFromCenterLineToPoint(touchHotspot);
 }
 
-
 // This returns quotient of the target area and its intersection with the touch area.
 // This will prioritize largest intersection and smallest area, while balancing the two against each other.
 float zoomableIntersectionQuotient(const IntPoint& touchHotspot, const IntRect& touchArea, const SubtargetGeometry& subtarget)
@@ -239,6 +242,34 @@
     return rect.size().area() / (float)intersection.size().area();
 }
 
+// Uses a hybrid of distance to center and intersect ratio, normalizing each
+// score between 0 and 1 and choosing the better score. The distance to
+// centerline works best for disambiguating clicks on targets such as links,
+// where the width may be significantly larger than the touch width. Using
+// area of overlap in such cases can lead to a bias towards shorter links.
+// Conversely, percentage of overlap can provide strong confidence in tapping
+// on a small target, where the overlap is often quite high, and works well
+// for tightly packed controls.
+float hybridDistanceFunction(const IntPoint& touchHotspot, const IntRect& touchArea, const SubtargetGeometry& subtarget)
+{
+    IntRect rect = subtarget.boundingBox();
+
+    // Convert from frame coordinates to window coordinates.
+    rect = subtarget.node()->document()->view()->contentsToWindow(rect);
+   
+    float touchWidth = touchArea.width();
+    float touchHeight = touchArea.height();
+    float distanceScale =  touchWidth * touchWidth + touchHeight * touchHeight;
+    float distanceToCenterScore = rect.distanceSquaredFromCenterLineToPoint(touchHotspot) / distanceScale;
+
+    float targetArea = rect.size().area();
+    rect.intersect(touchArea);
+    float intersectArea = rect.size().area();
+    float intersectionScore = 1 - intersectArea / targetArea;
+
+    return intersectionScore < distanceToCenterScore ? intersectionScore : distanceToCenterScore;
+}
+
 FloatPoint contentsToWindow(FrameView *view, FloatPoint pt)
 {
     int x = static_cast<int>(pt.x() + 0.5f);
@@ -320,6 +351,7 @@
     SubtargetGeometryList::const_iterator it = subtargets.begin();
     const SubtargetGeometryList::const_iterator end = subtargets.end();
     IntPoint adjustedPoint;
+
     for (; it != end; ++it) {
         Node* node = it->node();
         float distanceMetric = distanceFunction(touchHotspot, touchArea, *it);
@@ -330,12 +362,27 @@
                 targetNode = node;
                 bestDistanceMetric = distanceMetric;
             }
-        } else if (distanceMetric == bestDistanceMetric) {
-            // Try to always return the inner-most element.
-            if (node->isDescendantOf(targetNode) && snapTo(*it, touchHotspot, touchArea, adjustedPoint)) {
-                targetPoint = adjustedPoint;
-                targetNode = node;
-                targetArea = it->boundingBox();
+        } else if (distanceMetric - bestDistanceMetric < zeroTolerance) {
+            if (snapTo(*it, touchHotspot, touchArea, adjustedPoint)) {
+                if (node->isDescendantOf(targetNode)) {
+                    // Try to always return the inner-most element.
+                    targetPoint = adjustedPoint;
+                    targetNode = node;
+                    targetArea = it->boundingBox();
+                } else {
+                    // Minimize adjustment distance.
+                    float dx = targetPoint.x() - touchHotspot.x();
+                    float dy = targetPoint.y() - touchHotspot.y();
+                    float bestDistance = dx * dx + dy * dy;
+                    dx = adjustedPoint.x() - touchHotspot.x();
+                    dy = adjustedPoint.y() - touchHotspot.y();
+                    float distance = dx * dx + dy * dy;
+                    if (distance < bestDistance) {
+                        targetPoint = adjustedPoint;
+                        targetNode = node;
+                        targetArea = it->boundingBox();
+                    }
+                }
             }
         }
     }
@@ -352,7 +399,7 @@
     IntRect targetArea;
     TouchAdjustment::SubtargetGeometryList subtargets;
     TouchAdjustment::compileSubtargetList(nodeList, subtargets, TouchAdjustment::nodeRespondsToTapGesture);
-    return TouchAdjustment::findNodeWithLowestDistanceMetric(targetNode, targetPoint, targetArea, touchHotspot, touchArea, subtargets, TouchAdjustment::distanceSquaredToTargetCenterLine);
+    return TouchAdjustment::findNodeWithLowestDistanceMetric(targetNode, targetPoint, targetArea, touchHotspot, touchArea, subtargets, TouchAdjustment::hybridDistanceFunction);
 }
 
 bool findBestZoomableArea(Node*& targetNode, IntRect& targetArea, const IntPoint& touchHotspot, const IntRect& touchArea, const NodeList& nodeList)
_______________________________________________
webkit-changes mailing list
[email protected]
http://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to