- Revision
- 276746
- Author
- [email protected]
- Date
- 2021-04-28 18:36:32 -0700 (Wed, 28 Apr 2021)
Log Message
[iOS] Coalesce adjacent selection geometries when rendering individual selection quads
https://bugs.webkit.org/show_bug.cgi?id=225054
Reviewed by Tim Horton.
Source/WebCore:
Add a heuristic to coalesce adjacent selection geometries when `SelectionRenderingBehavior::UseIndividualQuads`
is active. Currently, we pass them directly through to the list of coalesced geometries inside
`RenderObject::collectSelectionGeometries`, but this can lead to almost perfectly adjacent selection quads
showing a narrow gap in between their selection views.
Instead, we can detect this case where neighboring quads can be united into a single quad without significantly
changing the geometry. Rather than append the new selection geometry in this case, we simply adjust the last
selection geometry's quad so that it encompasses the new selection geometry as well, by extending the top right
and bottom right extent points of the last selection quad to the new selection quad's extent points.
See below for more details.
Test: fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html
* html/HTMLElement.h: `WEBCORE_EXPORT` a couple of helper methods.
* rendering/RenderObject.cpp:
(WebCore::RenderObject::collectSelectionGeometriesInternal):
Avoid sorting selection geometries by bounding rect `logicalLeft` in the case where we're not coalescing
bounding rects (and also update the preceding comment accordingly).
(WebCore::coalesceSelectionGeometryWithAdjacentQuadsIfPossible):
Add a new helper to unite neighboring quads if: (1) the top right and bottom right of the current quad are very
close to the top left and bottom left of the next quad (respectively), and (2) the angle of rotation of the
bounding rect of the current quad is very close to that of the next quad.
(WebCore::RenderObject::collectSelectionGeometries):
Use the above helper method to either coalesce or append selection geometries to the final vector of coalesced
geometries.
Source/WebKit:
Make some further minor adjustments to selection behavior in image overlays on iOS. In particular, when using
selection handles to modify the selected range, shift the start and end of the selected range so that they're at
the beginning and end of text runs, respectively.
* WebProcess/WebPage/ios/WebPageIOS.mm:
(WebKit::insideImageOverlay):
(WebKit::expandForImageOverlay):
(WebKit::rangeForPointInRootViewCoordinates):
LayoutTests:
Add a new layout test to verify that adjacent selection rects rotated by the same amount can be coalesced.
* fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay-expected.txt: Added.
* fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html: Added.
Modified Paths
Added Paths
Diff
Modified: trunk/LayoutTests/ChangeLog (276745 => 276746)
--- trunk/LayoutTests/ChangeLog 2021-04-29 01:00:26 UTC (rev 276745)
+++ trunk/LayoutTests/ChangeLog 2021-04-29 01:36:32 UTC (rev 276746)
@@ -1,3 +1,15 @@
+2021-04-28 Wenson Hsieh <[email protected]>
+
+ [iOS] Coalesce adjacent selection geometries when rendering individual selection quads
+ https://bugs.webkit.org/show_bug.cgi?id=225054
+
+ Reviewed by Tim Horton.
+
+ Add a new layout test to verify that adjacent selection rects rotated by the same amount can be coalesced.
+
+ * fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay-expected.txt: Added.
+ * fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html: Added.
+
2021-04-28 Robert Jenner <[email protected]>
[ iOS wk2 EWS ] imported/w3c/web-platform-tests/workers/interfaces/DedicatedWorkerGlobalScope/postMessage/structured-clone-message.html is flaky crash
Added: trunk/LayoutTests/fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay-expected.txt (0 => 276746)
--- trunk/LayoutTests/fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay-expected.txt (rev 0)
+++ trunk/LayoutTests/fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay-expected.txt 2021-04-29 01:36:32 UTC (rev 276746)
@@ -0,0 +1,5 @@
+PASS selectionRectsAfterSelectingAll.length is 1
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
Added: trunk/LayoutTests/fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html (0 => 276746)
--- trunk/LayoutTests/fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html (rev 0)
+++ trunk/LayoutTests/fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html 2021-04-29 01:36:32 UTC (rev 276746)
@@ -0,0 +1,53 @@
+<!DOCTYPE html>
+<html>
+<head>
+<style>
+body, html {
+ margin: 0;
+}
+</style>
+<script src=""
+<script src=""
+</head>
+<body>
+<img src=""
+<script>
+jsTestIsAsync = true;
+
+addEventListener("load", async () => {
+ internals.installImageOverlay(document.querySelector("img"), [
+ {
+ topLeft : new DOMPointReadOnly(0.1, 0.3),
+ topRight : new DOMPointReadOnly(0.3, 0.1),
+ bottomRight : new DOMPointReadOnly(0.4, 0.2),
+ bottomLeft : new DOMPointReadOnly(0.2, 0.4),
+ children: [
+ {
+ text : "AB",
+ topLeft : new DOMPointReadOnly(0.1, 0.3),
+ topRight : new DOMPointReadOnly(0.2, 0.2),
+ bottomRight : new DOMPointReadOnly(0.3, 0.3),
+ bottomLeft : new DOMPointReadOnly(0.2, 0.4),
+ },
+ {
+ text : "CD",
+ topLeft : new DOMPointReadOnly(0.2, 0.2),
+ topRight : new DOMPointReadOnly(0.3, 0.1),
+ bottomRight : new DOMPointReadOnly(0.4, 0.2),
+ bottomLeft : new DOMPointReadOnly(0.3, 0.3),
+ }
+ ]
+ }
+ ]);
+
+ await UIHelper.longPressAtPoint(80, 120);
+ await UIHelper.waitForSelectionToAppear();
+ document.execCommand("SelectAll");
+ await UIHelper.ensurePresentationUpdate();
+ selectionRectsAfterSelectingAll = await UIHelper.getUISelectionViewRects();
+ shouldBe("selectionRectsAfterSelectingAll.length", "1");
+ finishJSTest();
+});
+</script>
+</body>
+</html>
\ No newline at end of file
Modified: trunk/Source/WebCore/ChangeLog (276745 => 276746)
--- trunk/Source/WebCore/ChangeLog 2021-04-29 01:00:26 UTC (rev 276745)
+++ trunk/Source/WebCore/ChangeLog 2021-04-29 01:36:32 UTC (rev 276746)
@@ -1,3 +1,42 @@
+2021-04-28 Wenson Hsieh <[email protected]>
+
+ [iOS] Coalesce adjacent selection geometries when rendering individual selection quads
+ https://bugs.webkit.org/show_bug.cgi?id=225054
+
+ Reviewed by Tim Horton.
+
+ Add a heuristic to coalesce adjacent selection geometries when `SelectionRenderingBehavior::UseIndividualQuads`
+ is active. Currently, we pass them directly through to the list of coalesced geometries inside
+ `RenderObject::collectSelectionGeometries`, but this can lead to almost perfectly adjacent selection quads
+ showing a narrow gap in between their selection views.
+
+ Instead, we can detect this case where neighboring quads can be united into a single quad without significantly
+ changing the geometry. Rather than append the new selection geometry in this case, we simply adjust the last
+ selection geometry's quad so that it encompasses the new selection geometry as well, by extending the top right
+ and bottom right extent points of the last selection quad to the new selection quad's extent points.
+
+ See below for more details.
+
+ Test: fast/images/image-extraction/ios/coalesce-selection-rects-in-image-overlay.html
+
+ * html/HTMLElement.h: `WEBCORE_EXPORT` a couple of helper methods.
+ * rendering/RenderObject.cpp:
+ (WebCore::RenderObject::collectSelectionGeometriesInternal):
+
+ Avoid sorting selection geometries by bounding rect `logicalLeft` in the case where we're not coalescing
+ bounding rects (and also update the preceding comment accordingly).
+
+ (WebCore::coalesceSelectionGeometryWithAdjacentQuadsIfPossible):
+
+ Add a new helper to unite neighboring quads if: (1) the top right and bottom right of the current quad are very
+ close to the top left and bottom left of the next quad (respectively), and (2) the angle of rotation of the
+ bounding rect of the current quad is very close to that of the next quad.
+
+ (WebCore::RenderObject::collectSelectionGeometries):
+
+ Use the above helper method to either coalesce or append selection geometries to the final vector of coalesced
+ geometries.
+
2021-04-28 Rob Buis <[email protected]>
[aspect-ratio] Treat border/padding correctly for box-sizing: border-box
Modified: trunk/Source/WebCore/html/HTMLElement.h (276745 => 276746)
--- trunk/Source/WebCore/html/HTMLElement.h 2021-04-29 01:00:26 UTC (rev 276745)
+++ trunk/Source/WebCore/html/HTMLElement.h 2021-04-29 01:36:32 UTC (rev 276746)
@@ -135,8 +135,8 @@
WEBCORE_EXPORT static bool shouldExtendSelectionToTargetNode(const Node& targetNode, const VisibleSelection& selectionBeforeUpdate);
WEBCORE_EXPORT bool hasImageOverlay() const;
- static bool isInsideImageOverlay(const SimpleRange&);
- static bool isInsideImageOverlay(const Node&);
+ WEBCORE_EXPORT static bool isInsideImageOverlay(const SimpleRange&);
+ WEBCORE_EXPORT static bool isInsideImageOverlay(const Node&);
WEBCORE_EXPORT static bool isImageOverlayText(const Node&);
WEBCORE_EXPORT static bool isImageOverlayText(const Node*);
Modified: trunk/Source/WebCore/rendering/RenderObject.cpp (276745 => 276746)
--- trunk/Source/WebCore/rendering/RenderObject.cpp 2021-04-29 01:00:26 UTC (rev 276745)
+++ trunk/Source/WebCore/rendering/RenderObject.cpp 2021-04-29 01:36:32 UTC (rev 276746)
@@ -2251,9 +2251,13 @@
// Adjust line height.
adjustLineHeightOfSelectionGeometries(geometries, numberOfGeometries, lineNumber, lineTop, lineBottom - lineTop);
- // Sort the rectangles and make sure there are no gaps. The rectangles could be unsorted when
- // there is ruby text and we could have gaps on the line when adjacent elements on the line
- // have a different orientation.
+ // When using SelectionRenderingBehavior::CoalesceBoundingRects, sort the rectangles and make sure there are no gaps.
+ // The rectangles could be unsorted when there is ruby text and we could have gaps on the line when adjacent elements
+ // on the line have a different orientation.
+ //
+ // Note that for selection geometries with SelectionRenderingBehavior::UseIndividualQuads, we avoid sorting in order to
+ // preserve the fact that the resulting geometries correspond to the order in which the quads are discovered during DOM
+ // traversal. This allows us to efficiently coalesce adjacent selection quads.
size_t firstRectWithCurrentLineNumber = 0;
for (size_t currentRect = 1; currentRect < numberOfGeometries; ++currentRect) {
if (geometries[currentRect].lineNumber() != geometries[currentRect - 1].lineNumber()) {
@@ -2263,6 +2267,9 @@
if (geometries[currentRect].logicalLeft() >= geometries[currentRect - 1].logicalLeft())
continue;
+ if (geometries[currentRect].behavior() != SelectionRenderingBehavior::CoalesceBoundingRects)
+ continue;
+
auto selectionRect = geometries[currentRect];
size_t i;
for (i = currentRect; i > firstRectWithCurrentLineNumber && selectionRect.logicalLeft() < geometries[i - 1].logicalLeft(); --i)
@@ -2303,6 +2310,35 @@
return { WTFMove(geometries), maxLineNumber };
}
+static bool coalesceSelectionGeometryWithAdjacentQuadsIfPossible(SelectionGeometry& current, const SelectionGeometry& next)
+{
+ auto nextQuad = next.quad();
+ if (nextQuad.isEmpty())
+ return true;
+
+ auto areCloseEnoughToCoalesce = [](const FloatPoint& first, const FloatPoint& second) {
+ constexpr float maxDistanceBetweenBoundaryPoints = 2;
+ return (first - second).diagonalLengthSquared() <= maxDistanceBetweenBoundaryPoints * maxDistanceBetweenBoundaryPoints;
+ };
+
+ auto currentQuad = current.quad();
+ if (!areCloseEnoughToCoalesce(currentQuad.p2(), nextQuad.p1()) || !areCloseEnoughToCoalesce(currentQuad.p3(), nextQuad.p4()))
+ return false;
+
+ if (std::abs(rotatedBoundingRectWithMinimumAngleOfRotation(currentQuad).angleInRadians - rotatedBoundingRectWithMinimumAngleOfRotation(nextQuad).angleInRadians) > radiansPerDegreeFloat)
+ return false;
+
+ currentQuad.setP2(nextQuad.p2());
+ currentQuad.setP3(nextQuad.p3());
+ current.setQuad(currentQuad);
+ current.setDirection(current.containsStart() || current.containsEnd() ? current.direction() : next.direction());
+ current.setContainsStart(current.containsStart() || next.containsStart());
+ current.setContainsEnd(current.containsEnd() || next.containsEnd());
+ current.setIsFirstOnLine(current.isFirstOnLine() || next.isFirstOnLine());
+ current.setIsLastOnLine(current.isLastOnLine() || next.isLastOnLine());
+ return true;
+}
+
Vector<SelectionGeometry> RenderObject::collectSelectionGeometries(const SimpleRange& range)
{
auto result = RenderObject::collectSelectionGeometriesInternal(range);
@@ -2315,10 +2351,10 @@
for (size_t i = 0; i < numberOfGeometries; ++i) {
auto& currentGeometry = result.geometries[i];
if (currentGeometry.behavior() == SelectionRenderingBehavior::UseIndividualQuads) {
- // FIXME: We still probably want some way to coalesce quads, probably by projecting them onto rotated bounding rects
- // and then checking whether the rotated bounding rects are overlapping and share the same rotation angle. Until then,
- // we simply append all non-empty quads.
- if (!currentGeometry.quad().isEmpty())
+ if (currentGeometry.quad().isEmpty())
+ continue;
+
+ if (coalescedGeometries.isEmpty() || !coalesceSelectionGeometryWithAdjacentQuadsIfPossible(coalescedGeometries.last(), currentGeometry))
coalescedGeometries.append(currentGeometry);
continue;
}
Modified: trunk/Source/WebKit/ChangeLog (276745 => 276746)
--- trunk/Source/WebKit/ChangeLog 2021-04-29 01:00:26 UTC (rev 276745)
+++ trunk/Source/WebKit/ChangeLog 2021-04-29 01:36:32 UTC (rev 276746)
@@ -1,3 +1,19 @@
+2021-04-28 Wenson Hsieh <[email protected]>
+
+ [iOS] Coalesce adjacent selection geometries when rendering individual selection quads
+ https://bugs.webkit.org/show_bug.cgi?id=225054
+
+ Reviewed by Tim Horton.
+
+ Make some further minor adjustments to selection behavior in image overlays on iOS. In particular, when using
+ selection handles to modify the selected range, shift the start and end of the selected range so that they're at
+ the beginning and end of text runs, respectively.
+
+ * WebProcess/WebPage/ios/WebPageIOS.mm:
+ (WebKit::insideImageOverlay):
+ (WebKit::expandForImageOverlay):
+ (WebKit::rangeForPointInRootViewCoordinates):
+
2021-04-28 Devin Rousso <[email protected]>
experiment with averaging sampling colors across the top of the page as the scroll area background
Modified: trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm (276745 => 276746)
--- trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm 2021-04-29 01:00:26 UTC (rev 276745)
+++ trunk/Source/WebKit/WebProcess/WebPage/ios/WebPageIOS.mm 2021-04-29 01:36:32 UTC (rev 276746)
@@ -1340,6 +1340,34 @@
return constrainedPoint;
}
+static bool insideImageOverlay(const VisiblePosition& position)
+{
+ auto container = makeRefPtr(position.deepEquivalent().containerNode());
+ return container && HTMLElement::isInsideImageOverlay(*container);
+}
+
+static Optional<SimpleRange> expandForImageOverlay(const SimpleRange& range)
+{
+ VisiblePosition expandedStart(makeContainerOffsetPosition(&range.startContainer(), range.startOffset()));
+ VisiblePosition expandedEnd(makeContainerOffsetPosition(&range.endContainer(), range.endOffset()));
+
+ for (auto start = expandedStart; insideImageOverlay(start); start = start.previous()) {
+ if (auto container = makeRefPtr(start.deepEquivalent().containerNode()); is<Text>(container)) {
+ expandedStart = firstPositionInNode(container.get()).downstream();
+ break;
+ }
+ }
+
+ for (auto end = expandedEnd; insideImageOverlay(end); end = end.next()) {
+ if (auto container = makeRefPtr(end.deepEquivalent().containerNode()); is<Text>(container)) {
+ expandedEnd = lastPositionInNode(container.get()).upstream();
+ break;
+ }
+ }
+
+ return makeSimpleRange({ expandedStart, expandedEnd });
+}
+
void WebPage::selectWithGesture(const IntPoint& point, GestureType gestureType, GestureRecognizerState gestureState, bool isInteractingWithFocusedElement, CompletionHandler<void(const WebCore::IntPoint&, GestureType, GestureRecognizerState, OptionSet<SelectionFlags>)>&& completionHandler)
{
if (static_cast<GestureRecognizerState>(gestureState) == GestureRecognizerState::Began)
@@ -1537,6 +1565,9 @@
range = makeSimpleRange(result, selectionEnd);
}
+ if (range && HTMLElement::isInsideImageOverlay(*range))
+ return expandForImageOverlay(*range);
+
return range;
}
