Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 60f57c6dce48e685b7a9a27020bfb1f1d45633f6
https://github.com/WebKit/WebKit/commit/60f57c6dce48e685b7a9a27020bfb1f1d45633f6
Author: Wenson Hsieh <[email protected]>
Date: 2026-07-21 (Tue, 21 Jul 2026)
Changed paths:
M LayoutTests/fast/text-extraction/debug-text-extraction-basic-expected.txt
M LayoutTests/fast/text-extraction/debug-text-extraction-basic.html
M
LayoutTests/fast/text-extraction/debug-text-extraction-scrolling-expected.txt
M Source/WebCore/page/text-extraction/TextExtraction.cpp
M Source/WebKit/Shared/TextExtractionToStringConversion.cpp
M Tools/TestWebKitAPI/Tests/WebKit/WKWebView/TextExtractionTests.mm
Log Message:
-----------
[Text Extraction] Simulated clicks should target the innermost descendant in
the targeted element
https://bugs.webkit.org/show_bug.cgi?id=319901
rdar://182822622
Reviewed by Abrar Rahman Protyasha.
Make a couple of adjustments which allow the agent to click the "Me" (login)
button on www.iq.com,
even when the promotional overlay is showing up over the rest of the page:
1. Surface more context for elements with only an accessibility role (like
`uid=… role=button`), so
that if the `class` and/or `id` happen to contain additional context,
agents are still able to
reason about it.
2. Currently, if the agent tries to click an element, we first try to simulate
real mousedown/up
events near the center of the targeted element. In the case where the
element is obscured by
another container (in this case, a modal promotional overlay), we fall back
to simulating a
programmatic click on the target. While this works well in most cases,
because it's dispatched
onto the target element directly, it doesn't bubble up the event target
chain properly (so if
the content expects the event target to be an inner descendant, the click
doesn't behave as
expected).
To fix this, we perform a piercing hit-test to find the innermost
descendant at the center of
the target element, and then simulate the programmatic click over that
innermost descendant
instead (allowing the event to bubble up the chain to the target).
Test: TextExtractionTests.InteractionClicksThroughOccludingOverlay
* LayoutTests/fast/text-extraction/debug-text-extraction-basic-expected.txt:
* LayoutTests/fast/text-extraction/debug-text-extraction-basic.html:
Augment an existing layout test to cover the changes for [1] above.
* Source/WebCore/page/text-extraction/TextExtraction.cpp:
(WebCore::TextExtraction::frontmostHitTestedElementInSubtree):
See [2] above.
(WebCore::TextExtraction::dispatchSimulatedClick):
* Source/WebKit/Shared/TextExtractionToStringConversion.cpp:
(WebKit::recognizedClassesAndIdForItem):
See [1] above.
* Tools/TestWebKitAPI/Tests/WebKit/WKWebView/TextExtractionTests.mm:
(TestWebKitAPI::TEST(TextExtractionTests,
InteractionClicksThroughOccludingOverlay)):
Add an API test for [2] above.
Canonical link: https://commits.webkit.org/317666@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
