Branch: refs/heads/main
Home: https://github.com/WebKit/WebKit
Commit: 602545c636e67d2e48f25962a51f872d6fe5d86b
https://github.com/WebKit/WebKit/commit/602545c636e67d2e48f25962a51f872d6fe5d86b
Author: Wenson Hsieh <[email protected]>
Date: 2026-08-24 (Mon, 24 Aug 2026)
Changed paths:
M Source/WebCore/page/text-extraction/TextExtraction.cpp
M Tools/TestWebKitAPI/Tests/WebKit/WKWebView/TextExtractionTests.mm
Log Message:
-----------
[Text Extraction] Add more text context before or after a targeted element
that's missing meaningful text context
https://bugs.webkit.org/show_bug.cgi?id=322436
rdar://184783170
Reviewed by Abrar Rahman Protyasha and Megan Gardner.
Improve both text-search-based targeting and summary strings for targeted
elements with little-to-no
accessible attributes or other context:
1. When targeting based on text, we currently require the search text to be
present in the rendered
DOM of the targeted element. However, in the case where the agent is trying
to target a button
or link that's a sibling of target text, this can cause confusing failures
(from the perspective
of the agent), since the target element doesn't (literally) contain the
search text.
Mitigate this by relaxing the constraints around text search, so that if we
don't find search
text underneath the target container, we fall back to searching the nearest
non-inline ancestor
of the target element for the search text.
2. Instead of surfacing an interaction description like:
> Click on span with classes “sprite-fm-mono icon-arrow-right expand”
...in the case where a targeted element has no text, we should pull text
context from rendered
text siblings within the same block-level container, such that the
interaction description can
look something more like:
> Click on span with classes “sprite-fm-mono icon-arrow-right expand”
before rendered text “Security”
Test: TextExtractionTests.InteractionDescriptionAndSearchTextForLabellessIcons
Test: Tools/TestWebKitAPI/Tests/WebKit/WKWebView/TextExtractionTests.mm
* Source/WebCore/page/text-extraction/TextExtraction.cpp:
(WebCore::TextExtraction::nearestNonInlineAncestor):
(WebCore::TextExtraction::searchForInteractionTargetText):
(WebCore::TextExtraction::resolveMouseTarget):
(WebCore::TextExtraction::adjacentRenderedTextLabel):
(WebCore::TextExtraction::adjacentRenderedTextLabelDescription):
(WebCore::TextExtraction::hasNoRenderedTextOrLabeledChild):
(WebCore::TextExtraction::textDescription):
(WebCore::TextExtraction::precedingRenderedTextLabel): Deleted.
* Tools/TestWebKitAPI/Tests/WebKit/WKWebView/TextExtractionTests.mm:
(TestWebKitAPI::TEST(TextExtractionTests,
InteractionDescriptionAndSearchTextForLabellessIcons)):
Canonical link: https://commits.webkit.org/319761@main
To unsubscribe from these emails, change your notification settings at
https://github.com/WebKit/WebKit/settings/notifications
