- Revision
- 286021
- Author
- [email protected]
- Date
- 2021-11-18 12:48:15 -0800 (Thu, 18 Nov 2021)
Log Message
[macOS] [Live Text] Avoid analyzing images in editable content
https://bugs.webkit.org/show_bug.cgi?id=233317
Reviewed by Megan Gardner.
Source/WebCore:
Make macOS Live Text behavior consistent with iOS, and avoid automatically triggering text recognition (Live
Text) when hovering over images in editable content. In addition to platform consistency, this also allows us to
avoid handling both image service controls and image overlay content inside editable image elements in Mail
compose.
Test: fast/images/text-recognition/mac/text-recognition-candidates.html
* page/EventHandler.cpp:
(WebCore::EventHandler::textRecognitionCandidateElement const):
Add the editability check here and return null.
* page/EventHandler.h:
Drive-by code cleanup -- move `textRecognitionCandidateElement` behind the `ENABLE(IMAGE_ANALYSIS)` compile-time
flag, since it's only used from image analysis code.
* testing/Internals.cpp:
(WebCore::Internals::textRecognitionCandidate const):
Add an internal testing hook to query the current text recognition candidate element (i.e. a suitable image that
is being hovered, or otherwise null).
* testing/Internals.h:
* testing/Internals.idl:
LayoutTests:
Add a layout test to exercise the change.
* fast/images/text-recognition/mac/text-recognition-candidates-expected.txt: Added.
* fast/images/text-recognition/mac/text-recognition-candidates.html: Added.
Modified Paths
Added Paths
Diff
Modified: trunk/LayoutTests/ChangeLog (286020 => 286021)
--- trunk/LayoutTests/ChangeLog 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/LayoutTests/ChangeLog 2021-11-18 20:48:15 UTC (rev 286021)
@@ -1,3 +1,15 @@
+2021-11-18 Wenson Hsieh <[email protected]>
+
+ [macOS] [Live Text] Avoid analyzing images in editable content
+ https://bugs.webkit.org/show_bug.cgi?id=233317
+
+ Reviewed by Megan Gardner.
+
+ Add a layout test to exercise the change.
+
+ * fast/images/text-recognition/mac/text-recognition-candidates-expected.txt: Added.
+ * fast/images/text-recognition/mac/text-recognition-candidates.html: Added.
+
2021-11-18 Yoshiaki Jitsukawa <[email protected]>
Implement JPEG XL image decoder using libjxl
Added: trunk/LayoutTests/fast/images/text-recognition/mac/text-recognition-candidates-expected.txt (0 => 286021)
--- trunk/LayoutTests/fast/images/text-recognition/mac/text-recognition-candidates-expected.txt (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/mac/text-recognition-candidates-expected.txt 2021-11-18 20:48:15 UTC (rev 286021)
@@ -0,0 +1,12 @@
+Verifies that Live Text does not trigger for images in editable content. This test requires WebKitTestRunner/DumpRenderTree.
+
+On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
+
+
+PASS topImage is internals.textRecognitionCandidate
+PASS internals.textRecognitionCandidate is null
+PASS successfullyParsed is true
+
+TEST COMPLETE
+
+
Added: trunk/LayoutTests/fast/images/text-recognition/mac/text-recognition-candidates.html (0 => 286021)
--- trunk/LayoutTests/fast/images/text-recognition/mac/text-recognition-candidates.html (rev 0)
+++ trunk/LayoutTests/fast/images/text-recognition/mac/text-recognition-candidates.html 2021-11-18 20:48:15 UTC (rev 286021)
@@ -0,0 +1,35 @@
+<!DOCTYPE html>
+<html>
+<head>
+<script src=""
+<style>
+img {
+ width: 200px;
+ height: 200px;
+}
+</style>
+</head>
+<body>
+<img id="top" src=""
+<div contenteditable><img id="bottom" src=""
+<script>
+function mouseOverElement(element) {
+ const bounds = element.getBoundingClientRect();
+ eventSender.mouseMoveTo(bounds.left + bounds.width / 2, bounds.top + bounds.height / 2);
+}
+
+description("Verifies that Live Text does not trigger for images in editable content. This test requires WebKitTestRunner/DumpRenderTree.");
+addEventListener("load", () => {
+ if (!window.internals || !window.eventSender)
+ return;
+
+ topImage = document.getElementById("top");
+ mouseOverElement(topImage);
+ shouldBe("topImage", "internals.textRecognitionCandidate");
+
+ mouseOverElement(document.getElementById("bottom"));
+ shouldBeNull("internals.textRecognitionCandidate");
+});
+</script>
+</body>
+</html>
\ No newline at end of file
Modified: trunk/Source/WebCore/ChangeLog (286020 => 286021)
--- trunk/Source/WebCore/ChangeLog 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/Source/WebCore/ChangeLog 2021-11-18 20:48:15 UTC (rev 286021)
@@ -1,3 +1,36 @@
+2021-11-18 Wenson Hsieh <[email protected]>
+
+ [macOS] [Live Text] Avoid analyzing images in editable content
+ https://bugs.webkit.org/show_bug.cgi?id=233317
+
+ Reviewed by Megan Gardner.
+
+ Make macOS Live Text behavior consistent with iOS, and avoid automatically triggering text recognition (Live
+ Text) when hovering over images in editable content. In addition to platform consistency, this also allows us to
+ avoid handling both image service controls and image overlay content inside editable image elements in Mail
+ compose.
+
+ Test: fast/images/text-recognition/mac/text-recognition-candidates.html
+
+ * page/EventHandler.cpp:
+ (WebCore::EventHandler::textRecognitionCandidateElement const):
+
+ Add the editability check here and return null.
+
+ * page/EventHandler.h:
+
+ Drive-by code cleanup -- move `textRecognitionCandidateElement` behind the `ENABLE(IMAGE_ANALYSIS)` compile-time
+ flag, since it's only used from image analysis code.
+
+ * testing/Internals.cpp:
+ (WebCore::Internals::textRecognitionCandidate const):
+
+ Add an internal testing hook to query the current text recognition candidate element (i.e. a suitable image that
+ is being hovered, or otherwise null).
+
+ * testing/Internals.h:
+ * testing/Internals.idl:
+
2021-11-18 Antoine Quint <[email protected]>
[Model] add support for getting and setting the camera
Modified: trunk/Source/WebCore/page/EventHandler.cpp (286020 => 286021)
--- trunk/Source/WebCore/page/EventHandler.cpp 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/Source/WebCore/page/EventHandler.cpp 2021-11-18 20:48:15 UTC (rev 286021)
@@ -2524,6 +2524,8 @@
return false;
}
+#if ENABLE(IMAGE_ANALYSIS)
+
RefPtr<Element> EventHandler::textRecognitionCandidateElement() const
{
RefPtr candidateElement = m_elementUnderMouse;
@@ -2535,6 +2537,9 @@
if (!candidateElement)
return nullptr;
+ if (candidateElement->hasEditableStyle())
+ return nullptr;
+
auto renderer = candidateElement->renderer();
if (!is<RenderImage>(renderer))
return nullptr;
@@ -2552,6 +2557,8 @@
return candidateElement;
}
+#endif // ENABLE(IMAGE_ANALYSIS)
+
void EventHandler::updateMouseEventTargetNode(const AtomString& eventType, Node* targetNode, const PlatformMouseEvent& platformMouseEvent, FireMouseOverOut fireMouseOverOut)
{
Ref<Frame> protectedFrame(m_frame);
Modified: trunk/Source/WebCore/page/EventHandler.h (286020 => 286021)
--- trunk/Source/WebCore/page/EventHandler.h 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/Source/WebCore/page/EventHandler.h 2021-11-18 20:48:15 UTC (rev 286021)
@@ -354,6 +354,10 @@
WEBCORE_EXPORT void invalidateClick();
+#if ENABLE(IMAGE_ANALYSIS)
+ WEBCORE_EXPORT RefPtr<Element> textRecognitionCandidateElement() const;
+#endif
+
static bool scrollableAreaCanHandleEvent(const PlatformWheelEvent&, ScrollableArea&);
private:
@@ -362,8 +366,6 @@
static const Seconds TextDragDelay;
#endif
- RefPtr<Element> textRecognitionCandidateElement() const;
-
bool eventActivatedView(const PlatformMouseEvent&) const;
bool updateSelectionForMouseDownDispatchingSelectStart(Node*, const VisibleSelection&, TextGranularity);
void selectClosestWordFromHitTestResult(const HitTestResult&, AppendTrailingWhitespace);
Modified: trunk/Source/WebCore/testing/Internals.cpp (286020 => 286021)
--- trunk/Source/WebCore/testing/Internals.cpp 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/Source/WebCore/testing/Internals.cpp 2021-11-18 20:48:15 UTC (rev 286021)
@@ -5775,8 +5775,6 @@
};
}
-#endif // ENABLE(IMAGE_ANALYSIS)
-
void Internals::requestTextRecognition(Element& element, RefPtr<VoidCallback>&& callback)
{
auto page = contextDocument()->page();
@@ -5785,18 +5783,22 @@
callback->handleEvent();
}
-#if ENABLE(IMAGE_ANALYSIS)
page->chrome().client().requestTextRecognition(element, { }, [callback = WTFMove(callback)] (auto&&) {
if (callback)
callback->handleEvent();
});
-#else
- UNUSED_PARAM(element);
- if (callback)
- callback->handleEvent();
-#endif
}
+RefPtr<Element> Internals::textRecognitionCandidate() const
+{
+ if (RefPtr frame = contextDocument()->frame())
+ return frame->eventHandler().textRecognitionCandidateElement();
+
+ return nullptr;
+}
+
+#endif // ENABLE(IMAGE_ANALYSIS)
+
void Internals::installImageOverlay(Element& element, Vector<ImageOverlayLine>&& lines)
{
if (!is<HTMLElement>(element))
Modified: trunk/Source/WebCore/testing/Internals.h (286020 => 286021)
--- trunk/Source/WebCore/testing/Internals.h 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/Source/WebCore/testing/Internals.h 2021-11-18 20:48:15 UTC (rev 286021)
@@ -919,7 +919,11 @@
~ImageOverlayLine();
};
void installImageOverlay(Element&, Vector<ImageOverlayLine>&&);
+
+#if ENABLE(IMAGE_ANALYSIS)
void requestTextRecognition(Element&, RefPtr<VoidCallback>&&);
+ RefPtr<Element> textRecognitionCandidate() const;
+#endif
bool isSystemPreviewLink(Element&) const;
bool isSystemPreviewImage(Element&) const;
Modified: trunk/Source/WebCore/testing/Internals.idl (286020 => 286021)
--- trunk/Source/WebCore/testing/Internals.idl 2021-11-18 20:04:48 UTC (rev 286020)
+++ trunk/Source/WebCore/testing/Internals.idl 2021-11-18 20:48:15 UTC (rev 286021)
@@ -939,7 +939,8 @@
boolean isSystemPreviewLink(Element element);
boolean isSystemPreviewImage(Element element);
- undefined requestTextRecognition(Element element, VoidCallback callback);
+ [Conditional=IMAGE_ANALYSIS] readonly attribute Element? textRecognitionCandidate;
+ [Conditional=IMAGE_ANALYSIS] undefined requestTextRecognition(Element element, VoidCallback callback);
undefined installImageOverlay(Element element, sequence<ImageOverlayLine> lines);
boolean usingAppleInternalSDK();
