Branch: refs/heads/main
  Home:   https://github.com/WebKit/WebKit
  Commit: c688b31fdd986a6fdbe6643d1d5983042b601af7
      
https://github.com/WebKit/WebKit/commit/c688b31fdd986a6fdbe6643d1d5983042b601af7
  Author: Samar Sunkaria <[email protected]>
  Date:   2024-09-16 (Mon, 16 Sep 2024)

  Changed paths:
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-capitalize-expected.txt
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-capitalize.html
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-lowercase-expected.txt
    A LayoutTests/accessibility/mac/text-operation/text-operation-lowercase.html
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-replace-expected.txt
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-replace-preserve-case-expected.txt
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-replace-preserve-case.html
    A LayoutTests/accessibility/mac/text-operation/text-operation-replace.html
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-select-expected.txt
    A LayoutTests/accessibility/mac/text-operation/text-operation-select.html
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-smart-replace-expected.txt
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-smart-replace.html
    A 
LayoutTests/accessibility/mac/text-operation/text-operation-uppercase-expected.txt
    A LayoutTests/accessibility/mac/text-operation/text-operation-uppercase.html
    M LayoutTests/platform/mac-wk1/TestExpectations
    M LayoutTests/platform/mac-wk2/TestExpectations
    M LayoutTests/resources/accessibility-helper.js
    M Source/WebCore/accessibility/AXCoreObject.h
    M Source/WebCore/accessibility/AccessibilityObject.cpp
    M Source/WebCore/accessibility/mac/WebAccessibilityObjectWrapperMac.mm
    M Tools/WebKitTestRunner/InjectedBundle/AccessibilityTextMarkerRange.cpp
    M Tools/WebKitTestRunner/InjectedBundle/AccessibilityTextMarkerRange.h
    M Tools/WebKitTestRunner/InjectedBundle/AccessibilityUIElement.cpp
    M Tools/WebKitTestRunner/InjectedBundle/AccessibilityUIElement.h
    M Tools/WebKitTestRunner/InjectedBundle/Bindings/AccessibilityUIElement.idl
    M Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm

  Log Message:
  -----------
  AX: Allow strict replacements via AXTextOperation
https://bugs.webkit.org/show_bug.cgi?id=278458
rdar://134409037

Reviewed by Tyler Wilcock.

The AXTextOperation accessibility API for macOS allows text replacement within 
a specified
list of text marker ranges, among other operations. This AX API is particularly 
beneficial
for clients like Grammarly for macOS, that utilize the accessibility subsystem 
to perform
text replacements. However, in its current form, AXTextOperation arbitrarily 
changes the
case of the replacement string and performs a “smart replacement” that adds 
spaces around
the replaced text, severely limiting the usability of the API. This commit 
introduces the
following modifications to the API to allow better control over the 
replacement. While the
same outcome can be achieved by chaining multiple calls to AX APIs/emitting 
CGEvents,
AXTextOperation allows for a more efficient implementation.

Preserving the Case of the Replacement String:

`TextOperationReplace` modifies the case of the replacement string supplied to 
the API,
changing it to either capitalized or lowercase, presumably to match the case of 
the text
being replaced. As a result, clients of AXTextOperation cannot enforce the case 
of the
replacement string they provide.

A real example of the existing behavior that makes incorrect/unexpected changes 
shows up
when the case of the replacement string must be preserved, e.g. when replacing 
“Mac OS X”
with “macOS”. The current implementation would infer that “Mac OS X” is 
capitalized and
would attempt to capitalize the replacement string “macOS” -> “MacOS” to 
preserve the
case.

To address this, an additional `AXTextOperationType` called
"TextOperationReplacePreserveCase" has been introduced, which allows for text 
replacements
that preserve the case of the replacement string for clients that know/want the
replacement string to maintain its case. When supplying the new type, the 
replacement
string will be used as is, without attempting to match its case to the string 
being
replaced. Since this is an additive change, the existing clients of 
AXTextOperation will
remain unaffected.

Allow Disabling Smart Replacements:

All AXTextOperation types that replace text (i.e. all operations types except 
“select”),
always perform a “smart replace”, which assumes that the replacement string is 
not
replacing partial words and attempts to add spaces around it. While this is an 
excellent
feature for certain replacements, it can severely limit the usability of 
AXTextOperation
for partial word replacements.

For example, when replacing “MacOS” with “macOS”, we could attempt to perform a 
minimal
change, where the “M” is replaced with “m” (given we already have
`TextOperationReplacePreserveCase` from above). The current implementation 
would perform
a smart replacement and, therefore, add a space after the replaced “m”,
“MacOS” -> “m acOS”.

To address this, a new key called "AXTextOperationSmartReplace" has been added 
to the
AXTextOperation dictionary to support _non-smart_ replacements. The value of 
this key is
a boolean, where NO/false disables smart replacements. Since this is an 
additive change,
the existing clients of AXTextOperation will remain unaffected.

Usage:

Client pseudo-code below:
```
// Replace “Mac OS X” -> “macOS”
CFTypeRef result;
AXUIElementCopyParameterizedAttributeValue(element, "AXTextOperation", @[
  NSAccessibilityTextOperationType: 
NSAccessibilityTextOperationReplacePreserveCase,
  NSAccessibilityTextOperationReplacementString: @"macOS",
  NSAccessibilityTextOperationMarkerRanges: @[ rangeOfMacOSX ]
], &result)

// Replace "MacOS" -> "macOS", by only replacing the first character
CFTypeRef result;
AXUIElementCopyParameterizedAttributeValue(element, "AXTextOperation", @[
  NSAccessibilityTextOperationType: 
NSAccessibilityTextOperationReplacePreserveCase,
  NSAccessibilityTextOperationReplacementString: @"m",
  NSAccessibilityTextOperationSmartReplace: kCFBooleanFalse,
  NSAccessibilityTextOperationMarkerRanges: @[ rangeOfM ]
], &result)
```

Layout tests have been introduced for the various AXTextOperation types. At the 
time of
writing, AXTextOperation has a bug with replacements/transformations over 
multiple
ranges, which is why some of the tests cases are expected to fail until the bug 
is
resolved. See https://bugs.webkit.org/show_bug.cgi?id=278928

* 
LayoutTests/accessibility/mac/text-operation/text-operation-capitalize-expected.txt:
 Added.
* LayoutTests/accessibility/mac/text-operation/text-operation-capitalize.html: 
Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-lowercase-expected.txt:
 Added.
* LayoutTests/accessibility/mac/text-operation/text-operation-lowercase.html: 
Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-replace-expected.txt:
 Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-replace-preserve-case-expected.txt:
 Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-replace-preserve-case.html:
 Added.
* LayoutTests/accessibility/mac/text-operation/text-operation-replace.html: 
Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-select-expected.txt:
 Added.
* LayoutTests/accessibility/mac/text-operation/text-operation-select.html: 
Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-smart-replace-expected.txt:
 Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-smart-replace.html: 
Added.
* 
LayoutTests/accessibility/mac/text-operation/text-operation-uppercase-expected.txt:
 Added.
* LayoutTests/accessibility/mac/text-operation/text-operation-uppercase.html: 
Added.
* LayoutTests/platform/mac-wk1/TestExpectations:
* LayoutTests/platform/mac-wk2/TestExpectations:
* Source/WebCore/accessibility/AXCoreObject.h:
(WebCore::AccessibilityTextOperation::AccessibilityTextOperation): Deleted.
* Source/WebCore/accessibility/AccessibilityObject.cpp:
(WebCore::AccessibilityObject::performTextOperation):
* Source/WebCore/accessibility/mac/WebAccessibilityObjectWrapperMac.mm:
(accessibilityTextOperationForParameterizedAttribute):
* Tools/WebKitTestRunner/InjectedBundle/AccessibilityTextMarkerRange.cpp:
(WTR::toTextMarkerRange):
* Tools/WebKitTestRunner/InjectedBundle/AccessibilityTextMarkerRange.h:
* Tools/WebKitTestRunner/InjectedBundle/AccessibilityUIElement.cpp:
(WTR::AccessibilityUIElement::performTextOperation):
* Tools/WebKitTestRunner/InjectedBundle/AccessibilityUIElement.h:
* Tools/WebKitTestRunner/InjectedBundle/Bindings/AccessibilityUIElement.idl:
* Tools/WebKitTestRunner/InjectedBundle/mac/AccessibilityUIElementMac.mm:
(WTR::textOperationParameterizedAttribute):
(WTR::AccessibilityUIElement::performTextOperation):

Canonical link: https://commits.webkit.org/283740@main



To unsubscribe from these emails, change your notification settings at 
https://github.com/WebKit/WebKit/settings/notifications
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to