Proposal: User Interface Independence for Accessible Rich Internet Applications

Mon, 30 Aug 2010 02:07:31 -0700 

The following proposal is for consideration by the PF Working Group, DOM 
Working Group, and HTML Working Group. It specifically solves a few outstanding 
accessibility problems, but has much broader implications, and could affect 
several recommendations, including ARIA 2, DOM 3, and HTML 5. (Note: duplicate 
emails are going out to a few relevant lists.)

Thanks,
James Craig
Title: User Interface Independence for Accessible Rich Internet Applications

User Interface Independence for Accessible Rich Internet Applications

W3C Proposal: 30 August 2010

Authors:
James Craig ([email protected]), Apple Inc.
Chris Fleizach ([email protected]), Apple Inc.

1. Introduction

This proposal is intended address the problem of device-, OS-, and localization-independent control of widgets in accessible rich internet applications. The authors of this document do not believe the existence of this problem should block adoption of WAI-ARIA 1.0. The PFWG ISSUE-352, initially raised in August 2009, has been deferred to consideration for ARIA 2.0.

1.1. Feature Implementation

The authors anticipate that this proposal will not be published as a standalone W3C Recommendation. Instead, the authors request that the features proposed in this document be integrated, as appropriate, into the specifications for HTML 5, DOM Level 3, and/or ARIA 2.

1.2. Problem Description

At the time of this writing, the current draft of WAI-ARIA states that:

Accessibility of web content requires semantic information about widgets, structures, and behaviors, in order to allow assistive technologies to convey appropriate information to persons with disabilities. [The WAI-ARIA] specification provides an ontology of roles, states, and properties that define accessible user interface elements and can be used to improve the accessibility and interoperability of web content and applications. These semantics are designed to allow an author to properly convey user interface behaviors and structural information to assistive technologies in document-level markup.

The WAI-ARIA 1.0 draft specifies an adequate and standardized means for web applications to convey accessibility semantics through declarative markup in the DOM. However, there is currently no standardized, robust means for assistive technology to assume control of customized user interface widgets, or convey the user's intent for control of the web application. Simple control over web applications can be achieved by using standard activation events, like click or DOMActivate, but full control over secondary actions (tree expansion, grid sorting, scrolling) can only be achieved by using a set of keyboard shortcuts promoted by a group outside the W3C WAI calling itself the DHTML Style Guide Working Group. The style guide recommends authors register for all key events and listen, for example, for Control+Shift+M to open a popup menu during a drag and drop operation.

While we appreciate the efforts of the DHTML Style Guide Working Group, we feel that user interface should be controlled by operating systems, and not defined as part of any technical specification. We also believe that there are many serious problems with the DHTML Style Guide, including the following items:

  • Adoption: Implementing complex behavior in ARIA 1.0 widgets requires that the web application authors adopt this set of keyboard commands, which may have been a reasonable requirement, except that it also requires all users of assistive technology to learn these commands in addition to their existing assistive technology interface, and understand that these commands will only behave as expected on a small minority of web applications that have adopted the DHTML Style Guide approach.
  • Localization: Web standards are intended for a global audience, and specific keyboard shortcuts, like Control+Shift+M, cannot be required for certain localizations. For example, not all keyboards have Control, Shift, or M keys, and even if they did, the mnemonic logic that "M opens a menu" does not apply in most languages.
  • Operating system independence: The DHTML Style Guide takes a Windows-centric approach, where the Control key is used as the primary modifier key. Other operating systems may have different primary modifier keys, such as the Command key on Mac OS X, and recommending an OS-centric set of keyboard shortcuts adds additional burden on the end user, web application author, or both.
  • User interface independence: Perhaps the most serious flaw in the assumption of control via keyboard shortcuts is that not all devices have a keyboard. Touch screen devices, or speech-controlled assistive technologies, for example, would be forced to simulate specific keypress events on all web pages, in the unlikely event that the web author might have registered for these events.

For these reasons and others, we believe adoption of complex behavior in ARIA 1.0 widgets will be slow for any control that cannot be activated with a single, easy-to-learn, easy-to-remember, localization-independent keystroke like arrow keys, PageUp, PageDown, Home, End, Spacebar, Enter, Delete, Escape, etc. We also believe that the keyboard shortcuts recommended in the DHTML Style Guide should only be used as a temporary stop-gap measure. The authors wish to encourage the HTML, DOM, and PF Working Groups to provide a standardized solution with the utmost expedience.

It is in this interest that we offer the following proposal, which we believe solves the aforementioned problems. The proposal is organized into three parts: UI Change Request Events for all user agents, Accessibility Events for assistive technology, and additions to the Navigator interface for Assistive Technology Identification and Notification.

2. UI Change Request Events

The core principle behind UI Change Request Events is that they operate on a completely backwards-compatible, opt-in basis. In other words, the web application author has to be aware of these events and register event listeners, or the user agent and assistive technology behave as they normally would.

Unlike other proposals that allow user agents or assistive technology to make direct changes to the DOM, these change request events do not cause any direct manipulation or mutation of the DOM. Instead, the event object conveys the user's intent to the web application, and allows the web application to make the appropriate changes to the DOM, on behalf of the user agent or assistive technology. If a web application is authored to understand the change request event, it can cancel the event using preventDefault(), which informs the user agent or assistive technology that the event has been captured and understood. If a web application does not cancel any change request event, the event returns to the user agent or assistive technology, which can then attempt fallback behavior or communicate to the user that the input has not been recognized.

Note: The UIRequestEvent interface does not inherit from AccessibilityEvent (proposed in the following section), because we believe it will ultimately be useful outside the context of assistive technology. The DOM Working Group may find this a lightweight, performant alternative to certain mutation events. For example, DOMAttrChangeRequest instead of DOMAttrModified.

Interface UIRequestEvent

interface UIRequestEvent : UIEvent {
  
  // UA or AT notifies web app of a change request
  const unsigned short    UNDO    =  1;
  const unsigned short    REDO    =  2;
  const unsigned short    CANCEL  =  3;
  const unsigned short    DELETE  =  4;
  // expect more type constants will be added for other event types…
  // ZOOM_IN / ZOOM_OUT ? perhaps that should just be slider widget, covered under ValueChangeRequest
  
  readonly attribute unsigned short    eventType;
  
  void initUIRequestEvent(
         in DOMString typeArg,
         in boolean canBubbleArg,
         in boolean cancelableArg,
         in views::AbstractView viewArg,
         in long detailArg,
         in unsigned short eventTypeArg
       );
};

The different types of UIRequestEvents that can occur are:

UndoRequest

Initiated when the user agent or assistive technology sends an 'undo' request to the web application.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: eventType
RedoRequest

Initiated when the user agent or assistive technology sends a 'redo' request to the web application.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: eventType
EscapeRequest

Initiated when the user agent or assistive technology sends a 'escape' request to the web application.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: eventType
DeleteRequest

Initiated when the user agent or assistive technology sends a 'delete' request to the web application.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: eventType

Examples

  1. Users, wanting to 'undo' a discrete action in a web application, can indicate their intent a number of ways, including pressing Control+Z on Windows or Linux, Command+Z on Mac OS X, and even by shaking some accelerometer- or gyroscope-enabled mobile devices. User agents understanding this intent should initiate an UndoRequest event. Web authors who have registered for this event should process the event to determine whether to cancel the event. If the 'undo' action is understood in the context of the web application, web authors should undo the user's change, and cancel the event using the event object's preventDefault() method. If the event is not cancelled by the web author, user agents may pass the literal interaction event to the web application; for example, as a keypress event.
  2. Users, wanting to 'escape' a web application state (for example, closing a modal dialog), can indicate their intent a number of ways, including pressing Escape on most keyboard-controlled operating systems, or by using a two-finger scrub gesture in VoiceOver on iOS or Mac OS X. User agents understanding this intent should initiate an EscapeRequest event. Web authors who have registered for this event should process the event to determine whether to cancel the event. If the 'escape' action is understood in the context of the web application, web authors should perform the appropriate action (such as closing the dialog), and cancel the event using the event object's preventDefault() method. If the event is not cancelled by the web author, user agents may pass the literal interaction event to the web application; for example, as a keypress event.

To-do: Explanation needed about event order precedence.

An area where this proposal is notably lacking is in regards to the precedence of event order in mainstream user agents, especially in regards to continuous input events. For example, user interactions with starting and ending events (touchstart/touchend, shakestart/shakeend, etc.) should probably take precendence over UIRequestEvents in mainstream user agents. UIRequestEvents are only intended to be used in mainstream user agents where appropriate, but the definition of "where appropriate" is not something we would attempt to define at the time of this proposal. Instead, we offer the previous examples for the sake of discussion.

  1. Note that the 'shake' interaction mentioned in the first example could be sent to the web app as a shakestart event which, if not captured, could then be sent as an UndoRequest. If the the web app does not capture either event, the shake interaction would be ignored by the web application.
  2. Also note that the 'scrub gesture' interaction mentioned in the second example is specific to assistive technology (VoiceOver on iOS, which intercepts all touch events), so it's appropriate for the UIRequestEvent to take precedence, and never send a touchstart event.

Interface UIScrollRequestEvent

interface UIScrollRequestEvent : UIRequestEvent {
  
  // for custom scroll views or widgets (e.g. carousels, lists, grids)
  const unsigned short    LEFT          =  1;
  const unsigned short    UP            =  2;
  const unsigned short    RIGHT         =  3;
  const unsigned short    DOWN          =  4;
  const unsigned short    LEFT_LIMIT    =  5;
  const unsigned short    TOP_LIMIT     =  6;
  const unsigned short    RIGHT_LIMIT   =  7;
  const unsigned short    BOTTOM_LIMIT  =  8;
  
  readonly attribute unsigned short   scrollType;
  
  void initUIScrollRequestEvent(
         in DOMString typeArg,
         in boolean canBubbleArg,
         in boolean cancelableArg,
         in views::AbstractView viewArg,
         in long detailArg,
         in unsigned short eventTypeArg
         in unsigned short scrollTypeArg
       );
};

The single type of UIScrollRequestEvent that can occur is:

ScrollRequest

Initiated when the user agent or assistive technology sends a scroll request to the web application. Scroll events need only be used on custom scroll views (lists and grids showing data subsets, carousels, etc.), as user agents and assistive technologies already manage scrolling of native scroll views.

Note: The scroll type constants are more or less equivalent to expected behavior for PageUp/PageDown and Home/End keys on native scroll views, but also allow horizontal scrolling.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: scrollType

Interface UIValueChangeRequestEvent

interface UIValueChangeRequestEvent : UIRequestEvent {
  
  // value changes (e.g. ranges)
  const unsigned short    INCREMENT        =  1;
  const unsigned short    INCREMENT_SMALL  =  2;
  const unsigned short    INCREMENT_LARGE  =  3;
  const unsigned short    INCREMENT_MAX    =  4;
  const unsigned short    DECREMENT        =  5;
  const unsigned short    DECREMENT_SMALL  =  6;
  const unsigned short    DECREMENT_LARGE  =  7;
  const unsigned short    DECREMENT_MIN    =  8;
  
  readonly attribute unsigned short   changeType;
  
  void initUIValueChangeRequestEvent(
         in DOMString typeArg,
         in boolean canBubbleArg,
         in boolean cancelableArg,
         in views::AbstractView viewArg,
         in long detailArg,
         in unsigned short eventTypeArg,
         in unsigned short changeTypeArg
       );
};

The single type of UIValueChangeRequestEvent that can occur is:

ValueChangeRequest

Initiated when the user agent or assistive technology sends a value change request to the web application.

Web authors should code applications to accept all values of the changeType argument. For example, if there is no special behavior for INCREMENT_SMALL or INCREMENT_LARGE, web applications would behave as if they had received a basic INCREMENT change type.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: eventType

Example

Users, wanting to change the value of a custom range widget (slider, media progressbar, etc.) in a web application, can indicate their intent a number of ways, including pressing various keys (Up, Down, Left, Right, PageUp, PageDown, Home, End) on most keyboard-controlled interfaces, and through gestures on many touch-enabled interfaces. User agents understanding this intent should initiate a ValueChangeRequest event. Web authors who have registered for this event, should process the event to determine whether to cancel the event. If the value change action is understood in the context of the web application, web authors should change the value of the associated widget by an amount determined via the changeType argument, and cancel the event using the event object's preventDefault() method. If the event is not cancelled by the web author, user agents may pass the literal interaction event to the web application; in this case, in the form of a keypress or touch event.

Interface DOMAttributeChangeRequestEvent

interface DOMAttributeChangeRequestEvent : UIRequestEvent {
  
  readonly attribute DOMString    attrName;
  readonly attribute DOMString    newValue;
  
  void initDOMAttributeChangeRequestEvent(
         in DOMString typeArg,
         in boolean canBubbleArg,
         in boolean cancelableArg,
         in views::AbstractView viewArg,
         in long detailArg,
         in unsigned short eventTypeArg,
         in DOMString attrNameArg,
         in DOMString newValueArg
       );
};

The single type of DOMAttributeChangeRequestEvent that can occur is:

DOMAttrChangeRequest

Initiated when the user agent or assistive technology sends an attribute change request to the web application. In order for web applications to understand the intent of change request events from the user agent or assistive technology, these change requests should be limited to attributes for which a change in value indicates a discrete, defined action for known types of widgets and UI elements.

Note: Currently, this limitation indicates that DOMAttrChangeRequest only applies to WAI-ARIA widgets, but it has the potential to be used with future iterations of HTML5 or SVG.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: attrName, newValue

Examples

  1. If a user wanted to expand or collapse the current node of an ARIA tree, the assistive technology would initiate a DOMAttrChangeRequest event on the tree item element, with the attrName equal to aria-expanded, and the newValue equal to true or false.
  2. If a user wanted to sort an ARIA grid on a particular column, the assistive technology would initiate a DOMAttrChangeRequest event on the column header element, with the attrName equal to aria-sort, and the newValue equal to ascending or descending.

3. Accessibility Events

The following events are only initiated by assistive technologies, as opposed to mainstream user agents.

Interface AccessibilityEvent

interface AccessibilityEvent : UIEvent {
  void initAccessibilityEvent(
         in DOMString typeArg,
         in boolean canBubbleArg,
         in boolean cancelableArg,
         in views::AbstractView viewArg,
         in long detailArg
       );
};

The different types of AccessibilityEvents that can occur are:

AXFocus (Editorial Note: possibly renamed AXFocusIn)

Initiated when the assistive technology cursor moves to a particular DOM node.

  • Bubbles: Yes
  • Cancelable: No
  • Context Info: None
AXBlur (Editorial Note: possibly renamed AXFocusOut)

Initiated when the assistive technology cursor leaves a particular DOM node.

  • Bubbles: Yes
  • Cancelable: No
  • Context Info: None
AXDragSelect

Initiated by the assistive technology in order to inform the web application than a draggable element (e.g. an element with an explicit aria-grabbed value) should be marked for dragging.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: None
AXDragCancel

Editorial Note: This event may not be necessary; could use general EscapeRequest instead.

Initiated by the assistive technology in order to inform the web application than the current drag operation should be cancelled.

  • Bubbles: Yes
  • Cancelable: No … Editorial Note: Does this need to be cancelable?
  • Context Info: None
AXDragDrop

Initiated by the assistive technology in order to inform the web application than a specific drop target (e.g. an element with an explicit aria-dropeffect value) should receive the drop event of the current drag operation.

  • Bubbles: Yes
  • Cancelable: Yes
  • Context Info: None

4. Assistive Technology Identification and Notification

In certain cases, it is beneficial for web authors to be aware of the assistive technology in use. This section defines a collection of attributes that can be used to determine, from script, the kind of assistive technology in use, in order to be aware of considerations for a particular assistive technology, including the need to support the UI Change Request Events or Accessibility Events defined in previous sections. Web authors should always limit client detection to detecting known versions. Web authors should always assume future versions and unknown versions to be fully compliant.

The accessibility attribute of the Navigator interface must return an instance of the Accessibility interface, which represents the identity and state of the assistive technology, and allows web pages to optionally notify assistive technology of certain interface events that cannot be indicated through declarative markup such as WAI-ARIA roles, states, and properties.

Interface Accessibility

interface Accessibility {
  readonly attribute InterfaceObject accessibility; // Editorial Note: Is it necessary to define this attribute in the IDL?
  // objects implementing the Accessibility interface also implement the ScreenReader and AccessibilityNotifications interfaces
};
Accessibility implements AccessibilityNotifications;
Accessibility implements ScreenReader;
Accessibility implements Magnifier;
Accessibility implements Speech;

[Supplemental, NoInterfaceObject]
interface AccessibilityNotifications {
  void elementsChanged();
  void screenChanged();
};

Methods for Interface Accessibility

window.navigator.accessibility.elementsChanged()
Returns void. Allows the web author to send a supplemental notification to the screen reader that certain aspects of relevant DOM elements have changed, informing the assistive technology to update its cache of the accessibility tree. For example, web authors might call this method after an animation has completed, allowing a screen reader to update its cursor location, if the focused element was moved.
window.navigator.accessibility.screenChanged()
Returns void. Allows the web author to send an explicit notification to the assistive technology that all aspects of a web application view have changed, informing the assistive technology to update its cache of the accessibility tree. Web authors should call this method when a majority of the web view changes without triggering a full page refresh in the browser.

Interface ScreenReader

The screenreader attribute of the Accessibility interface must return an instance of the ScreenReader interface:

interface ScreenReader {
  readonly attribute InterfaceObject screenreader; // Editorial Note: Is it necessary to define this attribute in the IDL?
  // objects implementing the ScreenReader interface also implement the interfaces given below
};
ScreenReader implements ScreenReaderID;
ScreenReader implements ScreenReaderStatus;

[Supplemental, NoInterfaceObject]
interface ScreenReaderID {
  readonly attribute DOMString name;
  readonly attribute DOMString version;
};

[Supplemental, NoInterfaceObject]
interface ScreenReaderStatus {
  readonly attribute boolean active;
};

Properties for Interface ScreenReader

window.navigator.accessibility.screenreader.active
Returns a boolean indicating whether or not a screen reader is in use. The user agent must return true if a screen reader is running, or false otherwise. User agents may return false if the user has chosen to disallow sharing this information with the requesting domain.
window.navigator.accessibility.screenreader.name
Returns the application name of the screen reader in use. The user agent must return either an empty string or a string representing the name of the screen reader in detail, e.g. "Apple VoiceOver". User agents may return an empty string if the user has chosen to disallow sharing this information with the requesting domain.
window.navigator.accessibility.screenreader.version
Returns the application version of the screen reader in use. The user agent must return either an empty string or a string representing the version of the screen reader in detail, e.g. "4.0 (220.2)". User agents may return an empty string if the user has chosen to disallow sharing this information with the requesting domain.

Note: The authors recommend that user agents adopt a domain-level security policy for the ScreenReader interface that is similar to the security policy for location data or cookies. A user should be able to explicitly disallow sharing of this information altogether, or on a per-domain basis.

Interface Magnifier

The magnifier attribute of the Accessibility interface must return an instance of the Magnifier interface:

interface Magnifier {
  readonly attribute InterfaceObject magnifier; // Editorial Note: Is it necessary to define this attribute in the IDL?
  // objects implementing the Magnifier interface also implement the interfaces given below
};
Magnifier implements MagnifierID;
Magnifier implements MagnifierStatus;
Magnifier implements MagnifierNotifications;

[Supplemental, NoInterfaceObject]
interface MagnifierID {
  readonly attribute DOMString name;
  readonly attribute DOMString version;
};

[Supplemental, NoInterfaceObject]
interface MagnifierStatus {
  readonly attribute boolean active;
};

[Supplemental, NoInterfaceObject]
interface MagnifierNotifications {
  // Editorial Note: I'm not partial to the method name focusPosition. Any other suggestions?
  void focusPosition(
         in DOMElement element,
         in optional array cursorRect,
         in optional array selectionPolygon
       );
};

Properties for Interface Magnifier

window.navigator.accessibility.magnifier.active
Returns a boolean indicating whether or not a screen magnifier is in use. The user agent must return true if a screen magnifier is enabled (even if the current zoom level is 1.0), or false if a magnifier is not enabled. User agents may return false if the user has chosen to disallow sharing this information with the requesting domain.
window.navigator.accessibility.magnifier.name
Returns the application name of the screen magnifier in use. The user agent must return either an empty string or a string representing the name of the screen magnifier in detail, e.g. "Apple Universal Access Zoom". User agents may return an empty string if the user has chosen to disallow sharing this information with the requesting domain.
window.navigator.accessibility.magnifier.version
Returns the application version of the screen magnifier in use. The user agent must return either an empty string or a string representing the version of the screen magnifier in detail, e.g. "4.0 (220.2)". User agents may return an empty string if the user has chosen to disallow sharing this information with the requesting domain.

Methods for Interface Magnifier

window.navigator.accessibility.magnifier.focusPosition ( element, cursorRect, selectionPolygon )

Returns void. Allows the web author to send an explicit notification informing the screen magnifier to update its cache of the cursor location. Web authors should call this method when updating the display or cursor position of custom views.

Parameters for Magnifier.focusPosition
1. DOMElement element
Required reference to the element (e.g. canvas) that has focus.
2. optional array cursorRect
Optional array representing the rectangular cursor location relative to the [0,0] position of focused element. [ x1, y1, x2, y2 ]
3. optional array selectionPolygon
Optional array representing the selection polygon relative to the [0,0] position of focused element. [ x1, y1, x2, y2, ... xn, yn ]

Editorial Note: It may be better if the optional params are shape objects with a coords array and a type string like 'rect' or 'poly'. That would allow the author to decide if they wanted to provide the extra polygon coords, or just use a simple shape.

Note: The focusPosition() method would be necessary for screen magnifiers to work with canvas-based web apps like Bespin or the 280 North projects. This method is not necessary for text views managed by the user agent, including contentEditable views.

Interface Speech

The speech attribute of the Accessibility interface must return an instance of the Speech interface:

interface Speech {
  // TBD, placeholder for speech-controlled user agents or assistive technology
  // may need speech.synthesis and speech.recognition; each could have name/version/engine, etc. 
  // potential method: speech.synthesis.speak() to trigger TTS from web app
  // potential method: speech.recognition.updateDictionary() to capture correct pronunciation of uncommon or app-specific words
};

Reply via email to