The problem I have with using only explicitly defined action functions
is that the whole point of simulating keyboard events is to test
whether or not the user interface is accessible using the keyboard!
(This is not just an issue for people using assistive technology like
a screen reader: there are a few things in Chrome that are impossible
to access without using a mouse now.)
I want to be able to write tests that try to edit bookmarks using the
Bookmark Manager, or change settings using the Options dialog, for
example. Having functions like selectNextBookmark() wouldn't help test
that they're actually accessible via the keyboard.
There aren't that many different keys needed for keyboard
accessibility: off the top of my head, you mostly need tab, shift-tab,
space, enter, and arrow keys - plus a few shortcuts to open a menu or
move the focus to a certain panel. What about having functions for
each of these keys plus other actions that can be triggered via a
keyboard shortcut?
tabToNextControl()
tabToPreviousControl()
pressSpaceBar()
pressEnterKey()
pressEscapeKey()
arrowUp()
arrowDown()
arrowLeft()
arrowRight()
openOptionsDialog()
openBookmarksManager()
focusLocationBar()
...
Then, in order to enter text into a text box in a dialog, perhaps a
function that pastes unicode text at the current cursor position would
be much easier than trying to deal with key codes and input methods.
- Dominic
On Fri, Jan 15, 2010 at 3:24 AM, James Su <su...@google.com> wrote:
> Hi Nico,
> I totally agree with your point. Using fake keyboard events for
> accessibility things would lead to compatibility nightmare. We should
> consider the approach of using explicitly defined action functions rather
> than emulating keyboard events. The keyboard events approach has at least
> following unsolvable problems:
> 1. Compatibility issue among different OS platforms.
> 2. Compatibility issue caused by user customized shortcuts.
> 3. Supports of different keyboard layouts.
> 4. Supports of inputting non-Latin text.
> Regards
> James Su
> 2010/1/15 Nico Weber <tha...@chromium.org>
>>
>> On Thu, Jan 14, 2010 at 8:37 AM, Dominic Mazzoni <dmazz...@google.com>
>> wrote:
>> > On Thu, Jan 14, 2010 at 8:19 AM, Nico Weber <tha...@google.com> wrote:
>> >> It probably depends if you want to use this for text input or "action"
>> >> inputs. A text-to-speech extension would probably want to set unicode
>> >> characters, while something that (say) hits cmd-f to open the find bar
>> >> probably wants to use keycodes.
>> >
>> > Text-to-speech is a different problem; this is for actions.
>> >
>> >> For example, on OS X hitting cmd-"f" with a hiragana keyboard layout
>> >> opens the findbar, while just pressing "f" opens an IME.
>> >>
>> >> If you want to use this for actons, I would find it more useful to
>> >> have a "performUiAction()" function instead though, since keyboard
>> >> shortcuts are different across platforms (all mac shortcuts use cmd
>> >> instead of ctrl for example, and the letters for a few shortcuts are
>> >> different on OS X too for various reasons).
>> >
>> > I completely agree; I would like to add many functions along the lines
>> > of performUiAction. However, there are a few actions for which typing
>> > a key really is the right thing to do - for example pressing Tab to
>> > move to the next control.
>>
>> Maybe performUiAction('focusNextViewInKeyLoop')?
>>
>> (By the way, OS X has a system-level setting that configures if tab
>> should focus only text fields or all controls such as buttons etc. I
>> don't think Chrome/Mac honors it, but I think there are plans to
>> support it. Stuff like this is surprising for a caller that sends
>> "tab" characters, while an explicit "focus next element" function
>> could have clearly defined semantics).
>>
>> >
>> > There's also testing. This function will allow you to use only
>> > javascript to test that a particular capability can be accessed using
>> > only the keyboard. I'm already using this for testing this API, by
>> > simulating pressing Ctrl+L to focus the location bar and then checking
>> > to see that the proper focus event is generated.
>>
>> Some (most?) accessibility shortcuts will vary by platform too. For
>> example, "focus toolbar" is ctrl-f5 on OS X, but only if full keyboard
>> access has been enabled (ctrl-f1). Cocoa the OS X framework don't
>> always trigger with synthetic keyboard events. (I'm not an a11y expert
>> – Avi, comments?)
>>
>> >
>> > So the way I look at it is: there are some great uses for this
>> > function. There are also things that this function could be used for
>> > but there'd be a better way. It could also be abused to do things that
>> > make no sense.
>> >
>> > I don't think that's an argument not to allow the function; I think it
>> > just means we should make sure it's defined properly for the intended
>> > use, and over time try to provide better alternatives for other
>> > possible uses.
>> >
>> > - Dominic
>> >
>> >> Nico
>> >>
>> >> On Thu, Jan 14, 2010 at 7:26 AM, Dominic Mazzoni <dmazz...@google.com>
>> >> wrote:
>> >>> Hi Darin,
>> >>>
>> >>> Erik suggested you might have some thoughts. In my proposed extension
>> >>> api for accessibility (http://codereview.chromium.org/402099) one of
>> >>> the functions is to simulate a key press. How should the client
>> >>> express the key they would like to press? The current proposed
>> >>> function prototype is:
>> >>>
>> >>> {
>> >>> "name": "simulateKeyPress",
>> >>> "type": "function",
>> >>> "description": "Simulate pressing a key.",
>> >>> "parameters": [
>> >>> {
>> >>> "type": "object",
>> >>> "name": "keyInfo",
>> >>> "properties": {
>> >>> "key": {"type": "integer", "description": "The code of
>> >>> the key to press, corresponding to event.keyCode."},
>> >>> "control": {"type": "boolean", "optional": true,
>> >>> "description": "True if the control key is down."},
>> >>> "shift": {"type": "boolean", "optional": true,
>> >>> "description": "True if the shift key is down."},
>> >>> "alt": {"type": "boolean", "optional": true,
>> >>> "description": "True if the alt key is down."}
>> >>> }
>> >>> }
>> >>> ]
>> >>> }
>> >>>
>> >>> What do you think? Should the key be a keyCode? A charCode?
>> >>>
>> >>> Should it be cross-platform, or should it match what would be returned
>> >>> by an onKeyDown handler? Are those even mutually exclusive?
>> >>>
>> >>> My current thinking is: the symmetry of using the same key codes
>> >>> returned by onKeyDown is appealing. Also, even though there are some
>> >>> differences, the most common keys needed to automate the UI (tab,
>> >>> enter, arrows, alphanumerics) are already consistent across platforms.
>> >>>
>> >>> Thanks,
>> >>> - Dominic
>> >>>
>> >>> --
>> >>> Chromium Developers mailing list: chromium-dev@googlegroups.com
>> >>> View archives, change email options, or unsubscribe:
>> >>> http://groups.google.com/group/chromium-dev
>> >>>
>> >>
>> >
>>
>> --
>> Chromium Developers mailing list: chromium-dev@googlegroups.com
>> View archives, change email options, or unsubscribe:
>> http://groups.google.com/group/chromium-dev
>
>
--
Chromium Developers mailing list: chromium-dev@googlegroups.com
View archives, change email options, or unsubscribe:
http://groups.google.com/group/chromium-dev
