Hi Samuel, On Wed, 3 Apr 2019 at 13:57, Samuel Thibault <samuel.thiba...@ens-lyon.org> wrote: > > I would like to open a discussion on accessibility features on Wayland. > Several things that happened to work on Xorg (sometimes rather by luck) > do not work any more on Wayland, so we need to find proper ways to > implement them on Wayland. > > > The needs are mostly on the input side. To avoid an XY problem I have > gathered a list of the *end-use* cases on > > https://www.freedesktop.org/wiki/Accessibility/Input/ > > as well as ideas of how they could be implemented. > > > Basically, the technical requirements we seem to end up with is: > > - some support (such as AccessX) needs to capture some input events > and emit other input events. > > - some support (e.g. key feedback) only needs to get notification of > input events (can be asynchronous), both actual keypresses and > composition / input method results. > > - some support (e.g. virtual keyboard) only needs to be able to emit > input events. > > - some support (screen reader shortcuts) needs to catch pressure of a > set of keyboard shortcuts. > > > > Most of these were previously implemented on Xorg by just snooping input > with XEvIE and synthesizing with XTest, and when XEvIE was removed the > snooping was integrated into toolkits themselves. These interfaces are > frowned upon at least for security reasons, but also some performance > reasons. > > > Some of the features can be implemented in the compositor (e.g. AccessX > is already implemented that way), some of these however should probably > not (e.g. a screen reader). > > > What we currently have in Wayland is support for AccessX, directly > implemented in mutter. The rest of accessibility features are currently > mostly broken.
Well, instead of "mostly broken", I'd say it's a WIP on the GNOME side. To recap the state of the accessibility features in GNOME: * Keyboard accessibility implemented in mutter (you mentioned it) - That includes the AccessX features previously found in the Xserver. That also includes mousekeys. * Mouse accessibility features from GNOME mousetweaks (aka "Click Assist") is being worked on, see these MR: https://gitlab.gnome.org/GNOME/gnome-settings-daemon/merge_requests/95 https://gitlab.gnome.org/GNOME/mutter/merge_requests/512 https://gitlab.gnome.org/GNOME/gnome-shell/merge_requests/474 * locate-pointer is also being worked on (not sure locate-pointner qualifies as a real accessibility feature): https://gitlab.gnome.org/GNOME/gnome-settings-daemon/merge_requests/86 https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/merge_requests/19 https://gitlab.gnome.org/GNOME/gnome-tweaks/merge_requests/35 https://gitlab.gnome.org/GNOME/mutter/merge_requests/453 https://gitlab.gnome.org/GNOME/gnome-shell/merge_requests/413 * For at-spi on Wayland, there is no global coordinates and no plan to add them. So for being able to generate pointer events without global coordinates and keyboard events without XTest for a replacement of dogtail which would work with GNOME on Wayland, there is ponytail: https://gitlab.gnome.org/ofourdan/gnome-ponytail-daemon Maybe that could be useful for accessibility as well? > Thinking of the implementation of AccessX support, it is currently > only within mutter, and not in other compositors, which is no good > for having accessibility supported on all systems. That's part of the problem with Wayland, previously we had one single implementation of the display server which was Xorg and all window managers and desktops would inherit the same features provided by Xorg, whereas now every Wayland compositor is a display server. That's why we have helper libs such as libweston and wlroots for example. > We could think > of moving the implementation to a shared library that compositors > would use, with the same principle as libinput. Such library could > implement all accessibility features that a compositor should implement > itself. Yet, I think this is quite different from what libinput does. These features are very deeply rooted in the event loop of the compositor, in the case of mutter, this is clutter. Reason for this is because it need to be able to delay, replace or inject both pointer (mousekeys) and keyboard events (slowkeys, stickykeys, bouncekeys). It also interfere with xkb state for modifiers (stickykeys). I reckon moving the logic out of the compositor is certainly doable, but would not necessarily make the code much simpler for other compositors... > It could also provide interfaces for plugging accessibility > features implemented in separate processus (e.g. screen reader). To > avoid keyloggers and such, such interface needs to be available only to > trusted accessibility processes. That does not need to be Wayland protocols though, these could be DBUS. But I guess we need to define what a "trusted accessibility processes" is and how the compositor can enforce that. > About the screen reader shortcuts, one issue we have with the current > implementation is that for each keyboard event the toolkit has to wait > for the screen reader to tell whether it ate the event or not. That > makes all keyboard events get delayed. This should probably be replaced > by a registration mechanism: the screen reader tells which shortcuts it > is interested in, and only the events for them are captured and provided > to the screen reader, asynchronously. > > Opinions on the whole thing? If the screenreader issue is just about keyboard shortcuts, then a DBUS interface would be doable. Cheers, Olivier _______________________________________________ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
