On Wed, 24 Feb 2021 15:59:59 -0600 (CST) Mike Gorse <mgo...@suse.com> said:
> As part of the GNOME project, we have Orca. It is a screen reader that reads > what an application is displaying out loud, or sends it to a Braille display. > It allows a blind user to interact with GNOME. There are currently several > issues with Orca on Wayland, since it (and AT-SPI, the accessibility API that > it uses) were originally designed for X11, without regard to the kind of > security constraints that Wayland has. > > Traditionally, Orca has used what amounts to a key filter implemented by > AT-SPI. The toolkit (gtk, qt, etc) would make a dbus method call to AT-SPI > when a key is pressed, passing on the keycode, keysym, and text associated > with the key (Orca uses the latter to announce the key that was just > pressed). The AT-SPI registry daemon would make a method call to Orca, Orca > would return a boolean to indicate whether it consumed the key or not, and > AT-SPI would return the result to the toolkit. This is no longer supported by > gtk 4, so I am trying to solve things another way. There is a good amount of > discussion at https://gitlab.gnome.org/GNOME/gtk/-/issues/1739 In this case, your issue would be with gtk. It should still support this. In a wayland world, security-wise this would hand the decision to talk to orca via at-spi and if that app wants to leak input somewhere else - it is that app's decision. It can leak only a subset of keys too and selectively pass or don't pass depending on the situation (e.g. if entering text in a password field. the client can choose to not pass the key input to at-spi and instead maybe just pass some dummy key much like password entries display *** instead of the letters). The compositor does not need to be involved. This would IMHO be certainly the best way to do this. > Orca uses this key filter for a few things. It wants to announce the key that > the user just pressed. If the user presses the control key, then Orca will > stop reading whatever it is reading. It also implements commands in this way > (the user can press a key to have it read the current line, to give one > example). For X11, I can have AT-SPI/Orca watch for XI_KeyPress events and > call XIGrabKeycode when Orca wants to grab a key, but I'm struggling to > figure out what to do on Wayland. I think that I would need to propose a new > protocol or two, but I foresee security being a challenge/concern, so I'm > wondering if anyone has any advice in particular. Xwayland-keyboard-grab > looks like it would do part of what I need, if Orca ran as an xwayland > client, but it wouldn't help in terms of passing key press notifications on > to Orca for the purposes of key echoing and interrupting speech, unless a > grab was active for the particular key. If I enable my X11 code for Wayland, > then I don't see XI_KeyPress events for keys going to other applications, > which I think is expected given Wayland's security model. As above - the client should do this. Keep in mind that I also do not exclude the compositor itself from being an AT-SPI client. If the compositor wishes to pass its own keybindings/shortcuts selectively etc.. It may be a compositor has it's own "blind" mode where it may actually directly output some information to both speakers and a braille keyboard to send feedback/information to the user and thus not need orca or AT-SPI. > Thanks, > -Mike > > _______________________________________________ > wayland-devel mailing list > wayland-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/wayland-devel > -- ------------- Codito, ergo sum - "I code, therefore I am" -------------- Carsten Haitzler - ras...@rasterman.com _______________________________________________ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
