Top Most application Rendered by weston
Hey, I am using weston as compositor for my platform. I am developing a feature in which I would want to know the top most application displayed by the compositor as multiple application can be opened at a single instance. Is there any variable or any function provided in weston by which I can know the top most application name rendered by the compositor. Regards Sameer Ranjan Research Engineer C-DOT Delhi ::Disclaimer:: The contents of this email and any attachment(s) are confidential and intended for the named recipient(s) only. It shall not attach any liability on C-DOT. Any views or opinions presented in this email are solely those of the author and may not necessarily reflect the opinions of C-DOT. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Keyboard issues w/Orca screen reader
Hi Mike!, On Wed, Feb 24, 2021 at 11:03 PM Mike Gorse wrote: > > As part of the GNOME project, we have Orca. It is a screen reader that reads > what an application is displaying out loud, or sends it to a Braille display. > It allows a blind user to interact with GNOME. There are currently several > issues with Orca on Wayland, since it (and AT-SPI, the accessibility API that > it uses) were originally designed for X11, without regard to the kind of > security constraints that Wayland has. > > Traditionally, Orca has used what amounts to a key filter implemented by > AT-SPI. The toolkit (gtk, qt, etc) would make a dbus method call to AT-SPI > when > a key is pressed, passing on the keycode, keysym, and text associated with the > key (Orca uses the latter to announce the key that was just pressed). The > AT-SPI registry daemon would make a method call to Orca, Orca would return a > boolean to indicate whether it consumed the key or not, and AT-SPI would > return > the result to the toolkit. This is no longer supported by gtk 4, so I am > trying > to solve things another way. There is a good amount of discussion at > https://gitlab.gnome.org/GNOME/gtk/-/issues/1739 > > Orca uses this key filter for a few things. It wants to announce the key that > the user just pressed. If the user presses the control key, then Orca will > stop > reading whatever it is reading. It also implements commands in this way (the > user can press a key to have it read the current line, to give one example). I may need to be reminded again... are there other commands that are contextual to the application? I mainly wonder about whether there's a reason to keep applications in the event emission chain. Otherwise, seems like part of this could be covered by a "description query" protocol. > For X11, I can have AT-SPI/Orca watch for XI_KeyPress events and call > XIGrabKeycode when Orca wants to grab a key, but I'm struggling to figure out > what to do on Wayland. I think that I would need to propose a new protocol or > two, but I foresee security being a challenge/concern, so I'm wondering if > anyone has any advice in particular. Xwayland-keyboard-grab looks like it > would > do part of what I need, if Orca ran as an xwayland client, but it wouldn't > help > in terms of passing key press notifications on to Orca for the purposes of key > echoing and interrupting speech, unless a grab was active for the particular > key. If I enable my X11 code for Wayland, then I don't see XI_KeyPress events > for keys going to other applications, which I think is expected given > Wayland's > security model. IIUC (and I'm sure I miss/forget things), there's 2 needs expressed here: 1. To make Orca part of the chain of command - Getting notified of all key events - Handle global keyboard shortcuts - And make others miss them (i.e. in a grabbing manner) 2. To make clients work seamlessly with Orca To fix 1, I tend to think this is something best left to an integration library (as IIUC the goal is also to maximize reutilization). The API sounds like it could be reasonably agnostic of compositor details. Fixing 2 sounds most definitely protocol land, which I wonder if it could be more semantic and less attached to input specifics. Cheers, Carlos ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Keyboard issues w/Orca screen reader
As part of the GNOME project, we have Orca. It is a screen reader that reads what an application is displaying out loud, or sends it to a Braille display. It allows a blind user to interact with GNOME. There are currently several issues with Orca on Wayland, since it (and AT-SPI, the accessibility API that it uses) were originally designed for X11, without regard to the kind of security constraints that Wayland has. Traditionally, Orca has used what amounts to a key filter implemented by AT-SPI. The toolkit (gtk, qt, etc) would make a dbus method call to AT-SPI when a key is pressed, passing on the keycode, keysym, and text associated with the key (Orca uses the latter to announce the key that was just pressed). The AT-SPI registry daemon would make a method call to Orca, Orca would return a boolean to indicate whether it consumed the key or not, and AT-SPI would return the result to the toolkit. This is no longer supported by gtk 4, so I am trying to solve things another way. There is a good amount of discussion at https://gitlab.gnome.org/GNOME/gtk/-/issues/1739 Orca uses this key filter for a few things. It wants to announce the key that the user just pressed. If the user presses the control key, then Orca will stop reading whatever it is reading. It also implements commands in this way (the user can press a key to have it read the current line, to give one example). For X11, I can have AT-SPI/Orca watch for XI_KeyPress events and call XIGrabKeycode when Orca wants to grab a key, but I'm struggling to figure out what to do on Wayland. I think that I would need to propose a new protocol or two, but I foresee security being a challenge/concern, so I'm wondering if anyone has any advice in particular. Xwayland-keyboard-grab looks like it would do part of what I need, if Orca ran as an xwayland client, but it wouldn't help in terms of passing key press notifications on to Orca for the purposes of key echoing and interrupting speech, unless a grab was active for the particular key. If I enable my X11 code for Wayland, then I don't see XI_KeyPress events for keys going to other applications, which I think is expected given Wayland's security model. Thanks, -Mike ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Keyboard issues w/Orca screen reader
On Wed, 24 Feb 2021 15:59:59 -0600 (CST) Mike Gorse said: > As part of the GNOME project, we have Orca. It is a screen reader that reads > what an application is displaying out loud, or sends it to a Braille display. > It allows a blind user to interact with GNOME. There are currently several > issues with Orca on Wayland, since it (and AT-SPI, the accessibility API that > it uses) were originally designed for X11, without regard to the kind of > security constraints that Wayland has. > > Traditionally, Orca has used what amounts to a key filter implemented by > AT-SPI. The toolkit (gtk, qt, etc) would make a dbus method call to AT-SPI > when a key is pressed, passing on the keycode, keysym, and text associated > with the key (Orca uses the latter to announce the key that was just > pressed). The AT-SPI registry daemon would make a method call to Orca, Orca > would return a boolean to indicate whether it consumed the key or not, and > AT-SPI would return the result to the toolkit. This is no longer supported by > gtk 4, so I am trying to solve things another way. There is a good amount of > discussion at https://gitlab.gnome.org/GNOME/gtk/-/issues/1739 In this case, your issue would be with gtk. It should still support this. In a wayland world, security-wise this would hand the decision to talk to orca via at-spi and if that app wants to leak input somewhere else - it is that app's decision. It can leak only a subset of keys too and selectively pass or don't pass depending on the situation (e.g. if entering text in a password field. the client can choose to not pass the key input to at-spi and instead maybe just pass some dummy key much like password entries display *** instead of the letters). The compositor does not need to be involved. This would IMHO be certainly the best way to do this. > Orca uses this key filter for a few things. It wants to announce the key that > the user just pressed. If the user presses the control key, then Orca will > stop reading whatever it is reading. It also implements commands in this way > (the user can press a key to have it read the current line, to give one > example). For X11, I can have AT-SPI/Orca watch for XI_KeyPress events and > call XIGrabKeycode when Orca wants to grab a key, but I'm struggling to > figure out what to do on Wayland. I think that I would need to propose a new > protocol or two, but I foresee security being a challenge/concern, so I'm > wondering if anyone has any advice in particular. Xwayland-keyboard-grab > looks like it would do part of what I need, if Orca ran as an xwayland > client, but it wouldn't help in terms of passing key press notifications on > to Orca for the purposes of key echoing and interrupting speech, unless a > grab was active for the particular key. If I enable my X11 code for Wayland, > then I don't see XI_KeyPress events for keys going to other applications, > which I think is expected given Wayland's security model. As above - the client should do this. Keep in mind that I also do not exclude the compositor itself from being an AT-SPI client. If the compositor wishes to pass its own keybindings/shortcuts selectively etc.. It may be a compositor has it's own "blind" mode where it may actually directly output some information to both speakers and a braille keyboard to send feedback/information to the user and thus not need orca or AT-SPI. > Thanks, > -Mike > > ___ > wayland-devel mailing list > wayland-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/wayland-devel > -- - Codito, ergo sum - "I code, therefore I am" -- Carsten Haitzler - ras...@rasterman.com ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Keyboard issues w/Orca screen reader
Simon Ser, le mer. 24 févr. 2021 23:07:46 +, a ecrit: > On Thursday, February 25th, 2021 at 12:03 AM, Samuel Thibault > wrote: > > > Simon Ser, le mer. 24 févr. 2021 22:57:02 +, a ecrit: > > > As prior art, I can think of wshowkeys [1] which displays which keys are > > > being > > > pressed on screen. It uses a helper suid process, opens > > > /dev/input/whatever, > > > drops privileges, and relays key presses to the main process. > > > > > > [1]: https://git.sr.ht/~sircmpwn/wshowkeys > > > > This would be very fragile: we do not know what keyboards wayland is > > actually reading. > > Wayland is very likely reading from all keyboards. > > Some cases where this doesn't hold: > > - The user has explicitly disabled a keyboard. > - The user has a multi-seat setup, and has keyboards assigned to > different logical seats. Yes, and we do want to support these cases seamlessly. Samuel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Keyboard issues w/Orca screen reader
Hello, Simon Ser, le mer. 24 févr. 2021 22:57:02 +, a ecrit: > As prior art, I can think of wshowkeys [1] which displays which keys are being > pressed on screen. It uses a helper suid process, opens /dev/input/whatever, > drops privileges, and relays key presses to the main process. > > [1]: https://git.sr.ht/~sircmpwn/wshowkeys This would be very fragile: we do not know what keyboards wayland is actually reading. Samuel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Keyboard issues w/Orca screen reader
On Thursday, February 25th, 2021 at 12:03 AM, Samuel Thibault wrote: > Simon Ser, le mer. 24 févr. 2021 22:57:02 +, a ecrit: > > As prior art, I can think of wshowkeys [1] which displays which keys are > > being > > pressed on screen. It uses a helper suid process, opens /dev/input/whatever, > > drops privileges, and relays key presses to the main process. > > > > [1]: https://git.sr.ht/~sircmpwn/wshowkeys > > This would be very fragile: we do not know what keyboards wayland is > actually reading. Wayland is very likely reading from all keyboards. Some cases where this doesn't hold: - The user has explicitly disabled a keyboard. - The user has a multi-seat setup, and has keyboards assigned to different logical seats. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Keyboard issues w/Orca screen reader
On Wednesday, February 24th, 2021 at 10:59 PM, Mike Gorse wrote: > As part of the GNOME project, we have Orca. Thanks for reaching out! It seems like Orca uses X11 keyboard grabs for two different things… > Orca uses this key filter for a few things. It wants to announce the key that > the user just pressed. The first one is announcing the keys. I don't think there's a way around having some kind of backdoor for this one. Orca needs to be a privileged process in order to listen to all key events. What "privileged" means isn't clear yet. As prior art, I can think of wshowkeys [1] which displays which keys are being pressed on screen. It uses a helper suid process, opens /dev/input/whatever, drops privileges, and relays key presses to the main process. [1]: https://git.sr.ht/~sircmpwn/wshowkeys > If the user presses the control key, then Orca will stop reading whatever it > is reading. It also implements commands in this way (the user can press a key > to have it read the current line, to give one example). This sounds like a separate requirement: global keybinds. In general these are configured directly in the compositor. With the status quo, a manual user action would be needed to set them up. There is a proposal to add a standardized way to setup compositor-wide keybindings [2]. Maybe you can have a look at it if you're interested? [2]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/56 ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: absolute positioning and other "missing features" of Wayland
Thanks everyone! This was quite a bit of interesting feedback. Let me add Alexandros to this thread, too, in case he has any input - if the existing hacks seem to be sufficient, or if it makes sense to create a new protocol. (Alexandros: I'm by no means an expert here, but saw your work on Wine, and thought it would be useful to start a discussion. If you're not already on the mailing list, see the full thread here: https://lists.freedesktop.org/archives/wayland-devel/2021-February/041718.html ) On Mon, Feb 22, 2021 at 9:23 AM Carsten Haitzler wrote: > On Mon, 22 Feb 2021 12:10:08 +0100 Jonas Ådahl said: > > > On Mon, Feb 22, 2021 at 10:49:33AM +, Simon Ser wrote: > > > On Monday, February 22nd, 2021 at 11:44 AM, Carsten Haitzler > > > wrote: > > > > > > > I also would want to avoid baking explicit absolute positioning into > > > > wayland protocol - be it as a core agreed to add-on to xdg-shell or > even > > > > a "commonly supported extension". What I'd like it some better > solution. > > > > For example - if an app wants to absolute position a window because > it's > > > > doing a custom "my own notification popup" much like Chrome and > Firefox > > > > now like to do in X11, then I'd prefer this be explicitly exposed as > a > > > > "notification surface with requested screen location hints" and so > the > > > > compositor can decide to do something more intelligent with it. I am > sure > > > > this list of use cases will probably be extensive and also depend on > the > > > > API in wine that is being wrapped and intercepted - the higher level > it > > > > is, the more it knows about the intended use case. > > > > > > I'd like to avoid making this general, if we go down this road. Make it > > > specific to the win32 API and wine. Wayland-native toolkits don't need > > > it. > > > > Sounds potentially not horrible in theory, but is it remotely possible? > > There are these approaches I can think of: > > It's still open to abuse eventually for the wrong reasons no matter where > it > is. > > > * Add a "wl_win32" to wayland-protocols > > > > Adoption by anyone wanting to bypass the Wayland windowing model can use > > it trivially, and we have a X11 like situation where everything grabs > > everything and arbitrary client driven window self management. > > > > * Make "wine" distribute a "wl_win32.xml" and make compositors depend on > >on wine-devel, implementing the protocol. > > > > Practically the same as above, just a 'cp ~/wine/protocols/wl-win32 > > protocols' away, and we're back with wierd grabs and client managing > > their own surfaces again. > > > > * A libwine-wayland, a wineBindWaylandDisplay() with some kind of half > >assed authentication. > > > > I assume noone wants this (including me). But client developers would > > have to be really obnoxious to work around it which might be enough to > > not make it general. > > > > A "allow/deny" nag/query (aka "Do you want this application you > > installed to work or want me to break it for you?") I wouldn't call a > > reasonable option either. > > I think the nag is probably the way to go. Or specifically the compositor > should ask once and remember the response (or have an option to never ask > and > always allow this if users so desire). > > > So, how would we make it not de-facto general? > > > > Not to meantion the head ache of letting clients in on configuring their > > window positions when any kind of HiDPI scaling, fractional or nat, is > > done. > > Indeed this is the problem as well as odd displays like circular ones as > well. > It opens the doors to clients making assumptions that may not be true given > various methods a compositor might use to handle multiple screens with > differing DPI/scaling etc. > > > > > > > Notifications popups are part of the desktop environment and clients > > > shouldn't try to display them directly. > > But they do. Give them an inch and they take a mile. :) > > > Agreed. > > > > > > Jonas > > > > > -- > - Codito, ergo sum - "I code, therefore I am" -- > Carsten Haitzler - ras...@rasterman.com > > ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
