Re: How to implement OSD overlay in Wayland/Weston
Nov 19, 2019 3:39:48 PM Simon Ser : > On Tuesday, November 19, 2019 1:37 PM, Pekka Paalanen wrote: > > >> On Tue, 19 Nov 2019 12:19:35 + >> Simon Ser cont...@emersion.fr wrote: >> >> >>> On Tuesday, November 19, 2019 1:08 PM, Pekka Paalanen ppaala...@gmail.com >>> wrote: >>> >>> On Tue, 19 Nov 2019 12:55:31 +0100 Guillermo Rodriguez guillerodriguez@gmail.com wrote: > Hi, > El mar., 19 nov. 2019 a las 12:38, Sebastian Krzyszkowiak > (d...@dosowisko.net) escribió: > > >> On 11/19/19, Pekka Paalanen ppaala...@gmail.com wrote: >> >> >>> yes. I'm not aware of any Wayland extension that would allow clients to >>> dictate always-on-top behaviour on the desktop. It has an obvious >>> attack vector (malicious or misbehaving app setting always-on-top >>> without user's consent) and a simple corner case (how do you stack >>> multiple always-on-top windows). >>> OSD as a concept seems like a desktop component, not a normal app. >>> Hence I would expect such a client to use a domain-specific or custom >>> protocol extension to set always-on-top. If your use case is not a >>> generic desktop but some special environment, then a domain-specific >>> protocol extension would be a way to go. For generic desktop >>> environments you'd integrate with the particular DE specifically, e.g. >>> a gnome-shell plugin or something. >>> That is, if the OSD really is an independent separate program and not a >>> part of some fullscreen application. >>> >> >> Sounds like something layer-shell protocol solves portably already. >> > > That sounds interesting. Is that supported by Weston? > No. Not yet at least. >>> >>> layer-shell's goal is to provide a way to create portable desktop >>> components, so it indeed seems like it's a nice fit for this use-case. >>> Note that for security reasons access to layer-shell could be >>> restricted, so layer-shell shouldn't be used by regular clients. >>> >> >> Yeah, if it can be implemented without messing up the normal shell >> functions or needing consideration in shell plugins, that might be even >> upstreamable into Weston. >> > > That would be nice. CC'ing myfreeweb. I haven't used Weston in almost a year now, due to discovering Wayfire :) I wrote an implementation of layer-shell for Weston back then, but it's 1) not really compliant, only sort of working 2) written in C++ so not upstreamable. But feel free to look at it, might be useful as a "how to do this in Weston" example. https://github.com/myfreeweb/weston-extra-dip/blob/master/layer-shell.cpp On the positive side, it was integrated with my capability based access control plugin (in the same repo), so maybe that's some extra inspiration. Basically my idea of how access control should work is that sets of capabilities are attached to clients, and the "desktop environment" process launched by the compositor gets all capabilities, and it can spawn clients (sockets) that have any subset of its capabilities. You could get by without the spawn-with-subset thing of course, but I think it's cool because e.g. the DE's launcher now gets to decide which apps can get privileged access, and it can see that some system-installed app is a special widget that the user wants to overlay, and decide to spawn it on a socket with the layer-shell overlay capability. Also an interesting observation: capabilities can be very granular, you can only require the capability for the overlay layer (probably top too) but not bottom, so desktop widgets like conky would not require privileges. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
Hello, El mar., 19 nov. 2019 a las 13:28, Pekka Paalanen () escribió: > > On Tue, 19 Nov 2019 12:48:10 +0100 > Guillermo Rodriguez wrote: > > > El mar., 19 nov. 2019 a las 11:47, Pekka Paalanen > > () escribió: > > > > > > On Tue, 19 Nov 2019 09:53:39 +0100 > > > Guillermo Rodriguez wrote: > > > > > > > Hi all, > > > > > > > > This is a follow up to my earlier email "Let pointer events pass > > > > through". > > > > > > > > I have an use case where there is one full screen application (I'll > > > > call this the "main" application) and I need to show a OSD overlay on > > > > top of it. The OSD overlay is actually based on GStreamer (think PiP) > > > > and is managed by a different process. > > > > The OSD overlay should not handle pointer events; these should pass > > > > through to the main application, which knows how to handle them. > > > > > > > > I am not sure how to approach this in Wayland/Weston. I have > > > > considered the following: > > > > > > > > 1. > > > > Use wl_surface::set_input_region so that pointer events are not > > > > delivered to the OSD overlay, but instead pass through and are handled > > > > by the main application. > > > > This "almost" works, however when the main application receives the > > > > event, Weston brings it on top, thus hiding the OSD overlay. > > > > I searched for a way to specify that a given surface should always > > > > stay "on top". This does not seem to be supported in Wayland. > > ... > > > The OSD is conceptually part of the fullscreen application but due to > > several reasons it is implemented as a separate process with a CLI > > that controls the OSD contents and PiP function; the main application > > communicates with the OSD process by writing commands to the OSD > > process' stdin, and reading responses from the OSD process' stdout. > > > > There are several reasons for this approach, but the main one is that > > if there are any issues in the GStreamer side this should not crash > > the whole application. With the current approach, the main application > > can just respawn the OSD process if it dies. > > I see. Then conceptually, you more or less need a custom window manager > that knows to handle your OSD specially. This is exactly the pluggable > (shell) modules that Weston supports. More on that below. > > > OK, so it seems that 2 and 3 are going to be difficult due to > > architectural reasons. Let's go back to 1. > > Mind that using separate independent processes makes it nearly > impossible to synchronize their content updates. This would be an issue > if you need to move or resize the OSD in sync with the main window > contents. Yes. But that's not a problem in my case since neither the OSD or the main application can be moved or resized. > > > I understand that my main problem is controlling the Z-order of > > top-level surfaces. If that is not possible at the application level, > > then I guess I need to mess around with Weston. > > > > From what I see > > (https://vignatti.com/2013/03/05/ui-customization-on-wayland/), Weston > > supports the concept of plugabble shells that are in charge of > > defining "how surfaces will be mapped on the screen". I guess then > > that I would need to write a custom shell controlling Z-order of the > > surfaces. Does this make sense? > > > > I have been looking for documentation on this (the API for custom > > Weston shells), but I cannot find anything. Can you point me in the > > right direction? > > Unfortunately it's not documented even still, yet. > > One approach is to fork Weston's desktop-shell/. But since your use > case is quite simple, instead of replacing the whole shell plugin, you > could write a new add-on plugin that implements only the new behaviour > you need. I see. So I can extend the capabilities of Weston's desktop-shell by adding plugins that implement protocol extensions. I wasn't aware of this possibility; this should be much easier than writing a custom shell (or forking an existing one). > > You need a little protocol extension that will allow a client to send a > request that "gives a wl_surface the role of an OSD" and any parameters > you want the client to be in control of. Alternatively the plugin might > implement wlroot's layer-shell interfaces. Since layer-shell seems to perfectly fit my use case, I assume that implementing this is better than rolling my own protocol extension. > On initialization, the > plugin creates a wl_global for the protocol extension and handles the > OSD-specific requests through that. > > Examples of such custom wl_surface roles are weston_desktop.set_panel > and weston_desktop.set_background requests in Weston's > protocol/weston-desktop-shell.xml. The server implementation of these > can be found in desktop-shell/shell.c. > > Since it's always-on-top, you create a new weston_layer with a position > around WESTON_LAYER_POSITION_TOP_UI or so. When the OSD wl_surface > needs to be mapped, the weston_view of it is set into that >
Re: How to implement OSD overlay in Wayland/Weston
Hi Daniel, El mar., 19 nov. 2019 a las 13:44, Daniel Stone () escribió: > > Hi, > > On Tue, 19 Nov 2019 at 11:48, Guillermo Rodriguez > wrote: > > El mar., 19 nov. 2019 a las 11:47, Pekka Paalanen () > > escribió: > > > If the OSD is just a piece of a bigger application and only needs to be > > > on top of that application's windows, you could probably use xdg > > > extensions or a sub-surface to realize it. This does mean that the OSD > > > component must be in-process with the main app and share the same > > > wl_display instance. > > > > The OSD is conceptually part of the fullscreen application but due to > > several reasons it is implemented as a separate process with a CLI > > that controls the OSD contents and PiP function; the main application > > communicates with the OSD process by writing commands to the OSD > > process' stdin, and reading responses from the OSD process' stdout. > > > > There are several reasons for this approach, but the main one is that > > if there are any issues in the GStreamer side this should not crash > > the whole application. With the current approach, the main application > > can just respawn the OSD process if it dies. > > > > > I think this should be possible also with GStreamer. > > > > Not in the current implementation of the waylandsink plugin -- it > > always creates its own top level window > > (https://gstreamer.freedesktop.org/documentation/waylandsink/index.html?gi-language=c), > > and does not support drawing to an existing surface. But this could be > > patched if needed. > > It is completely possible, using the GstVideoOverlay API. See > gst-plugins-bad/tests/examples/waylandsink/main.c for an example of > providing your own wl_surface. Nice. I assumed this was not possible as the documentation I linked explicitly states that the waylandsink creates its own window. This will definitely make my life easier :-) Thanks for the pointer, Guillermo Rodriguez ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
Hi, On Tue, 19 Nov 2019 at 11:48, Guillermo Rodriguez wrote: > El mar., 19 nov. 2019 a las 11:47, Pekka Paalanen () > escribió: > > If the OSD is just a piece of a bigger application and only needs to be > > on top of that application's windows, you could probably use xdg > > extensions or a sub-surface to realize it. This does mean that the OSD > > component must be in-process with the main app and share the same > > wl_display instance. > > The OSD is conceptually part of the fullscreen application but due to > several reasons it is implemented as a separate process with a CLI > that controls the OSD contents and PiP function; the main application > communicates with the OSD process by writing commands to the OSD > process' stdin, and reading responses from the OSD process' stdout. > > There are several reasons for this approach, but the main one is that > if there are any issues in the GStreamer side this should not crash > the whole application. With the current approach, the main application > can just respawn the OSD process if it dies. > > > I think this should be possible also with GStreamer. > > Not in the current implementation of the waylandsink plugin -- it > always creates its own top level window > (https://gstreamer.freedesktop.org/documentation/waylandsink/index.html?gi-language=c), > and does not support drawing to an existing surface. But this could be > patched if needed. It is completely possible, using the GstVideoOverlay API. See gst-plugins-bad/tests/examples/waylandsink/main.c for an example of providing your own wl_surface. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
On Tuesday, November 19, 2019 1:37 PM, Pekka Paalanen wrote: > On Tue, 19 Nov 2019 12:19:35 + > Simon Ser cont...@emersion.fr wrote: > > > On Tuesday, November 19, 2019 1:08 PM, Pekka Paalanen ppaala...@gmail.com > > wrote: > > > > > On Tue, 19 Nov 2019 12:55:31 +0100 > > > Guillermo Rodriguez guillerodriguez@gmail.com wrote: > > > > > > > Hi, > > > > El mar., 19 nov. 2019 a las 12:38, Sebastian Krzyszkowiak > > > > (d...@dosowisko.net) escribió: > > > > > > > > > On 11/19/19, Pekka Paalanen ppaala...@gmail.com wrote: > > > > > > > > > > > yes. I'm not aware of any Wayland extension that would allow > > > > > > clients to > > > > > > dictate always-on-top behaviour on the desktop. It has an obvious > > > > > > attack vector (malicious or misbehaving app setting always-on-top > > > > > > without user's consent) and a simple corner case (how do you stack > > > > > > multiple always-on-top windows). > > > > > > OSD as a concept seems like a desktop component, not a normal app. > > > > > > Hence I would expect such a client to use a domain-specific or > > > > > > custom > > > > > > protocol extension to set always-on-top. If your use case is not a > > > > > > generic desktop but some special environment, then a domain-specific > > > > > > protocol extension would be a way to go. For generic desktop > > > > > > environments you'd integrate with the particular DE specifically, > > > > > > e.g. > > > > > > a gnome-shell plugin or something. > > > > > > That is, if the OSD really is an independent separate program and > > > > > > not a > > > > > > part of some fullscreen application. > > > > > > > > > > Sounds like something layer-shell protocol solves portably already. > > > > > > > > That sounds interesting. Is that supported by Weston? > > > > > > No. Not yet at least. > > > > layer-shell's goal is to provide a way to create portable desktop > > components, so it indeed seems like it's a nice fit for this use-case. > > Note that for security reasons access to layer-shell could be > > restricted, so layer-shell shouldn't be used by regular clients. > > Yeah, if it can be implemented without messing up the normal shell > functions or needing consideration in shell plugins, that might be even > upstreamable into Weston. That would be nice. CC'ing myfreeweb. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
On Tue, 19 Nov 2019 12:19:35 + Simon Ser wrote: > On Tuesday, November 19, 2019 1:08 PM, Pekka Paalanen > wrote: > > > On Tue, 19 Nov 2019 12:55:31 +0100 > > Guillermo Rodriguez guillerodriguez@gmail.com wrote: > > > > > Hi, > > > El mar., 19 nov. 2019 a las 12:38, Sebastian Krzyszkowiak > > > (d...@dosowisko.net) escribió: > > > > > > > On 11/19/19, Pekka Paalanen ppaala...@gmail.com wrote: > > > > > > > > > yes. I'm not aware of any Wayland extension that would allow clients > > > > > to > > > > > dictate always-on-top behaviour on the desktop. It has an obvious > > > > > attack vector (malicious or misbehaving app setting always-on-top > > > > > without user's consent) and a simple corner case (how do you stack > > > > > multiple always-on-top windows). > > > > > OSD as a concept seems like a desktop component, not a normal app. > > > > > Hence I would expect such a client to use a domain-specific or custom > > > > > protocol extension to set always-on-top. If your use case is not a > > > > > generic desktop but some special environment, then a domain-specific > > > > > protocol extension would be a way to go. For generic desktop > > > > > environments you'd integrate with the particular DE specifically, e.g. > > > > > a gnome-shell plugin or something. > > > > > That is, if the OSD really is an independent separate program and not > > > > > a > > > > > part of some fullscreen application. > > > > > > > > Sounds like something layer-shell protocol solves portably already. > > > > > > That sounds interesting. Is that supported by Weston? > > > > No. Not yet at least. > > layer-shell's goal is to provide a way to create portable desktop > components, so it indeed seems like it's a nice fit for this use-case. > Note that for security reasons access to layer-shell could be > restricted, so layer-shell shouldn't be used by regular clients. Yeah, if it can be implemented without messing up the normal shell functions or needing consideration in shell plugins, that might be even upstreamable into Weston. Thanks, pq > Regarding Weston support, there's [1] which seems to implement part of > layer-shell. I don't know how up-to-date and complete it is. > > [1]: https://github.com/myfreeweb/weston-extra-dip pgpENnSjSVVnI.pgp Description: OpenPGP digital signature ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
On Tue, 19 Nov 2019 12:48:10 +0100 Guillermo Rodriguez wrote: > El mar., 19 nov. 2019 a las 11:47, Pekka Paalanen > () escribió: > > > > On Tue, 19 Nov 2019 09:53:39 +0100 > > Guillermo Rodriguez wrote: > > > > > Hi all, > > > > > > This is a follow up to my earlier email "Let pointer events pass through". > > > > > > I have an use case where there is one full screen application (I'll > > > call this the "main" application) and I need to show a OSD overlay on > > > top of it. The OSD overlay is actually based on GStreamer (think PiP) > > > and is managed by a different process. > > > The OSD overlay should not handle pointer events; these should pass > > > through to the main application, which knows how to handle them. > > > > > > I am not sure how to approach this in Wayland/Weston. I have > > > considered the following: > > > > > > 1. > > > Use wl_surface::set_input_region so that pointer events are not > > > delivered to the OSD overlay, but instead pass through and are handled > > > by the main application. > > > This "almost" works, however when the main application receives the > > > event, Weston brings it on top, thus hiding the OSD overlay. > > > I searched for a way to specify that a given surface should always > > > stay "on top". This does not seem to be supported in Wayland. ... > The OSD is conceptually part of the fullscreen application but due to > several reasons it is implemented as a separate process with a CLI > that controls the OSD contents and PiP function; the main application > communicates with the OSD process by writing commands to the OSD > process' stdin, and reading responses from the OSD process' stdout. > > There are several reasons for this approach, but the main one is that > if there are any issues in the GStreamer side this should not crash > the whole application. With the current approach, the main application > can just respawn the OSD process if it dies. I see. Then conceptually, you more or less need a custom window manager that knows to handle your OSD specially. This is exactly the pluggable (shell) modules that Weston supports. More on that below. > OK, so it seems that 2 and 3 are going to be difficult due to > architectural reasons. Let's go back to 1. Mind that using separate independent processes makes it nearly impossible to synchronize their content updates. This would be an issue if you need to move or resize the OSD in sync with the main window contents. > I understand that my main problem is controlling the Z-order of > top-level surfaces. If that is not possible at the application level, > then I guess I need to mess around with Weston. > > From what I see > (https://vignatti.com/2013/03/05/ui-customization-on-wayland/), Weston > supports the concept of plugabble shells that are in charge of > defining "how surfaces will be mapped on the screen". I guess then > that I would need to write a custom shell controlling Z-order of the > surfaces. Does this make sense? > > I have been looking for documentation on this (the API for custom > Weston shells), but I cannot find anything. Can you point me in the > right direction? Unfortunately it's not documented even still, yet. One approach is to fork Weston's desktop-shell/. But since your use case is quite simple, instead of replacing the whole shell plugin, you could write a new add-on plugin that implements only the new behaviour you need. You need a little protocol extension that will allow a client to send a request that "gives a wl_surface the role of an OSD" and any parameters you want the client to be in control of. Alternatively the plugin might implement wlroot's layer-shell interfaces. On initialization, the plugin creates a wl_global for the protocol extension and handles the OSD-specific requests through that. Examples of such custom wl_surface roles are weston_desktop.set_panel and weston_desktop.set_background requests in Weston's protocol/weston-desktop-shell.xml. The server implementation of these can be found in desktop-shell/shell.c. Since it's always-on-top, you create a new weston_layer with a position around WESTON_LAYER_POSITION_TOP_UI or so. When the OSD wl_surface needs to be mapped, the weston_view of it is set into that weston_layer. That should be all, on a high level idea. The catch here is, that you would need to get GStreamer use the OSD role instead of the top-level window role for the wl_surface. I believe that should be well possible in a couple of ways even: you could write an app-sink element, or, I do believe that waylandsink should allow using an existing wl_display and a sub-surface... or some Wayland-enabled sink should. I'm not too familiar with Gst but I'm sure it's doable without too much hassle somehow. Doing the above, you are essentially making the OSD a "desktop component", so to speak. Thanks, pq pgp5GJiEINLiC.pgp Description: OpenPGP digital signature ___ wayland-devel mailing list
Re: How to implement OSD overlay in Wayland/Weston
On Tuesday, November 19, 2019 1:08 PM, Pekka Paalanen wrote: > On Tue, 19 Nov 2019 12:55:31 +0100 > Guillermo Rodriguez guillerodriguez@gmail.com wrote: > > > Hi, > > El mar., 19 nov. 2019 a las 12:38, Sebastian Krzyszkowiak > > (d...@dosowisko.net) escribió: > > > > > On 11/19/19, Pekka Paalanen ppaala...@gmail.com wrote: > > > > > > > yes. I'm not aware of any Wayland extension that would allow clients to > > > > dictate always-on-top behaviour on the desktop. It has an obvious > > > > attack vector (malicious or misbehaving app setting always-on-top > > > > without user's consent) and a simple corner case (how do you stack > > > > multiple always-on-top windows). > > > > OSD as a concept seems like a desktop component, not a normal app. > > > > Hence I would expect such a client to use a domain-specific or custom > > > > protocol extension to set always-on-top. If your use case is not a > > > > generic desktop but some special environment, then a domain-specific > > > > protocol extension would be a way to go. For generic desktop > > > > environments you'd integrate with the particular DE specifically, e.g. > > > > a gnome-shell plugin or something. > > > > That is, if the OSD really is an independent separate program and not a > > > > part of some fullscreen application. > > > > > > Sounds like something layer-shell protocol solves portably already. > > > > That sounds interesting. Is that supported by Weston? > > No. Not yet at least. layer-shell's goal is to provide a way to create portable desktop components, so it indeed seems like it's a nice fit for this use-case. Note that for security reasons access to layer-shell could be restricted, so layer-shell shouldn't be used by regular clients. Regarding Weston support, there's [1] which seems to implement part of layer-shell. I don't know how up-to-date and complete it is. [1]: https://github.com/myfreeweb/weston-extra-dip ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
On Tue, 19 Nov 2019 12:55:31 +0100 Guillermo Rodriguez wrote: > Hi, > > > El mar., 19 nov. 2019 a las 12:38, Sebastian Krzyszkowiak > () escribió: > > > > On 11/19/19, Pekka Paalanen wrote: > > > yes. I'm not aware of any Wayland extension that would allow clients to > > > dictate always-on-top behaviour on the desktop. It has an obvious > > > attack vector (malicious or misbehaving app setting always-on-top > > > without user's consent) and a simple corner case (how do you stack > > > multiple always-on-top windows). > > > > > > OSD as a concept seems like a desktop component, not a normal app. > > > Hence I would expect such a client to use a domain-specific or custom > > > protocol extension to set always-on-top. If your use case is not a > > > generic desktop but some special environment, then a domain-specific > > > protocol extension would be a way to go. For generic desktop > > > environments you'd integrate with the particular DE specifically, e.g. > > > a gnome-shell plugin or something. > > > > > > That is, if the OSD really is an independent separate program and not a > > > part of some fullscreen application. > > > > Sounds like something layer-shell protocol solves portably already. > > That sounds interesting. Is that supported by Weston? No. Not yet at least. Thanks, pq pgpdYgMEvUiT0.pgp Description: OpenPGP digital signature ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
Hi, El mar., 19 nov. 2019 a las 12:38, Sebastian Krzyszkowiak () escribió: > > On 11/19/19, Pekka Paalanen wrote: > > yes. I'm not aware of any Wayland extension that would allow clients to > > dictate always-on-top behaviour on the desktop. It has an obvious > > attack vector (malicious or misbehaving app setting always-on-top > > without user's consent) and a simple corner case (how do you stack > > multiple always-on-top windows). > > > > OSD as a concept seems like a desktop component, not a normal app. > > Hence I would expect such a client to use a domain-specific or custom > > protocol extension to set always-on-top. If your use case is not a > > generic desktop but some special environment, then a domain-specific > > protocol extension would be a way to go. For generic desktop > > environments you'd integrate with the particular DE specifically, e.g. > > a gnome-shell plugin or something. > > > > That is, if the OSD really is an independent separate program and not a > > part of some fullscreen application. > > Sounds like something layer-shell protocol solves portably already. That sounds interesting. Is that supported by Weston? Guillermo ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
El mar., 19 nov. 2019 a las 11:47, Pekka Paalanen () escribió: > > On Tue, 19 Nov 2019 09:53:39 +0100 > Guillermo Rodriguez wrote: > > > Hi all, > > > > This is a follow up to my earlier email "Let pointer events pass through". > > > > I have an use case where there is one full screen application (I'll > > call this the "main" application) and I need to show a OSD overlay on > > top of it. The OSD overlay is actually based on GStreamer (think PiP) > > and is managed by a different process. > > The OSD overlay should not handle pointer events; these should pass > > through to the main application, which knows how to handle them. > > > > I am not sure how to approach this in Wayland/Weston. I have > > considered the following: > > > > 1. > > Use wl_surface::set_input_region so that pointer events are not > > delivered to the OSD overlay, but instead pass through and are handled > > by the main application. > > This "almost" works, however when the main application receives the > > event, Weston brings it on top, thus hiding the OSD overlay. > > I searched for a way to specify that a given surface should always > > stay "on top". This does not seem to be supported in Wayland. > > Hi, > > yes. I'm not aware of any Wayland extension that would allow clients to > dictate always-on-top behaviour on the desktop. It has an obvious > attack vector (malicious or misbehaving app setting always-on-top > without user's consent) and a simple corner case (how do you stack > multiple always-on-top windows). > > OSD as a concept seems like a desktop component, not a normal app. > Hence I would expect such a client to use a domain-specific or custom > protocol extension to set always-on-top. If your use case is not a > generic desktop but some special environment, then a domain-specific It is actually a special environment; a single fullscreen app on an embedded system. > protocol extension would be a way to go. For generic desktop > environments you'd integrate with the particular DE specifically, e.g. > a gnome-shell plugin or something. > > That is, if the OSD really is an independent separate program and not a > part of some fullscreen application. > > If the OSD is just a piece of a bigger application and only needs to be > on top of that application's windows, you could probably use xdg > extensions or a sub-surface to realize it. This does mean that the OSD > component must be in-process with the main app and share the same > wl_display instance. The OSD is conceptually part of the fullscreen application but due to several reasons it is implemented as a separate process with a CLI that controls the OSD contents and PiP function; the main application communicates with the OSD process by writing commands to the OSD process' stdin, and reading responses from the OSD process' stdout. There are several reasons for this approach, but the main one is that if there are any issues in the GStreamer side this should not crash the whole application. With the current approach, the main application can just respawn the OSD process if it dies. > I think this should be possible also with GStreamer. Not in the current implementation of the waylandsink plugin -- it always creates its own top level window (https://gstreamer.freedesktop.org/documentation/waylandsink/index.html?gi-language=c), and does not support drawing to an existing surface. But this could be patched if needed. > > > 2. > > Setup things so that the OSD overlay is not an additional top level > > surface, but a child surface owned by the main application. > > This would mean having the main application create a child surface for > > the OSD, and pass this surface to the OSD process. > > In X this could be achieved by passing a Windows ID. In Wayland > > apparently it is not possible to share surfaces between processes. > > I'd avoid a design with two separate processes. As Ilia wrote, > xdg-foreign exists, but I'm not sure which compositors implement it. > > In Wayland, all protocol objects are private to the Wayland connection > (wl_display) where they were created. xdg-foreign is a way to work > around that, but I'd leave xdg-foreign only to cases that really need > to have separate processes for e.g. security reasons. > > > 3. > > Let the OSD overlay actually handle pointer events, but "send" them to > > the main application. > > In X this could be achieved with XSendEvent. In Wayland apparently it > > is not possible to have one process send events to another process. > > Once an input event has been delivered to a client, there is no way to > send it back or to another client in Wayland. Input event routing is > one-way. > > In Wayland, each client is isolated by default, and we try hard to keep > things that way so there is no spying or injection. OK, so it seems that 2 and 3 are going to be difficult due to architectural reasons. Let's go back to 1. I understand that my main problem is controlling the Z-order of top-level surfaces. If that is not
Re: How to implement OSD overlay in Wayland/Weston
On 11/19/19, Pekka Paalanen wrote: > yes. I'm not aware of any Wayland extension that would allow clients to > dictate always-on-top behaviour on the desktop. It has an obvious > attack vector (malicious or misbehaving app setting always-on-top > without user's consent) and a simple corner case (how do you stack > multiple always-on-top windows). > > OSD as a concept seems like a desktop component, not a normal app. > Hence I would expect such a client to use a domain-specific or custom > protocol extension to set always-on-top. If your use case is not a > generic desktop but some special environment, then a domain-specific > protocol extension would be a way to go. For generic desktop > environments you'd integrate with the particular DE specifically, e.g. > a gnome-shell plugin or something. > > That is, if the OSD really is an independent separate program and not a > part of some fullscreen application. Sounds like something layer-shell protocol solves portably already. SK ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
You can try to use a compositor which supports xdg-foreign, except if you don't already use private weston protocols. In this case, you can just extend weston with a custom protocol to get the placement you want to have. On Tue, 19 Nov 2019, 11:42 Guillermo Rodriguez, < guillerodriguez@gmail.com> wrote: > Hello, > > Indeed I looked into xdg-foreign but it does not seem to be supported in > Weston. > > Other ideas? > > Guillermo > > El mar., 19 nov. 2019 a las 10:21, Ilia Bozhinov () > escribió: > > > > Hi, > > > > I am not entirely familiar with your setup, but you can try to use a > combination of: > > > > 1. Setting your on-top window's input region to empty as you already > have tried. > > 2. Use xdg-foreign protocol to set the main window as parent of your > on-top window. > >> > > This will work on all compositors that support xdg-foreign, however I am > not sure whether weston supports it. > > > > Regards, > > Ilia Bozhinov > > > > On Tue, 19 Nov 2019, 09:53 Guillermo Rodriguez, < > guillerodriguez@gmail.com> wrote: > >> > >> Hi all, > >> > >> This is a follow up to my earlier email "Let pointer events pass > through". > >> > >> I have an use case where there is one full screen application (I'll > >> call this the "main" application) and I need to show a OSD overlay on > >> top of it. The OSD overlay is actually based on GStreamer (think PiP) > >> and is managed by a different process. > >> The OSD overlay should not handle pointer events; these should pass > >> through to the main application, which knows how to handle them. > >> > >> I am not sure how to approach this in Wayland/Weston. I have > >> considered the following: > >> > >> 1. > >> Use wl_surface::set_input_region so that pointer events are not > >> delivered to the OSD overlay, but instead pass through and are handled > >> by the main application. > >> This "almost" works, however when the main application receives the > >> event, Weston brings it on top, thus hiding the OSD overlay. > >> I searched for a way to specify that a given surface should always > >> stay "on top". This does not seem to be supported in Wayland. > >> > >> 2. > >> Setup things so that the OSD overlay is not an additional top level > >> surface, but a child surface owned by the main application. > >> This would mean having the main application create a child surface for > >> the OSD, and pass this surface to the OSD process. > >> In X this could be achieved by passing a Windows ID. In Wayland > >> apparently it is not possible to share surfaces between processes. > >> > >> 3. > >> Let the OSD overlay actually handle pointer events, but "send" them to > >> the main application. > >> In X this could be achieved with XSendEvent. In Wayland apparently it > >> is not possible to have one process send events to another process. > >> > >> Not sure what else to try. > >> > >> Any ideas? Either ways to make one of the above work, or new > approaches... > >> > >> TIA, > >> > >> Guillermo Rodriguez > >> ___ > >> wayland-devel mailing list > >> wayland-devel@lists.freedesktop.org > >> https://lists.freedesktop.org/mailman/listinfo/wayland-devel > ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
On Tue, 19 Nov 2019 09:53:39 +0100 Guillermo Rodriguez wrote: > Hi all, > > This is a follow up to my earlier email "Let pointer events pass through". > > I have an use case where there is one full screen application (I'll > call this the "main" application) and I need to show a OSD overlay on > top of it. The OSD overlay is actually based on GStreamer (think PiP) > and is managed by a different process. > The OSD overlay should not handle pointer events; these should pass > through to the main application, which knows how to handle them. > > I am not sure how to approach this in Wayland/Weston. I have > considered the following: > > 1. > Use wl_surface::set_input_region so that pointer events are not > delivered to the OSD overlay, but instead pass through and are handled > by the main application. > This "almost" works, however when the main application receives the > event, Weston brings it on top, thus hiding the OSD overlay. > I searched for a way to specify that a given surface should always > stay "on top". This does not seem to be supported in Wayland. Hi, yes. I'm not aware of any Wayland extension that would allow clients to dictate always-on-top behaviour on the desktop. It has an obvious attack vector (malicious or misbehaving app setting always-on-top without user's consent) and a simple corner case (how do you stack multiple always-on-top windows). OSD as a concept seems like a desktop component, not a normal app. Hence I would expect such a client to use a domain-specific or custom protocol extension to set always-on-top. If your use case is not a generic desktop but some special environment, then a domain-specific protocol extension would be a way to go. For generic desktop environments you'd integrate with the particular DE specifically, e.g. a gnome-shell plugin or something. That is, if the OSD really is an independent separate program and not a part of some fullscreen application. If the OSD is just a piece of a bigger application and only needs to be on top of that application's windows, you could probably use xdg extensions or a sub-surface to realize it. This does mean that the OSD component must be in-process with the main app and share the same wl_display instance. I think this should be possible also with GStreamer. > 2. > Setup things so that the OSD overlay is not an additional top level > surface, but a child surface owned by the main application. > This would mean having the main application create a child surface for > the OSD, and pass this surface to the OSD process. > In X this could be achieved by passing a Windows ID. In Wayland > apparently it is not possible to share surfaces between processes. I'd avoid a design with two separate processes. As Ilia wrote, xdg-foreign exists, but I'm not sure which compositors implement it. In Wayland, all protocol objects are private to the Wayland connection (wl_display) where they were created. xdg-foreign is a way to work around that, but I'd leave xdg-foreign only to cases that really need to have separate processes for e.g. security reasons. > 3. > Let the OSD overlay actually handle pointer events, but "send" them to > the main application. > In X this could be achieved with XSendEvent. In Wayland apparently it > is not possible to have one process send events to another process. Once an input event has been delivered to a client, there is no way to send it back or to another client in Wayland. Input event routing is one-way. In Wayland, each client is isolated by default, and we try hard to keep things that way so there is no spying or injection. Thanks, pq pgpN3eJn0TmCo.pgp Description: OpenPGP digital signature ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
Hello, Indeed I looked into xdg-foreign but it does not seem to be supported in Weston. Other ideas? Guillermo El mar., 19 nov. 2019 a las 10:21, Ilia Bozhinov () escribió: > > Hi, > > I am not entirely familiar with your setup, but you can try to use a > combination of: > > 1. Setting your on-top window's input region to empty as you already have > tried. > 2. Use xdg-foreign protocol to set the main window as parent of your on-top > window. >> > This will work on all compositors that support xdg-foreign, however I am not > sure whether weston supports it. > > Regards, > Ilia Bozhinov > > On Tue, 19 Nov 2019, 09:53 Guillermo Rodriguez, > wrote: >> >> Hi all, >> >> This is a follow up to my earlier email "Let pointer events pass through". >> >> I have an use case where there is one full screen application (I'll >> call this the "main" application) and I need to show a OSD overlay on >> top of it. The OSD overlay is actually based on GStreamer (think PiP) >> and is managed by a different process. >> The OSD overlay should not handle pointer events; these should pass >> through to the main application, which knows how to handle them. >> >> I am not sure how to approach this in Wayland/Weston. I have >> considered the following: >> >> 1. >> Use wl_surface::set_input_region so that pointer events are not >> delivered to the OSD overlay, but instead pass through and are handled >> by the main application. >> This "almost" works, however when the main application receives the >> event, Weston brings it on top, thus hiding the OSD overlay. >> I searched for a way to specify that a given surface should always >> stay "on top". This does not seem to be supported in Wayland. >> >> 2. >> Setup things so that the OSD overlay is not an additional top level >> surface, but a child surface owned by the main application. >> This would mean having the main application create a child surface for >> the OSD, and pass this surface to the OSD process. >> In X this could be achieved by passing a Windows ID. In Wayland >> apparently it is not possible to share surfaces between processes. >> >> 3. >> Let the OSD overlay actually handle pointer events, but "send" them to >> the main application. >> In X this could be achieved with XSendEvent. In Wayland apparently it >> is not possible to have one process send events to another process. >> >> Not sure what else to try. >> >> Any ideas? Either ways to make one of the above work, or new approaches... >> >> TIA, >> >> Guillermo Rodriguez >> ___ >> wayland-devel mailing list >> wayland-devel@lists.freedesktop.org >> https://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: How to implement OSD overlay in Wayland/Weston
Hi, I am not entirely familiar with your setup, but you can try to use a combination of: 1. Setting your on-top window's input region to empty as you already have tried. 2. Use xdg-foreign protocol to set the main window as parent of your on-top window. This will work on all compositors that support xdg-foreign, however I am not sure whether weston supports it. Regards, Ilia Bozhinov On Tue, 19 Nov 2019, 09:53 Guillermo Rodriguez, < guillerodriguez@gmail.com> wrote: > Hi all, > > This is a follow up to my earlier email "Let pointer events pass through". > > I have an use case where there is one full screen application (I'll > call this the "main" application) and I need to show a OSD overlay on > top of it. The OSD overlay is actually based on GStreamer (think PiP) > and is managed by a different process. > The OSD overlay should not handle pointer events; these should pass > through to the main application, which knows how to handle them. > > I am not sure how to approach this in Wayland/Weston. I have > considered the following: > > 1. > Use wl_surface::set_input_region so that pointer events are not > delivered to the OSD overlay, but instead pass through and are handled > by the main application. > This "almost" works, however when the main application receives the > event, Weston brings it on top, thus hiding the OSD overlay. > I searched for a way to specify that a given surface should always > stay "on top". This does not seem to be supported in Wayland. > > 2. > Setup things so that the OSD overlay is not an additional top level > surface, but a child surface owned by the main application. > This would mean having the main application create a child surface for > the OSD, and pass this surface to the OSD process. > In X this could be achieved by passing a Windows ID. In Wayland > apparently it is not possible to share surfaces between processes. > > 3. > Let the OSD overlay actually handle pointer events, but "send" them to > the main application. > In X this could be achieved with XSendEvent. In Wayland apparently it > is not possible to have one process send events to another process. > > Not sure what else to try. > > Any ideas? Either ways to make one of the above work, or new approaches... > > TIA, > > Guillermo Rodriguez > ___ > wayland-devel mailing list > wayland-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/wayland-devel
