Re: Sub-surface protocol
Kai-Uwe Behrmann wrote: However, we had seen quite some objections around subwindows at that time. Did something substancial change on that matter? I don't see anything there that applies to Wayland. The link to the original proposal is also dead. the old thread starter from Tomas for reference: http://www.spinics.net/lists/xorg/msg35268.html I think the objection is that under X11 a subwindow allocates an entire new drawing buffer. This is a pain for toolkits and applications that now have to manage these multiple buffers and switch drawing contexts (which loses things like the current clip and transform) to draw this portion of their display. It also wastes memory for the unused part of the main window image buffer. However it sounds like Wayland will allow a client to define a subsurface that reuses the same image buffer as the main surface, by indicating that a subrectangle of that image buffer is the subsurface contents. This means the client can just draw this area mixed in any way it wants with the drawing of everything else in the main window. The only problem with this idea is that there cannot be any partial transparency in the area with a different color correction, since the result is that it will be composited atop the same image drawn with the normal color correction. Since the main reason for this is to show images and videos which are opaque I would think this is not a big problem. It may also be possible to indicate that a subsurface holes the parent surface, the compositor would then not draw that area of the parent (which the current compositor is already doing when an overlapping surface is known to be opaque). I like the idea of reusing existing wayland data structures and for this reason I think using subsurfaces for this will work very well. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Mon, 17 Dec 2012 15:47:43 + Richard Hughes hughsi...@gmail.com wrote: On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. Sorry to be late to the party. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. The use case are image viewers like EOG or editors like GIMP. Gimp wants to color correct the main preview window converting from source ICC profile to monitor ICC profile, but we really don't want to turn off the sRGB-monitor profile for the selector widgets. By using a subsurface we get the behaviour we want without adding tons of code. At the moment Oyranos has some kind of xatom protocol to let the compositor know an opt-out-region and this isn't ideal or suited at all to wayland. Anyway, I just wanted to say rock on and maybe to keep ICC color correction at the back of your mind for the sub-surface. :) Cool, yeah, I think color management of individual sub-surfaces will not be a problem, as long as the color management interfaces deal with wl_surface and wl_buffer objects from the core protocol. In my current plan, sub-surfaces are wl_surface objects with a particular role, so color management should interact in obvious ways, and work with minimal additional changes. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Tue, Dec 18, 2012 at 6:40 AM, Bill Spitzak spit...@gmail.com wrote: On Dec 17, 2012, at 5:01 PM, John Kåre Alsaker wrote: Then a client such as gimp could draw all it's display into a single buffer. To get the different color correction of the center display, it would declare a subsurface containing this and set it's color correction differently using the wayland api. This would only allocate one buffer, which would save memory, but more importantly it should make internal code in gimp easier as you could work around a toolkit that assumes only one output buffer. My approach to color correct rendering in Wayland is to let the compositor handle it and have the clients simply specify which color space they are using. Only the compositor can render clients correctly when they are displayed on multiple monitors (unless we introduce a way to set a buffer per-output). Yes this is what I am assuming. What I am asking is if it makes sense for a client to draw two different color spaces into the same buffer. The larger area is color space A, and a rectangular subarea is color space B. It then tells wayland to make the main window color space A, and then makes a subwindow with the clipped subarea and tells wayland that it is color space B. The reason is to save buffer space, and also to work around toolkits and drawing api's where it is a lot more practical to draw everything into the same buffer. I don't see any problem with doing that unless you want different bit-depths. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Tue, Dec 18, 2012 at 2:59 PM, John Kåre Alsaker john.kare.alsa...@gmail.com wrote: On Tue, Dec 18, 2012 at 6:40 AM, Bill Spitzak spit...@gmail.com wrote: On Dec 17, 2012, at 5:01 PM, John Kåre Alsaker wrote: Then a client such as gimp could draw all it's display into a single buffer. To get the different color correction of the center display, it would declare a subsurface containing this and set it's color correction differently using the wayland api. This would only allocate one buffer, which would save memory, but more importantly it should make internal code in gimp easier as you could work around a toolkit that assumes only one output buffer. My approach to color correct rendering in Wayland is to let the compositor handle it and have the clients simply specify which color space they are using. Only the compositor can render clients correctly when they are displayed on multiple monitors (unless we introduce a way to set a buffer per-output). Yes this is what I am assuming. What I am asking is if it makes sense for a client to draw two different color spaces into the same buffer. The larger area is color space A, and a rectangular subarea is color space B. It then tells wayland to make the main window color space A, and then makes a subwindow with the clipped subarea and tells wayland that it is color space B. The reason is to save buffer space, and also to work around toolkits and drawing api's where it is a lot more practical to draw everything into the same buffer. I don't see any problem with doing that unless you want different bit-depths. You could also construct the window decorations out of 4 subsurfaces and get alpha working on the center subsurface, but I doubt that's particularly useful. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Tue, Dec 18, 2012 at 10:29 AM, Pekka Paalanen ppaala...@gmail.com wrote: On Mon, 17 Dec 2012 15:47:43 + Richard Hughes hughsi...@gmail.com wrote: On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. Sorry to be late to the party. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. The use case are image viewers like EOG or editors like GIMP. Gimp wants to color correct the main preview window converting from source ICC profile to monitor ICC profile, but we really don't want to turn off the sRGB-monitor profile for the selector widgets. By using a subsurface we get the behaviour we want without adding tons of code. At the moment Oyranos has some kind of xatom protocol to let the compositor know an opt-out-region and this isn't ideal or suited at all to wayland. Anyway, I just wanted to say rock on and maybe to keep ICC color correction at the back of your mind for the sub-surface. :) Cool, yeah, I think color management of individual sub-surfaces will not be a problem, as long as the color management interfaces deal with wl_surface and wl_buffer objects from the core protocol. In my current plan, sub-surfaces are wl_surface objects with a particular role, so color management should interact in obvious ways, and work with minimal additional changes. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel My plan has always been to attach color space profiles to wl_surfaces, which will just end to working with subsurfaces. When combined with clipping, you can attach color profiles to any part of your surface. On Tue, Dec 18, 2012 at 10:45 AM, Kai-Uwe Behrmann k...@gmx.de wrote: Am 17.12.2012 16:47, schrieb Richard Hughes: On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. Sounds natural to help with CM compositing needs and was therefore discussed [1] for OpenICC's Color Management Near X project in 2008. However, we had seen quite some objections around subwindows at that time. Did something substancial change on that matter? I don't see anything there that applies to Wayland. The link to the original proposal is also dead. To put the question in other words, do Qt and Gtk developers now or soon play with the idea to use wayland as the internal compositing core? I believe both have paths to use subwindows/subsurfaces for video and OpenGL. These can probably be used for clients with special needs, such as image viewers and editors which would simply attach an ICC profile to the subsurface. For compositing they could use subsurfaces to optimize scrolling performance. I expect that to happen with browsers first and I believe Android already does that. kind regards Kai-Uwe Behrmann [1] https://mail.gnome.org/archives/gtk-devel-list/2008-June/msg00150.html ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
Am 18.12.2012 06:40, schrieb Bill Spitzak: On Dec 17, 2012, at 5:01 PM, John Kåre Alsaker wrote: Then a client such as gimp could draw all it's display into a single buffer. To get the different color correction of the center display, it would declare a subsurface containing this and set it's color correction differently using the wayland api. This would only allocate one buffer, which would save memory, but more importantly it should make internal code in gimp easier as you could work around a toolkit that assumes only one output buffer. My approach to color correct rendering in Wayland is to let the compositor handle it and have the clients simply specify which color space they are using. Only the compositor can render clients correctly when they are displayed on multiple monitors (unless we introduce a way to set a buffer per-output). Yes this is what I am assuming. What I am asking is if it makes sense for a client to draw two different color spaces into the same buffer. The larger area is color space A, and a rectangular subarea is color space B. It then tells wayland to make the main window color space A, and then makes a subwindow with the clipped subarea and tells wayland that it is color space B. That is how the X Color Management spec implementations communicate between clients and servers. Take a window handle and tag it with a region, which in turn points to a ICC profile. I think Martin had a similar idea to place regions on own textures for KWin some time ago. The reason is to save buffer space, and also to work around toolkits and drawing api's where it is a lot more practical to draw everything into the same buffer. For this reason Xcm adopted the per region tagging approach. Qt and Gtk people tould us, they do not like to expose subwindows. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Tue, Dec 18, 2012 at 10:45 AM, Kai-Uwe Behrmann k...@gmx.de wrote: Am 17.12.2012 16:47, schrieb Richard Hughes: On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. Sounds natural to help with CM compositing needs and was therefore discussed [1] for OpenICC's Color Management Near X project in 2008. However, we had seen quite some objections around subwindows at that time. Did something substancial change on that matter? I don't see anything there that applies to Wayland. The link to the original proposal is also dead. the old thread starter from Tomas for reference: http://www.spinics.net/lists/xorg/msg35268.html To put the question in other words, do Qt and Gtk developers now or soon play with the idea to use wayland as the internal compositing core? I believe both have paths to use subwindows/subsurfaces for video and OpenGL. These can probably be used for clients with special needs, such as image viewers and editors which would simply attach an ICC profile to the subsurface. For compositing they could use subsurfaces to optimize scrolling performance. I expect that to happen with browsers first and I believe Android already does that. Reads encouraging. [1] https://mail.gnome.org/archives/gtk-devel-list/2008-June/msg00150.html ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. Sorry to be late to the party. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. The use case are image viewers like EOG or editors like GIMP. Gimp wants to color correct the main preview window converting from source ICC profile to monitor ICC profile, but we really don't want to turn off the sRGB-monitor profile for the selector widgets. By using a subsurface we get the behaviour we want without adding tons of code. At the moment Oyranos has some kind of xatom protocol to let the compositor know an opt-out-region and this isn't ideal or suited at all to wayland. Anyway, I just wanted to say rock on and maybe to keep ICC color correction at the back of your mind for the sub-surface. :) Thanks, Richard ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
That brings up an interesting possibility: can a subsurface (or any surface) image be a subrectangle of an image used for another surface? I'm thinking this would be possible, provided the client can tell the wayland server to use an arbitrary stride between rows. Then a client such as gimp could draw all it's display into a single buffer. To get the different color correction of the center display, it would declare a subsurface containing this and set it's color correction differently using the wayland api. This would only allocate one buffer, which would save memory, but more importantly it should make internal code in gimp easier as you could work around a toolkit that assumes only one output buffer. The main problem I see is that the subsurface has to be opaque, because any transparent parts would just show the same data with the wrong color correction. But that would cover a lot of uses. Richard Hughes wrote: On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. Sorry to be late to the party. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. The use case are image viewers like EOG or editors like GIMP. Gimp wants to color correct the main preview window converting from source ICC profile to monitor ICC profile, but we really don't want to turn off the sRGB-monitor profile for the selector widgets. By using a subsurface we get the behaviour we want without adding tons of code. At the moment Oyranos has some kind of xatom protocol to let the compositor know an opt-out-region and this isn't ideal or suited at all to wayland. Anyway, I just wanted to say rock on and maybe to keep ICC color correction at the back of your mind for the sub-surface. :) Thanks, Richard ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Mon, Dec 17, 2012 at 11:40 PM, Bill Spitzak spit...@gmail.com wrote: That brings up an interesting possibility: can a subsurface (or any surface) image be a subrectangle of an image used for another surface? I'm thinking this would be possible, provided the client can tell the wayland server to use an arbitrary stride between rows. You should be able to attach the same wl_buffer to multiple wl_surfaces and set clipping (when that is added) accordingly. Then a client such as gimp could draw all it's display into a single buffer. To get the different color correction of the center display, it would declare a subsurface containing this and set it's color correction differently using the wayland api. This would only allocate one buffer, which would save memory, but more importantly it should make internal code in gimp easier as you could work around a toolkit that assumes only one output buffer. My approach to color correct rendering in Wayland is to let the compositor handle it and have the clients simply specify which color space they are using. Only the compositor can render clients correctly when they are displayed on multiple monitors (unless we introduce a way to set a buffer per-output). The main problem I see is that the subsurface has to be opaque, because any transparent parts would just show the same data with the wrong color correction. But that would cover a lot of uses. Richard Hughes wrote: On 5 December 2012 14:32, Pekka Paalanen ppaala...@gmail.com wrote: One of the most important use cases is a video player in a window. It has XRGB or ARGB window decorations, usually the video content in YUV, and possibly an ARGB overlay for subtitles etc. Currently, the client has to color-convert the video, and merge it with the decorations and subtitles, before sending the grand ARGB buffer to the compositor. Sorry to be late to the party. A subsurface idea was what Øyvind and I were discussing at the OpenICC hackfest when we were talking about tagging a surface with a known ICC profile. This would allow a toolkit to declare a window area to be AdobeRGB or some home-created camera profile from a jpeg that has been tagged with an embedded color profile. The use case are image viewers like EOG or editors like GIMP. Gimp wants to color correct the main preview window converting from source ICC profile to monitor ICC profile, but we really don't want to turn off the sRGB-monitor profile for the selector widgets. By using a subsurface we get the behaviour we want without adding tons of code. At the moment Oyranos has some kind of xatom protocol to let the compositor know an opt-out-region and this isn't ideal or suited at all to wayland. Anyway, I just wanted to say rock on and maybe to keep ICC color correction at the back of your mind for the sub-surface. :) Thanks, Richard ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Dec 17, 2012, at 5:01 PM, John Kåre Alsaker wrote: Then a client such as gimp could draw all it's display into a single buffer. To get the different color correction of the center display, it would declare a subsurface containing this and set it's color correction differently using the wayland api. This would only allocate one buffer, which would save memory, but more importantly it should make internal code in gimp easier as you could work around a toolkit that assumes only one output buffer. My approach to color correct rendering in Wayland is to let the compositor handle it and have the clients simply specify which color space they are using. Only the compositor can render clients correctly when they are displayed on multiple monitors (unless we introduce a way to set a buffer per-output). Yes this is what I am assuming. What I am asking is if it makes sense for a client to draw two different color spaces into the same buffer. The larger area is color space A, and a rectangular subarea is color space B. It then tells wayland to make the main window color space A, and then makes a subwindow with the clipped subarea and tells wayland that it is color space B. The reason is to save buffer space, and also to work around toolkits and drawing api's where it is a lot more practical to draw everything into the same buffer. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Thu, 13 Dec 2012 14:47:57 -0800 Bill Spitzak spit...@gmail.com wrote: FLOATING WINDOWS: PLEASE consider reusing this code for floating windows. THEY ARE THE SAME THING!!! The only difference is that the compositor can insert other surfaces between floating children and the parent. This would allow all work done to make subsurfaces not blink and to move in lock-step also be used for floating windows, which would address a deficiency in all existing window systems. It will also force the floating window api to allow the client to be in final control of the stacking order, a deficiency in all existing window systems. Conversely floating windows have a lot of stuff figured out for focus and event handling, like grabs, and cooperation between tasks and threads. All of this is needed for subwindows so it makes sense to reuse it. No, sub-surfaces are not sub-windows. A set of sub-surfaces forms a single solid window. Surface management belongs to the core, window management to the shell. At first I was thinking that I could implement tooltips as sub-surfaces since they do not need any shell features, but then I realized that even that is nonsense, as it would break the _window_ geometry. CLIP: Main windows need clip as well. This is so they can draw edge decorations that are clipped off when the window is maximized, rather than having to alter their drawings depending on maximization. More important surfaces should be able to inverse-clip (the clip is bigger than the image) for cases where the shell api needs a particular size but the client knows that a portion is not visible. We need this to clip off any number of edges less than 4 (for maximize horizontal/vertical), and to allow subwindows to be used to add the borders to a main window that is actually smaller than it tells the shell. I don't think so. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Thu, 13 Dec 2012 12:33:33 -0500 Kristian Høgsberg hoegsb...@gmail.com wrote: On Thu, Dec 13, 2012 at 04:07:44PM +0100, John Kåre Alsaker wrote: On Thu, Dec 13, 2012 at 3:30 PM, Pekka Paalanen ppaala...@gmail.com wrote: On Thu, 13 Dec 2012 14:51:17 +0100 John Kåre Alsaker john.kare.alsa...@gmail.com wrote: I see that in weston the shell is very glad in inspecting geometry. We also have to alpha property per-surface instead of per wl_shell_surface. For fading windows we may want to compose the wl_surface_group into a single image. I don't think we want to do intermediate composites, unless we can put it into a plane. Otherwise it's just extra data movement. It seems to be very easy to hook a compound window into the input and repaint procedures by just adding all the wl_surfaces into the big surface list created during weston_output_repaint(). I don't have to revamp either input nor repaint code. I actually got that part working right now. For alpha, we can easily enough change the surface alpha for all... wait, that would not really work right. We'd have to manipulate the repaint regions, too. Maybe that's not even enough, oh well. We have to compose it or we'll see through parts that's supposed to be opaque (I think Cinnamon does that for it's menu, it looks odd). Yeah, that's a good point. Blending of full-surface alpha may become a source of great pain, but we'll tackle it when we have a test case. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On 12/14/2012 12:29 AM, John Kåre Alsaker wrote: On Thu, Dec 13, 2012 at 11:47 PM, Bill Spitzak spit...@gmail.com wrote: It will also force the floating window api to allow the client to be in final control of the stacking order, a deficiency in all existing window systems. Do you have an example of an UI which require the client to be in control of the stacking order? If that's really desired it probably won't hurt if is shared the stacking order protocol with subsurfaces. Not really require, but the list is practically endless. I want pidgin to show the window that last received a message, unless I'm typing in one, unless I actively click on another one. I want gimp to show the window with my tools on top of the image I'm editing, even if my last click went to the image window. I want my video-selector to stay in the background while a video is playing, even if I click in the selector to queue another one while the first one is running. The compositor will *never* know the correct stacking of windows in a multi-window program. The program should always know the default and the user should always be able to change it, forcing the compositor to ignore the program's wishes if he/she so desires. That last bit is pretty important, actually. The always on top feature from Gnome 2 was awesome. I used that a lot, and it could, with some major annoyance, be used to work around even the most recalcitrant windowing issues. -- Andreas Ericsson andreas.erics...@op5.se OP5 AB www.op5.se Tel: +46 8-230225 Fax: +46 8-230231 Considering the successes of the wars on alcohol, poverty, drugs and terror, I think we should give some serious thought to declaring war on peace. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Wed, 12 Dec 2012 14:22:49 + Daniel Stone dan...@fooishbar.org wrote: Hi, On 6 December 2012 01:32, Pekka Paalanen ppaala...@gmail.com wrote: Clipping The term sub-surface sounds like a sub-window, which may cause one to think, that it will be clipped to the parent surface area. I do not think we should force or even allow such clipping. Forcing the clipping would waste memory: for every pixel in sub-surfaces there would have to be a pixel in the parent surface. For instance, drawing window decorations in four sub-surfaces around the main content would not only waste memory, but also not solve the problem of GL applications, where one needs to reserve a margin for the decorations. (Hello, window decoration libraries? :-) Right, so we're coming at this from completely different angles then. I approached this more or less purely from the embedding sense, where you either have a video element (e.g. GStreamer's waylandvideosink) or a plugin such as Flash in one parent window. In this case, you do absolutely want clipping. If I was looking at it from the point of view of having the decorations and window content have a subsurface relationship, I'd be much more tempted to express it as a group of surfaces instead. In this case, rather than having a strict parent/child relationship, we'd have a group of surfaces with an internal stacking order, and no concept of a parent at all. This avoids the need for either disabling occlusion to the parent's bounding box (which is IMO dangerous and misleading with a parent/child scheme, even if it does allow for a usecase it wasn't in any way designed for), or having the parent wl_surface be effectively larger than it actually is. Something to think about. Merely allowing clipping is just not the Wayland style. If the *client* wants to clip something, it should not draw it in the first place. For my usecases, clipping is pretty important since you don't want your video content creeping out over the window borders. _Not_ allowing clipping is wasteful in some cases: consider a Flash video in your browser, which is scrolled half-offscreen. You'd have to copy the entire decoded buffer into an intermediate buffer, then copy only the visible part of your data into a wl_buffer. Every time you scrolled, you'd have to create a differently-sized wl_buffer, involving still more copies. That being said, I don't think clipping to the parent's bounds is sufficient for this (since you also don't want the child surface drawing all over your decorations). I think we can get around this though: see below. Ahh, now I realize we have been talking about completely different kinds of clipping. Neither of us really wants or needs automatic clipping to another wl_surface, be it the parent or something else, right? But we definitely could use an arbitrary clipping rectangle, that the client can explicitly set, and is not modified by the server on its own. This leads us to the question, should the clipping feature be tied to subsurfaces, or be completely orthogonal to sub-surfaces. Either way, I do not think it would invalidate much of the work I am currently doing for sub-surfaces, so the question is not blocking me. Clipping, and scaling, would detach wl_surface size from the wl_buffer size. That poses a lot of new design questions on the protocol level. After sub-surfaces and clipping, I wonder if we have reinvented the X Window object... at least we don't have window properties yet ;-) * The pending state of all sub-surfaces is applied only on the parent surface commit. Sub-surface commit is a no-op. This will nicely guarantee atomicity. The set of surfaces forms a single application window, and all the surfaces must be updated in sync to avoid flickering a bad composite. Resizing will absolutely require this to work reliably and without glitches. The downside is, that e.g. a video sink component cannot just happily push buffers to its own sub-surface. It has to signal the application to commit the parent surface for every single frame. Some flash, plugin, or video APIs might not support that. Do we care? It's some amount of overhead, and for the 99% case of embedded media, unnecessary if you go with #3 below. * Implicit fallback to requiring parent commit on sub-surface configuration change. (by daniels) This means, that sub-surface commit works like a normal surface's commit, if the sub-surface configuration (position, buffer size) does not change. When the configuration changes, the compositor will apply the new sub-surface state only on the parent surface commit, and until then it will not complete the sub-surface commit. This will allow a video sink to run on its own, as long as the sub-surface size or position, e.g. via wl_surface.attach, does not change. Resizing can be handled by the application first signalling a resize to the
Re: Sub-surface protocol
Here is my subsurface proposal. I don't like video sinks (or other things) running on an independent framerate. I don't want to maintain more state in the compositor side for them or have increased complexity in the protocol. They are inefficient and can be solved by a number of other ways. In those cases you can change the API, clients can buffer the video sink state for themselves (which you should be able to do if you're porting to Wayland), or the worst case, render them off-screen and let the client draw them as they please. My proposal uses a wl_surface_group which is a group of wl_surfaces with a defined Z-order and positions relative to the main surface. It has a concept of a main surface which is the surface which acts as the wl_surface_group for other APIs like wl_shell_surface. Member surfaces cannot have a wl_shell_surface. main_surface is created in the create_surface_group request so it can have a different implementation (which probably won't happen in Weston). interface name=wl_surface_group_factory version=1 request name=create_surface_group description summary=remove a surface from the group This creates a new wl_surface_group with 'main_surface' as the main surface. The commit for the main surface commits all pending place_above, place_below, set_position, remove requests and pending state for all member surfaces. Doing commit on member surface will be a no-op. Moving the main surface will move all member surfaces too. Only the main surface can have an associated wl_shell_surface or other shell interfaces. /description arg name=new_surface_group type=new_id interface=wl_surface_group/ arg name=main_surface type=new_id interface=wl_surface/ /request /interface interface name=wl_surface_group version=1 request name=destroy type=destructor /request request name=remove description summary=remove a surface from the group This removes a surface from the group. The main surface cannot be removed. /description arg name=surface type=object interface=wl_surface/ /request request name=set_position description summary=remove a surface from the group This sets the position of a member surface relative to the main surface. /description arg name=surface type=object interface=wl_surface/ arg name=x type=int/ arg name=y type=int/ /request request name=place_above description summary=place a surface above another This places 'surface' right above 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request request name=place_below description summary=place a surface below another This places 'surface' right below 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request /interface I see that in weston the shell is very glad in inspecting geometry. We also have to alpha property per-surface instead of per wl_shell_surface. For fading windows we may want to compose the wl_surface_group into a single image. On Thu, Dec 13, 2012 at 12:54 PM, Pekka Paalanen ppaala...@gmail.com wrote: On Wed, 12 Dec 2012 14:22:49 + Daniel Stone dan...@fooishbar.org wrote: Hi, On 6 December 2012 01:32, Pekka Paalanen ppaala...@gmail.com wrote: Clipping The term sub-surface sounds like a sub-window, which may cause one to think, that it will be clipped to the parent surface area. I do not think we should force or even allow such clipping. Forcing the clipping would waste memory: for every pixel in sub-surfaces there would have to be a pixel in the parent surface. For instance, drawing window decorations in four sub-surfaces around the main content would not only waste memory, but also not solve the problem of GL applications, where one needs to reserve a margin for the decorations. (Hello, window decoration libraries? :-) Right, so we're coming at this from completely different angles
Re: Sub-surface protocol
Hi John, On Thu, 13 Dec 2012 14:51:17 +0100 John Kåre Alsaker john.kare.alsa...@gmail.com wrote: Here is my subsurface proposal. I don't like video sinks (or other things) running on an independent framerate. I don't want to maintain more state in the compositor side for them or have increased complexity in the protocol. They are inefficient and can be solved by a number of other ways. In those cases you can change the API, clients can buffer the video sink state for themselves (which you should be able to do if you're porting to Wayland), or the worst case, render them off-screen and let the client draw them as they please. Ok, that is the reason you chose that sub-surface commits are no-op, right? My proposal uses a wl_surface_group which is a group of wl_surfaces with a defined Z-order and positions relative to the main surface. It has a concept of a main surface which is the surface which acts as the wl_surface_group for other APIs like wl_shell_surface. Member surfaces cannot have a wl_shell_surface. main_surface is created in the create_surface_group request so it can have a different implementation (which probably won't happen in Weston). My thoughts exactly, just slightly different objects in the protocol. I'm hooking the sub-surface role into the weston_surface::configure pointer, which automatically excludes all other roles for a surface. interface name=wl_surface_group_factory version=1 request name=create_surface_group description summary=remove a surface from the group This creates a new wl_surface_group with 'main_surface' as the main surface. The commit for the main surface commits all pending place_above, place_below, set_position, remove requests and pending state for all member surfaces. Doing commit on member surface will be a no-op. Moving the main surface will move all member surfaces too. Only the main surface can have an associated wl_shell_surface or other shell interfaces. /description arg name=new_surface_group type=new_id interface=wl_surface_group/ arg name=main_surface type=new_id interface=wl_surface/ /request /interface interface name=wl_surface_group version=1 request name=destroy type=destructor /request request name=remove description summary=remove a surface from the group This removes a surface from the group. The main surface cannot be removed. /description arg name=surface type=object interface=wl_surface/ /request request name=set_position description summary=remove a surface from the group This sets the position of a member surface relative to the main surface. /description arg name=surface type=object interface=wl_surface/ arg name=x type=int/ arg name=y type=int/ /request request name=place_above description summary=place a surface above another This places 'surface' right above 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request request name=place_below description summary=place a surface below another This places 'surface' right below 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request /interface This looks very much like what I am working with, with the exceptions: - instead of wl_surface_group, each sub-surface has a wl_subsurface object, whose creation defines also the parent. - the requests you have in wl_surface_group, I have in wl_subsurface, practically identical otherwise - what you call main surface, I call parent The description in your create_surface_group is pretty much the same semantics I am implementing, with the difference in sub-surface commit behaviour. I have my version in a WIP branch: http://cgit.collabora.com/git/user/pq/wayland.git/tree/protocol/wayland.xml?h=subsurface-wip#n1358 There are still some semantics to be explained more. It's a simple addition to protocol, but the semantics and implementation are
Re: Sub-surface protocol
I added buffered commit_surface in wl_surface_group, which allows clients to atomically update the surfaces of their choice. This way we don't have to change wl_surface.commit. We can also update the main or parent surface and subsurfaces independently. interface name=wl_surface_group_factory version=1 request name=create_surface_group description summary=remove a surface from the group This creates a new wl_surface_group with 'main_surface' as the main surface. Moving the main surface will move all member surfaces too. Only the main surface can have an associated wl_shell_surface or other shell interfaces. /description arg name=new_surface_group type=new_id interface=wl_surface_group/ arg name=main_surface type=object interface=wl_surface/ /request /interface interface name=wl_surface_group version=1 request name=destroy type=destructor /request request name=remove description summary=remove a surface from the group This removes a surface from the group. The main surface cannot be removed. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ /request request name=set_position description summary=remove a surface from the group This sets the position of a member surface relative to the main surface. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ arg name=x type=int/ arg name=y type=int/ /request request name=place_above description summary=place a surface above another This places 'surface' right above 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request request name=place_below description summary=place a surface below another This places 'surface' right below 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request request name=commit_surface description summary=commit a surface This calls wl_surface.commit on the surface. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ /request request name=commit description summary=execute all pending requests This executes all pending remove, place_above, place_below, set_position and commit_surface requests atomically. /description /request /interface On Thu, Dec 13, 2012 at 3:30 PM, Pekka Paalanen ppaala...@gmail.com wrote: Hi John, On Thu, 13 Dec 2012 14:51:17 +0100 John Kåre Alsaker john.kare.alsa...@gmail.com wrote: Here is my subsurface proposal. I don't like video sinks (or other things) running on an independent framerate. I don't want to maintain more state in the compositor side for them or have increased complexity in the protocol. They are inefficient and can be solved by a number of other ways. In those cases you can change the API, clients can buffer the video sink state for themselves (which you should be able to do if you're porting to Wayland), or the worst case, render them off-screen and let the client draw them as they please. Ok, that is the reason you chose that sub-surface commits are no-op, right? My proposal uses a wl_surface_group which is a group of wl_surfaces with a defined Z-order and positions relative to the main surface. It has a concept of a main surface which is the
Re: Sub-surface protocol
On Thu, Dec 13, 2012 at 04:07:44PM +0100, John Kåre Alsaker wrote: I added buffered commit_surface in wl_surface_group, which allows clients to atomically update the surfaces of their choice. This way we don't have to change wl_surface.commit. We can also update the main or parent surface and subsurfaces independently. interface name=wl_surface_group_factory version=1 request name=create_surface_group description summary=remove a surface from the group This creates a new wl_surface_group with 'main_surface' as the main surface. Moving the main surface will move all member surfaces too. Only the main surface can have an associated wl_shell_surface or other shell interfaces. /description arg name=new_surface_group type=new_id interface=wl_surface_group/ arg name=main_surface type=object interface=wl_surface/ /request /interface interface name=wl_surface_group version=1 request name=destroy type=destructor /request request name=remove description summary=remove a surface from the group This removes a surface from the group. The main surface cannot be removed. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ /request request name=set_position description summary=remove a surface from the group This sets the position of a member surface relative to the main surface. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ arg name=x type=int/ arg name=y type=int/ /request request name=place_above description summary=place a surface above another This places 'surface' right above 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request request name=place_below description summary=place a surface below another This places 'surface' right below 'relative_to_surface'. If 'surface' isn't a member of this group, it will become a member. It is an error to add a surface belonging to another wl_surface_group. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ arg name=relative_to_surface type=object interface=wl_surface/ /request request name=commit_surface description summary=commit a surface This calls wl_surface.commit on the surface. This won't take effect until wl_surface_group.commit is called. /description arg name=surface type=object interface=wl_surface/ /request request name=commit description summary=execute all pending requests This executes all pending remove, place_above, place_below, set_position and commit_surface requests atomically. /description /request /interface On Thu, Dec 13, 2012 at 3:30 PM, Pekka Paalanen ppaala...@gmail.com wrote: Hi John, On Thu, 13 Dec 2012 14:51:17 +0100 John Kåre Alsaker john.kare.alsa...@gmail.com wrote: Here is my subsurface proposal. I don't like video sinks (or other things) running on an independent framerate. I don't want to maintain more state in the compositor side for them or have increased complexity in the protocol. They are inefficient and can be solved by a number of other ways. In those cases you can change the API, clients can buffer the video sink state for themselves (which you should be able to do if you're porting to Wayland), or the worst case, render them off-screen and let the client draw them as they please. Ok, that is the reason you chose that sub-surface commits are no-op, right? My proposal uses a wl_surface_group which is a group of wl_surfaces with a defined Z-order and positions relative to the main
Re: Sub-surface protocol
I see no reason for extra objects. What I would do is add a parent to the normal surface. If it is NULL then it is a main surface. If it points at another surface then it is a subsurface or a floating window. The parent can be changed arbitrarily. We must be able to change an existing surface from being a main to a subsurface to a floating window at any time, so making them different types is not acceptable. If the parent is another subsurface that controls the ordering, in an obvious manner without any additional api! FLOATING WINDOWS: PLEASE consider reusing this code for floating windows. THEY ARE THE SAME THING!!! The only difference is that the compositor can insert other surfaces between floating children and the parent. This would allow all work done to make subsurfaces not blink and to move in lock-step also be used for floating windows, which would address a deficiency in all existing window systems. It will also force the floating window api to allow the client to be in final control of the stacking order, a deficiency in all existing window systems. Conversely floating windows have a lot of stuff figured out for focus and event handling, like grabs, and cooperation between tasks and threads. All of this is needed for subwindows so it makes sense to reuse it. CLIP: Main windows need clip as well. This is so they can draw edge decorations that are clipped off when the window is maximized, rather than having to alter their drawings depending on maximization. More important surfaces should be able to inverse-clip (the clip is bigger than the image) for cases where the shell api needs a particular size but the client knows that a portion is not visible. We need this to clip off any number of edges less than 4 (for maximize horizontal/vertical), and to allow subwindows to be used to add the borders to a main window that is actually smaller than it tells the shell. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Thu, Dec 13, 2012 at 11:47 PM, Bill Spitzak spit...@gmail.com wrote: I see no reason for extra objects. What I would do is add a parent to the normal surface. If it is NULL then it is a main surface. If it points at another surface then it is a subsurface or a floating window. The parent can be changed arbitrarily. We could also go the other way and add a child pointer, but linked lists require more operations to do things like raise a surface above another. We must be able to change an existing surface from being a main to a subsurface to a floating window at any time, so making them different types is not acceptable. I don't see why we have to be able to change the surface type. Creating a new surface for each purpose is cheap. I'm not against being able to do that, I just don't see it as a requirement. If the parent is another subsurface that controls the ordering, in an obvious manner without any additional api! FLOATING WINDOWS: PLEASE consider reusing this code for floating windows. THEY ARE THE SAME THING!!! The only difference is that the compositor can insert other surfaces between floating children and the parent. Floating windows can have interactions with the shell (animations, etc) while subsurfaces are a way to offload rendering work from the clients and represent a single window. This would allow all work done to make subsurfaces not blink and to move in lock-step also be used for floating windows, which would address a deficiency in all existing window systems. We should probably have a way to atomically update multiple wl_surface anyway (which could be shared for subsurfaces and floating windows). Too bad atomicity wasn't added generically to the wayland protocol. We wouldn't have to add complexity to the code or protocol. It will also force the floating window api to allow the client to be in final control of the stacking order, a deficiency in all existing window systems. Do you have an example of an UI which require the client to be in control of the stacking order? If that's really desired it probably won't hurt if is shared the stacking order protocol with subsurfaces. Conversely floating windows have a lot of stuff figured out for focus and event handling, like grabs, and cooperation between tasks and threads. All of this is needed for subwindows so it makes sense to reuse it. That's mostly wl_surface things which will just work for subsurfaces. CLIP: Main windows need clip as well. This is so they can draw edge decorations that are clipped off when the window is maximized, rather than having to alter their drawings depending on maximization. Most existing toolkits alters and redraws for the maximized state or even resizes already. I think we all agree that clipping and perhaps eventually scaling, which the compositor can do more efficiently has nothing to do with subsurface and should be added to wl_surface. More important surfaces should be able to inverse-clip (the clip is bigger than the image) for cases where the shell api needs a particular size but the client knows that a portion is not visible. We need this to clip off any number of edges less than 4 (for maximize horizontal/vertical), and to allow subwindows to be used to add the borders to a main window that is actually smaller than it tells the shell. I'm not sure what you mean by that. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
John Kåre Alsaker wrote: On Thu, Dec 13, 2012 at 11:47 PM, Bill Spitzak spit...@gmail.com wrote: I see no reason for extra objects. What I would do is add a parent to the normal surface. If it is NULL then it is a main surface. If it points at another surface then it is a subsurface or a floating window. The parent can be changed arbitrarily. We could also go the other way and add a child pointer, but linked lists require more operations to do things like raise a surface above another. The implementation would probably use a linked list of *all* surfaces that are visible (ie surfaces belonging to every client). But this would be a data structure inside the shell, and has nothing to do with the client api. What I am talking about is the method that the client controls how the shell puts surfaces into this sorted list. The surfaces linked by the parent pointer have an enforced order that the shell should not mix up. But this data has little or nothing to do with the shell order since that will contain other surfaces that are not in this linked set. The simpler api also means the shell has to do less checking. I believe in mine the only thing the shell has to watch out for is loops. If the client provides a list then it has to make sure a surface is not in there twice and that a surface is not in two lists. We must be able to change an existing surface from being a main to a subsurface to a floating window at any time, so making them different types is not acceptable. I don't see why we have to be able to change the surface type. If a window is build up of panes that are docked together, and the user is allowed to close either pane, there is a requirement that either pane be able to become the main window. We should probably have a way to atomically update multiple wl_surface anyway (which could be shared for subsurfaces and floating windows). Too bad atomicity wasn't added generically to the wayland protocol. We wouldn't have to add complexity to the code or protocol. This is what all the commit stuff is about. Everybody is in agreement that commit is needed to make subsurfaces work. I also think it will fix blinking floating windows with almost zero extra effort if we just make them be the same thing! Do you have an example of an UI which require the client to be in control of the stacking order? If that's really desired it probably won't hurt if is shared the stacking order protocol with subsurfaces. Imagine two main windows and a toolbox that is intended to be above both of them. When the user attempts to raise one of the main windows, the toolbox must remain atop them both. The easiest way to do this is to have the client switch the toolbox from remaining above window A to be above window B along with the raise of window B (as an atomic operation). Schemes that let the shell handle this would require the client to send an arbitary directed acyclic graph of surfaces, rather than a simple list or tree. The clients must still be able to make arbitrary changes to this DAG when surfaces are created and destroyed or when state changes such that the order must change. I suspect the simpler and more intunitive list or tree (instead of a DAG) is a big win even though the some clients will need to also make changes on window reorder as well as create and destroy and state change. Conversely floating windows have a lot of stuff figured out for focus and event handling, like grabs, and cooperation between tasks and threads. All of this is needed for subwindows so it makes sense to reuse it. That's mostly wl_surface things which will just work for subsurfaces. I thought all the grab stuff was wl_shell_surface api? CLIP: Main windows need clip as well. This is so they can draw edge decorations that are clipped off when the window is maximized, rather than having to alter their drawings depending on maximization. Most existing toolkits alters and redraws for the maximized state or even resizes already. More important surfaces should be able to inverse-clip I'm not sure what you mean by that. I want to allow clients to do partially off-screen surfaces by either making the buffer only be the clipped area, or making the buffer be full-sized and drawing everything, or by making the buffer be full-sized and only drawing the clipped area. To avoid need for the client and shell to agree on the meaning of lots of bits, and to allow the client to choose the approach arbitrarily and dynamically change it, I would allow the client to tell the shell that the window is clipped to a larger rectangle than the wl_surface actually has. In a correctly-working client all this extra area is not really visible (ie it is outside the boundaries of the output, or hidden behind panels). Maximize would be done by the client telling the shell how thick the edges of the window are (the part that is not seen when maximized). The shell would maximize a window by
Re: Sub-surface protocol
Hi, On 6 December 2012 04:43, Tiago Vignatti tiago.vigna...@linux.intel.comwrote: indeed. On my less intrusive draft of subsurface, I've first started brainstorming the input focus behavior [0]. That's quite useful for the video player example that wants some kind of input control or a dialog stick window that might not. So we'll need a way to tell which subsurface gets the input focus. The way I did was enumerating methods for the subsurface, like transient for passing away the focus and child for a regular surface that wants it.. not a nice name, I agree. Unless I'm missing something, we can just do this right now (with the wl_subsurface draft I posted) by emptying the input region of the child surface. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
Hi, On 6 December 2012 01:32, Pekka Paalanen ppaala...@gmail.com wrote: Clipping The term sub-surface sounds like a sub-window, which may cause one to think, that it will be clipped to the parent surface area. I do not think we should force or even allow such clipping. Forcing the clipping would waste memory: for every pixel in sub-surfaces there would have to be a pixel in the parent surface. For instance, drawing window decorations in four sub-surfaces around the main content would not only waste memory, but also not solve the problem of GL applications, where one needs to reserve a margin for the decorations. (Hello, window decoration libraries? :-) Right, so we're coming at this from completely different angles then. I approached this more or less purely from the embedding sense, where you either have a video element (e.g. GStreamer's waylandvideosink) or a plugin such as Flash in one parent window. In this case, you do absolutely want clipping. If I was looking at it from the point of view of having the decorations and window content have a subsurface relationship, I'd be much more tempted to express it as a group of surfaces instead. In this case, rather than having a strict parent/child relationship, we'd have a group of surfaces with an internal stacking order, and no concept of a parent at all. This avoids the need for either disabling occlusion to the parent's bounding box (which is IMO dangerous and misleading with a parent/child scheme, even if it does allow for a usecase it wasn't in any way designed for), or having the parent wl_surface be effectively larger than it actually is. Something to think about. Merely allowing clipping is just not the Wayland style. If the *client* wants to clip something, it should not draw it in the first place. For my usecases, clipping is pretty important since you don't want your video content creeping out over the window borders. _Not_ allowing clipping is wasteful in some cases: consider a Flash video in your browser, which is scrolled half-offscreen. You'd have to copy the entire decoded buffer into an intermediate buffer, then copy only the visible part of your data into a wl_buffer. Every time you scrolled, you'd have to create a differently-sized wl_buffer, involving still more copies. That being said, I don't think clipping to the parent's bounds is sufficient for this (since you also don't want the child surface drawing all over your decorations). I think we can get around this though: see below. * The pending state of all sub-surfaces is applied only on the parent surface commit. Sub-surface commit is a no-op. This will nicely guarantee atomicity. The set of surfaces forms a single application window, and all the surfaces must be updated in sync to avoid flickering a bad composite. Resizing will absolutely require this to work reliably and without glitches. The downside is, that e.g. a video sink component cannot just happily push buffers to its own sub-surface. It has to signal the application to commit the parent surface for every single frame. Some flash, plugin, or video APIs might not support that. Do we care? It's some amount of overhead, and for the 99% case of embedded media, unnecessary if you go with #3 below. * Implicit fallback to requiring parent commit on sub-surface configuration change. (by daniels) This means, that sub-surface commit works like a normal surface's commit, if the sub-surface configuration (position, buffer size) does not change. When the configuration changes, the compositor will apply the new sub-surface state only on the parent surface commit, and until then it will not complete the sub-surface commit. This will allow a video sink to run on its own, as long as the sub-surface size or position, e.g. via wl_surface.attach, does not change. Resizing can be handled by the application first signalling a resize to the video sink, and then repainting and committing the parent surface. Both the parent and the sub-surface will be updated atomically to the screen. (the application signals a resize to the video sink, the video sink reconfigures and commits its child surface, signals the parent that it's ready to continue, then the parent commits its own surface) For debugging, if the video sink does something unexpected, like pass non-zero x,y to wl_surface.attach, the problem will be clearly visible as the video will stop until the parent surface updates. The downside is, that the client cannot force a sync to the parent surface commit without changing the sub-surface configuration. Therefore sub-surfaces animating in sync will be very awkward to implement, though one can argue, that such applications should not be using sub-surfaces to begin with. Yeah, I don't think there's any justification for applications wanting to animate multiple subsurfaces in sync with each other, at least while they retain the same configuration. * An explicit
Re: Sub-surface protocol
On Wed, 05 Dec 2012 15:43:18 -0200 Tiago Vignatti tiago.vigna...@linux.intel.com wrote: Hi, On 12/05/2012 12:32 PM, Pekka Paalanen wrote: I have not even thought about sub-surfaces' implications to input handling or the shell yet. Sub-surfaces probably need to be able to receive input. The shell perhaps needs a bounding box of the set of surfaces to be able to pick an initial position for the window, etc. indeed. On my less intrusive draft of subsurface, I've first started brainstorming the input focus behavior [0]. That's quite useful for the video player example that wants some kind of input control or a dialog stick window that might not. So we'll need a way to tell which subsurface gets the input focus. The way I did was enumerating methods for the subsurface, like transient for passing away the focus and child for a regular surface that wants it.. not a nice name, I agree. That being said, I'm afraid that the input focus together with the configuration, positing and stacking, belongs more to the shells than the compositor itself, which should only control the attach - damage - commit. Just the concept of windows itself doesn't sound that good for a simple shell for instance, so at the moment it's being hard for me draw the picture why this is not an extension of the shell_surface. That's why I started drafting from this other direction. Well, my first attack on the input management would be to do nothing special. :-) I.e. a sub-surface and a parent surface are all just wl_surfaces, and would not differ at all from a window in a single wl_surface case. On the input side, the client would just deal with a set of surfaces instead of one surface. It would get leave and enter events as usual, when the pointer or kbd focus changes from sub-surface to another. If a sub-surface wants or does not want input, the client sets the input region accordingly. The client (toolkit) just needs to be prepared to combine input events from the sub-surfaces, and e.g. direct keyboard events to the whole window first, rather than starting with just the one wl_surface. On the compositor side, I would not have to modify the input code at all. Weston's core would not need to deal much with composite windows, i.e. a collection of a parent wl_surface and a set of sub-surfaces. It only needs to maintain the associations, and maybe offer helpers like the bounding box computation. Therefore it should not really need the concept of a window. Window management is the shell's job, and for instance the force-move binding is a shell feature, which can look up and move the whole composite window, when the user tries to move a sub-surface. So, instead of trying to be smart about input foci between the sub-surfaces and the parent, I would just punt that to the clients' input handling. I hope that works also in practice. Anyways, nice write-up and summary Pekka. Thanks for bringing this up! Thank you! [0] http://cgit.freedesktop.org/~vignatti/wayland/commit/?h=xwm-client-OLD http://cgit.freedesktop.org/~vignatti/weston/commit/?h=xwm-client-OLD I see, right. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Wed, 05 Dec 2012 22:45:14 -0800 Bill Spitzak spit...@gmail.com wrote: Pekka Paalanen wrote: I am currently looking into sub-surfaces, first to sketch the protocol extension, and I have some open questions. I decided to write an exhaustive document, so we would all be on the same page, and also to clarify my own thoughts. Glad to see this is being worked on. I have been trying to figure this out as well, so I have a number of comments and ideas: Our design principles seem to be very different, which becomes obvious a few paragraphs below. The main thing I would do is merge this with the popup/menu/transient windows. Subsurfaces are simply child windows that are stuck to the main one: the shell is not allowed to place another window between them and their parent. The reason for this is that everything done to make subsurfaces not blink or move incorrectly is just as useful for floating menus and transient windows, and everything for handling events and grabs for transient windows is useful for subwindows. Surfaces and windows are different concepts. Windows do not exist in the core, they exist in the shell. Some of the window types, like perhaps tooltips, could be implemented as sub-surfaces instead, since they do not have additional semantics, unlike for instance a menu which needs(?) a grab. If there are any semantics or additional behaviour involved, as opposed to simply just being a piece of a solid window, it needs shell protocol and is not suitable as simply a sub-surface. In my mind, sub-surface is a role for a surface, and so excludes any shell specific behaviour as those are different roles. Just like the pointer image role. In my design this is part of wl_shell_surface, not the wl_surface. There is no difference between children and main surfaces, in fact they can change from one to the other. The new request would be called set_parent and it is done to the child wl_shell_surface. The request has the following arguments: (all the binary ones would be bits in a single integer): Right. I went intentionally for the core protocol instead of shell protocol, since I think this is a more low-level feature. It can be shell agnostic, and useful on non-desktop shells. Also, I want to keep it as simple as possible. No re-parenting, no nesting, no role changes. After all, the primary aim is to offload some of the window content compositing to the server, to allow the use of special hardware. I will probably drop the idea of scaling for now, too, and see how it looks like. Our design starting points are very different, core versus shell. Clipping Subsurfaces are exactly like transient windows and thus are not clipped by the parent, exactly like you propose. For instance, drawing window decorations in four sub-surfaces around the main content would not only waste memory, but also not solve the problem of GL applications, where one needs to reserve a margin for the decorations. (Hello, window decoration libraries? :-) Did you make a typo? It sounds like 4 surrounding surfaces *will* solve the problems for GL applications, right? No, the above paragraph means, that if we force sub-surfaces to be clipped to the parent surface, it will not solve the problem with GL; because the parent surface must be larger than the GL content, so that decoration surfaces do not get clipped away. If we do not clip, all is good, problem solved. The shell needs to pretend the window is all of these together. I think this has some implications for how the current maximize and fullscreen work. My preference is that the client can just tell it to act like the main window is a particular size, while the buffer is a smaller rectangle. I also want this to allow maximize and docking to clip off edges but not require the client to allocate buffer space or draw the clipped edges. Yes, I haven't thought about how that could work. I hope I can find a way to manage it by using a configure event only for the parent surface, when the composite surface is being resized, and then leave it up to the client to a) use a smaller parent surface than configure suggests for, and b) adjust the sub-surfaces so that the composite window's size becomes the suggested size. If it is about fullscreen, the client could also just remove the decoration sub-surfaces, and go properly fullscreen with a single wl_surface; the parent. It feels like that should all fall into place quite nicely, but we'll see. The shell will probably not even know about sub-surface objects at all, just like it does not know about pointer image surfaces, IIRC. Committing changes I think it may work that a commit on a parent is an implied commit on all the children. To make a set of child surfaces all resize in unison, change them all but don't call commit on any, and call commit on the main window after all are updated. An async video player process could call commit on the child
Re: Sub-surface protocol
On Fri, 7 Dec 2012 12:34:46 +0200 Pekka Paalanen ppaala...@gmail.com wrote: On Wed, 05 Dec 2012 22:45:14 -0800 Bill Spitzak spit...@gmail.com wrote: Committing changes I think it may work that a commit on a parent is an implied commit on all the children. To make a set of child surfaces all resize in unison, change them all but don't call commit on any, and call commit on the main window after all are updated. Actually, I do like this one. Does anyone have anything against it? An async video player process could call commit on the child surface for it's changes as the frames play. It could even change the resolution by changing the local transform of the surface, though it has to result in the same-sized rectangle as before. It is possible it may be very difficult for toolkits to be written to defer the commits for the window pieces. In that case some of your ideas may be necessary. Yes, it is otherwise a nice idea, but does not easily work for the independent video sink case. We probably need more information on how these video sinks and flash plugin APIs work, whether they have any API for synchronizing resizes, or do they just expect the window system to notify them directly like I assume X11 does. Or should we just design a protocol that is nice, clean, and works, and expect clients to deal with it. That would be the Wayland way. Daniel, Kristian? This really needs suggestions from toolkit implementors, too. Thanks pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Fri, Dec 07, 2012 at 10:31:20AM +0200, Pekka Paalanen wrote: On Wed, 05 Dec 2012 15:43:18 -0200 Tiago Vignatti tiago.vigna...@linux.intel.com wrote: Hi, On 12/05/2012 12:32 PM, Pekka Paalanen wrote: I have not even thought about sub-surfaces' implications to input handling or the shell yet. Sub-surfaces probably need to be able to receive input. The shell perhaps needs a bounding box of the set of surfaces to be able to pick an initial position for the window, etc. indeed. On my less intrusive draft of subsurface, I've first started brainstorming the input focus behavior [0]. That's quite useful for the video player example that wants some kind of input control or a dialog stick window that might not. So we'll need a way to tell which subsurface gets the input focus. The way I did was enumerating methods for the subsurface, like transient for passing away the focus and child for a regular surface that wants it.. not a nice name, I agree. That being said, I'm afraid that the input focus together with the configuration, positing and stacking, belongs more to the shells than the compositor itself, which should only control the attach - damage - commit. Just the concept of windows itself doesn't sound that good for a simple shell for instance, so at the moment it's being hard for me draw the picture why this is not an extension of the shell_surface. That's why I started drafting from this other direction. Well, my first attack on the input management would be to do nothing special. :-) I.e. a sub-surface and a parent surface are all just wl_surfaces, and would not differ at all from a window in a single wl_surface case. On the input side, the client would just deal with a set of surfaces instead of one surface. It would get leave and enter events as usual, when the pointer or kbd focus changes from sub-surface to another. If a sub-surface wants or does not want input, the client sets the input region accordingly. The client (toolkit) just needs to be prepared to combine input events from the sub-surfaces, and e.g. direct keyboard events to the whole window first, rather than starting with just the one wl_surface. On the compositor side, I would not have to modify the input code at all. Weston's core would not need to deal much with composite windows, i.e. a collection of a parent wl_surface and a set of sub-surfaces. It only needs to maintain the associations, and maybe offer helpers like the bounding box computation. Therefore it should not really need the concept of a window. Window management is the shell's job, and for instance the force-move binding is a shell feature, which can look up and move the whole composite window, when the user tries to move a sub-surface. So, instead of trying to be smart about input foci between the sub-surfaces and the parent, I would just punt that to the clients' input handling. Yup, I fully agree. Kristian I hope that works also in practice. Anyways, nice write-up and summary Pekka. Thanks for bringing this up! Thank you! [0] http://cgit.freedesktop.org/~vignatti/wayland/commit/?h=xwm-client-OLD http://cgit.freedesktop.org/~vignatti/weston/commit/?h=xwm-client-OLD I see, right. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On Fri, Dec 07, 2012 at 01:07:33PM +0200, Pekka Paalanen wrote: On Fri, 7 Dec 2012 12:34:46 +0200 Pekka Paalanen ppaala...@gmail.com wrote: On Wed, 05 Dec 2012 22:45:14 -0800 Bill Spitzak spit...@gmail.com wrote: Committing changes I think it may work that a commit on a parent is an implied commit on all the children. To make a set of child surfaces all resize in unison, change them all but don't call commit on any, and call commit on the main window after all are updated. Actually, I do like this one. Does anyone have anything against it? I always thought that the commit on the parent surface would trigger the sub-surfaces, but not replace sub-surface commit. Only commit those sub-surfaces that had commit called on them. This way, a library or component can manage a sub-surface and doesn't need to know whether it's a sub-surface (and thus doesn't need commit) or if it's a top-level surface (and needs a commit). Another way to think about it is that the sub-surface commit is latched state of the parent surface. Kristian ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
On 12/07/2012 06:39 AM, Kristian Høgsberg wrote: On Fri, Dec 07, 2012 at 01:07:33PM +0200, Pekka Paalanen wrote: On Fri, 7 Dec 2012 12:34:46 +0200 Pekka Paalanen ppaala...@gmail.com wrote: On Wed, 05 Dec 2012 22:45:14 -0800 Bill Spitzak spit...@gmail.com wrote: Committing changes I think it may work that a commit on a parent is an implied commit on all the children. To make a set of child surfaces all resize in unison, change them all but don't call commit on any, and call commit on the main window after all are updated. Actually, I do like this one. Does anyone have anything against it? I always thought that the commit on the parent surface would trigger the sub-surfaces, but not replace sub-surface commit. Only commit those sub-surfaces that had commit called on them. This way, a library or component can manage a sub-surface and doesn't need to know whether it's a sub-surface (and thus doesn't need commit) or if it's a top-level surface (and needs a commit). Another way to think about it is that the sub-surface commit is latched state of the parent surface. I believe he wants to avoid the need for a video-playback library to need a way to tell the main program you have to call commit on your surface because I just changed the video frame. Changes in the transform from parent to child would require a commit on the parent (so that when the client is resized, the frame and video player resize in unison). But changing what buffer to show as the child would only require a child commit. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: Sub-surface protocol
Pekka Paalanen wrote: I am currently looking into sub-surfaces, first to sketch the protocol extension, and I have some open questions. I decided to write an exhaustive document, so we would all be on the same page, and also to clarify my own thoughts. Glad to see this is being worked on. I have been trying to figure this out as well, so I have a number of comments and ideas: The main thing I would do is merge this with the popup/menu/transient windows. Subsurfaces are simply child windows that are stuck to the main one: the shell is not allowed to place another window between them and their parent. The reason for this is that everything done to make subsurfaces not blink or move incorrectly is just as useful for floating menus and transient windows, and everything for handling events and grabs for transient windows is useful for subwindows. In my design this is part of wl_shell_surface, not the wl_surface. There is no difference between children and main surfaces, in fact they can change from one to the other. The new request would be called set_parent and it is done to the child wl_shell_surface. The request has the following arguments: (all the binary ones would be bits in a single integer): parent: the parent surface, or None to make this a top-level surface (which make it ignore all the other controls). Parent probably has to belong to the same client as child. If a loop is made the results are undefined but the shell must not crash. It may also be possible to set the parent to a process object which acts like None but is used by panels to group windows into a single indicator. below: if true the shell must always make this surface be below the parent, if false then the shell must always make this surface be above the parent. subsurface: if true then the shell must not allow any surface that has a different parent from being between this surface and the parent. transform: transform between parent surface and this one locked: if true then changes to the transforms of the parent are applied to this one, so they always move in unison adjust: various flags to tell the shell to adjust the transform so the child is not clipped by panels and outputs. Matches the current popup request. grab: various flags to emulate the current popup grab request. probably lots of other flags, these can be added without changing the protocol version as long as there are less than 32 of them! The child (and any necessary children of it) is moved up/down in the stacking order until it reaches the first position that agrees with the rules. The window stacking would not change until a commit is done to the parent (or the child if parent is None). If there was an old parent then a commit has to be done to the old parent too. A raise of a surface would find every above child plus every below subsurface child, and put them all as high in the main surface's layer as possible without changing their current order and without changing the order of any other surfaces. Raise is a request from the client (the shell can only send a raise-requested event to the client). The effects of raise are deferred until a commit on the surface, and are mixed with any set_parent requests the client does before the commit. The main purpose is so that a transient window can appear to belong to more than one main window, by the client changing it's parent to the raised one. This seems to be far easier than supporting an arbitrary DAG of child windows. Clipping Subsurfaces are exactly like transient windows and thus are not clipped by the parent, exactly like you propose. For instance, drawing window decorations in four sub-surfaces around the main content would not only waste memory, but also not solve the problem of GL applications, where one needs to reserve a margin for the decorations. (Hello, window decoration libraries? :-) Did you make a typo? It sounds like 4 surrounding surfaces *will* solve the problems for GL applications, right? The shell needs to pretend the window is all of these together. I think this has some implications for how the current maximize and fullscreen work. My preference is that the client can just tell it to act like the main window is a particular size, while the buffer is a smaller rectangle. I also want this to allow maximize and docking to clip off edges but not require the client to allocate buffer space or draw the clipped edges. Committing changes I think it may work that a commit on a parent is an implied commit on all the children. To make a set of child surfaces all resize in unison, change them all but don't call commit on any, and call commit on the main window after all are updated. An async video player process could call commit on the child surface for it's changes as the frames play. It could even change the resolution by changing the local transform of the surface, though it has to result in the
Re: Sub-surface protocol
Hi, On 12/05/2012 12:32 PM, Pekka Paalanen wrote: I have not even thought about sub-surfaces' implications to input handling or the shell yet. Sub-surfaces probably need to be able to receive input. The shell perhaps needs a bounding box of the set of surfaces to be able to pick an initial position for the window, etc. indeed. On my less intrusive draft of subsurface, I've first started brainstorming the input focus behavior [0]. That's quite useful for the video player example that wants some kind of input control or a dialog stick window that might not. So we'll need a way to tell which subsurface gets the input focus. The way I did was enumerating methods for the subsurface, like transient for passing away the focus and child for a regular surface that wants it.. not a nice name, I agree. That being said, I'm afraid that the input focus together with the configuration, positing and stacking, belongs more to the shells than the compositor itself, which should only control the attach - damage - commit. Just the concept of windows itself doesn't sound that good for a simple shell for instance, so at the moment it's being hard for me draw the picture why this is not an extension of the shell_surface. That's why I started drafting from this other direction. Anyways, nice write-up and summary Pekka. Thanks for bringing this up! [0] http://cgit.freedesktop.org/~vignatti/wayland/commit/?h=xwm-client-OLD http://cgit.freedesktop.org/~vignatti/weston/commit/?h=xwm-client-OLD Tiago ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
