Re: [RFC] wl_surface scale and crop protocol extension
Pekka Paalanen wrote: On Fri, 03 May 2013 12:11:17 -0700 Bill Spitzak spit...@gmail.com wrote: Pekka Paalanen wrote: What chip do you have in mind, that can do arbitrary matrix-based transforms during an overlay scanout? That just means the surface cannot use the overlay, or the compositor has to use an intermediate image. There are lots of other reasons the surface does not use the overlay, such as not being the top one. Ok, so there is no use for what you suggested. Very good. Can you explain what you are talking about? Hardware that can't apply an arbitrary transform during overlay can often do an arbitrary transform when doing an over composite used for the non-overlay windows. I certainly expect the scaling api to be available for these even if it means the window will not use the overlay hardware. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Fri, 03 May 2013 12:11:17 -0700 Bill Spitzak spit...@gmail.com wrote: Pekka Paalanen wrote: What chip do you have in mind, that can do arbitrary matrix-based transforms during an overlay scanout? That just means the surface cannot use the overlay, or the compositor has to use an intermediate image. There are lots of other reasons the surface does not use the overlay, such as not being the top one. Ok, so there is no use for what you suggested. Very good. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Documentation cross-reference (Re: [RFC] wl_surface scale and crop protocol extension)
On Fri, 3 May 2013 19:16:41 +0100 Daniel Stone dan...@fooishbar.org wrote: On 3 May 2013 18:50, Bill Spitzak spit...@gmail.com wrote: Currently one of the things you can do to a wl_surface is that you can find the wl_shell object and ask it to create a wl_shell_surface object given a wl_surface id, and then you can do things to this wl_shell_surface object. However this is not listed under wl_surface, instead it is listed under two other sections with no links to them (there are backward links but that is pretty useless): wl_shell and wl_shell_surface. I think this is extremely confusing and was probably one of the biggest obstacles I had trying to figure wayland out. What I would like to see is that under wl_surface you see a section that says something like wl_scaler:. Listed under that are the methods that are currently listed under wl_surface_scaler. The title wl_scaler: implies all this information: the actual calling convention is to find the global wl_scaler object, call a method called get_scaler_surface with the wl_surface id, and that returns a new object called a wl_scaler_surface, and you call the listed methods on *this* object. The documentation is now where a user can find it, and a lot of boilerplate is elided. One thing that would help a lot is for there to be a standard as to whether a wl_foo global object, when asked to create an object for a wl_bar object, that the object is called wl_foo_bar and not wl_bar_foo. Currently the wl_shell and wl_scaler disagree about this. Since wl_shell_surface is in much more than wl_surface_scaler use I recommend standardizing on putting the global object's type first. OK, so it pretty much looks like just cosmetic/documentation changes - none of which I'm against at all. Obviously we can't break existing protocol though (such as renaming wl_shell_surface to wl_surface_shell), but we can improve the documentation. I'm sure patches for that would be gratefully accepted. How about automatically generating a cross-reference list for each object class, which would contain every request and event that has this object class as an argument (and make those links to the corresponding doc)? This would be for the html/pdf docs. Now, who would implement that... Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Thu, 02 May 2013 12:16:23 -0700 Bill Spitzak spit...@gmail.com wrote: Jason Ekstrand wrote: Agreed. Sending transform matrices or the like has HUGE rounding problems. Particularly when we're using wl_fixed which is 24.8. Other methods would require adding rounding conventions etc. I was assuming IEEE 32-bit floating point would be in the api for at least the ABCD of the matrix (the XY could be wl_fixed I suppose). If you looked at the Wayland specification, you'd see that there are no floats. However what you call rounding conventions are needed even for the proposed integer api, and this is also why a source rectangle is needed. It has nothing to do with inaccuracy. It is because filtered needs to know it, since the filter will extend outside the source region (it will for down-scales for any reasonable filter, and even for up-scaling for mitchell/sync style filters). Samples outside this region must be treated specially and this rounding convention must be defined by wayland (almost certainly you want to define it as using the nearest pixel inside the region). That's not a rounding convention. Rounding convention is about converting real numbers into integers, and I really meant that. Not filtering. And I really do *not* want to specify a filtering method at *any* level, because whatever we specify, there will always be hardware that does it differently. I will not write down any guarantees of the resulting quality, or the scaling method used, because I do not want to exclude any scaling hardware. So yes a source rectangle must be provided in the api, but not for the reason you think. The destination rectangle is then a nice way to produce rational fractions for scale and also provide a wl_surface size that is different than the buffer size. My reason for using a source rectangle is because I want to be able to crop. I would make one change to the proposal because I think arbitrary transforms will need to be considered some day. The source rectangle has to remain orthogonal in source space (otherwise it is useless for filtering), while the destination has to be orthogonal in surface space (since it also controls a lot of other wl_surface details), therefore any future arbitrary transform must be between these. I think the current buffer transform should be considered the start of any future arbitrary transform, so for that reason the source rectangle should be in actual buffer pixels, not in buffer transform space. What chip do you have in mind, that can do arbitrary matrix-based transforms during an overlay scanout? - pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Thu, 2 May 2013 09:42:42 -0500 Jason Ekstrand ja...@jlekstrand.net wrote: On Thu, May 2, 2013 at 5:51 AM, Pekka Paalanen ppaala...@gmail.com wrote: On Tue, 30 Apr 2013 10:54:25 -0500 Jason Ekstrand ja...@jlekstrand.net wrote: ... On Tue, Apr 30, 2013 at 2:33 PM, Pekka Paalanen ppaala...@gmail.com wrote: ... This interface allows to define the source rectangle (src_x, src_y, src_width, src_height) from where to take the wl_buffer contents, and scale that to destination size (dst_width, dst_height). This state is double-buffered, and is applied on the next wl_surface.commit. Before the first set request, the wl_surface still behaves as if there was no crop and scale state. That is, no scaling is applied, and the surface size is the buffer size. On compositing, source rectangle coordinates are evaluated after wl_surface.set_buffer_transform is evaluated. This means that changing the buffer transform and correspondingly the client rendering does not require sending new source rectangle coordinates to keep the exact same image source rectangle. In other words, the source rectangle is given in the not-scaled-and-cropped surface coordinates, not buffer data coordinates. I agree with Zhi, this needs to be re-worded Yeah, did you understand what I was trying to explain? Any suggestions? If I understood correctly, you meant the sensible thing. i.e., buffers are in (possibly transformed) buffer coordinates while surfaces are in surface coordinates. In other words, the crop/scale is applied after the buffer transform. How about this (actually a paragraph higher): The source rectangle is specified in (possibly transformed) buffer coordinates. This means that changing the buffer transform and correspondingly the client rendering does not require sending new source rectangle coordinates. In other words, the source rectangle is given in the coordinates that the surface would have without the scaled-and-cropped transformation. This is a little better, avoids evaluated, but still I wonder if we could separate buffer coordinates and orientation-transformed buffer coordinates better. We have: A. Buffer pixel coordinates, which are essentially equivalent to byte addresses of pixels in the buffer. apply buffer transform to get: B. oriented buffer coordinates (I wish I would have suggested the term orientation when the buffer transform was proposed, now we have several transformations.) apply crop scale to get: C. surface coordinates, i.e. the local coordinates where all window management and input happens And the rest is private to a compositor. Before crop scale, C was equal to B, and now it is not. If we just had a term to use for B consistently, describing everything would be a lot easier. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Thu, 2 May 2013 14:43:38 -0500 Jason Ekstrand ja...@jlekstrand.net wrote: On Thu, May 2, 2013 at 1:22 PM, Daniel Stone dan...@fooishbar.org wrote: Hi, On 2 May 2013 15:42, Jason Ekstrand ja...@jlekstrand.net wrote: Ok, I see it now. Sorry, but I missed it on my first read-through. Yes, it fixes the problem, but in an extremely confusing way. The reason I say it is confusing is because it inherently mixes buffer and surface coordinate systems. I really think we need to isolate buffer coordinates from surface coordinates more. Perhaps what we need is two requests: set_source_rect and set_dest_rect and completely ignore the x and y from attach. This both provides clarity to the coordinate systems and provides a little separation between crop and scale. Yes, this little rough corner bothered me, too. I tried to weasel out of it by saying that wl_surface_scaler and wl_surface.attach must be used together, so you can use the right values. While we still use x,y from wl_surface.attach, nothing else really makes sense. The x,y must be in the surface coordinate frame. Since surface coordinates were equal to oriented buffer coordinates (see my email sent just before this one), I think we could still change the wording for wl_surface.attach specification to talk about surface coordinates instead of buffer coordinates. Would that solve your concern? Ideally, when wl_surface::commit was added, wl_surface::attach should've been broken out into wl_surface::attach and wl_surface::set_position. Oh well. Exactly. That's what my suggestion was trying to fix (at least in the transformed surface case). Would that really have made any difference? Clients would still be able to use the x,y regardless, and we need to specify how they interact with wl_surface_scaler. That being said, we can't ignore the x and y from attach, because that _moves the window_ on screen (think resizing from top left), whereas this is all about how we map the contents of a buffer into that window - totally unrelated to moving. Yes, I realize that the x and y from attach are used for moving or scaling from top or left. My suggestion was to replace the x and y in attach with an x and y in set_dest_rect. We wouldn't be losing functionality, just moving it in a certain case. This way surface coordinates are kept with surfaces and buffer coordinates are kept with buffers. In this case, the defined behavior would be that if you create a scaler for a surface, the x and y in attach are disabled and set_dest_rect takes over. This way older clients can just use attach like they used to and clients that use surface scalers use the destination rectangle. It's not a perfect fix, but I think it moves in the right direction. And, for what it's worth, it doesn't make things significantly more complicated because anything that's going to scale from the top-left will have to mess with both the surface and the scaler anyway. Yes, that would be one option, and it would support the I am forcing that other component's surface size better, since nothing in wl_surface would affect what is set in wl_surface_scaler anymore. Except the wl_buffer size, but if we *really* want to go that way, we could say that areas outside of a buffer are transparent. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Hi, On 2 May 2013 20:33, Bill Spitzak spit...@gmail.com wrote: Daniel Stone wrote: I also think all of wl_shell should be a core requirement. Not all compositors are user sessions. Think about nested compositors for browsers, or capture, or also very stripped-down usecases where they really don't want to have to deal with this kind of thing. For simple displays like one that is a single full-screen window always, the api works in that raises determine which window is visible, and attempts to resize or turn off full-screen are ignored. Sure, it's a thing you can do, but it's a lot more typing and intrinsically discourages people from making nested or even just simple compositors. We want to lower the barrier to entry here, rather than essentially just forcing everyone to use Weston always. Though nested compositors are an interesting idea, it is clear from how the subsurfaces are being designed that it is not felt that nested compositors are not really useful. The biggest problem I see is that the nested compositor probably loses any high-speed optimizations that only the real compositor can do. I really suspect the only use of a nested compositor will be to test a wayland server inside a window on another one. Subsurfaces are being designed for in-process cases, such as a media player inside a browser. Foreign surfaces are intended for cross-process buffer/surface sharing, but I think nested compositors is actually better for the usecase I had in mind, which is WebKit2. So it would be nice if they weren't unnecessarily complex to implement. I don't think it's an unreasonable requirement, and really like the design it has at the moment, where attaching the scaler object suppresses the resize-on-attach behaviour, and destroying it reverts to previous. It's pretty elegant, and totally in the vein of wl_shell_surface stacking on top of wl_surface. I don't see how inventing more elaborate extension mechanisms on top of our existing extension mechanism helps anything. No what I propose is to figure out how to document it so that the wayland api documentation is easier to follow. Make the rules for these sub-api objects match. Then I would expect to see the set-scaling listed under the wl_surface api. There would be a small indication that this api is through the wl_scaler subapi and the rest of the details would be defined in a single section of the wayland documentation. I don't think having 'sub-extensions' makes anything more clear: in fact, exactly the opposite. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Daniel Stone wrote: Subsurfaces are being designed for in-process cases, such as a media player inside a browser. Foreign surfaces are intended for cross-process buffer/surface sharing, but I think nested compositors is actually better for the usecase I had in mind, which is WebKit2. So it would be nice if they weren't unnecessarily complex to implement. The final compositor is able to do the fully optimized compositing of surfaces into the output frame buffer, including all the work various people have done to not composite obscured areas. I fail to see how a nested compositor can take advantage of this except by transmitting all the individual surfaces to the final compositor. This means that buffers and buffer allocation have to be transparently passed through the intermediate compositors. I don't think this has been addressed, and it seems really difficult compared to just reusing the existing single connection to the server. The proposal where the parent task creates subsurfaces and then does some setup so the child task can draw them seems far more like what webkit is going to want and use. I don't think having 'sub-extensions' makes anything more clear: in fact, exactly the opposite. I'm not describing this right. What I want is to be able to read the documentation, and see a list of all the things you can do to a wl_surface. Currently one of the things you can do to a wl_surface is that you can find the wl_shell object and ask it to create a wl_shell_surface object given a wl_surface id, and then you can do things to this wl_shell_surface object. However this is not listed under wl_surface, instead it is listed under two other sections with no links to them (there are backward links but that is pretty useless): wl_shell and wl_shell_surface. I think this is extremely confusing and was probably one of the biggest obstacles I had trying to figure wayland out. What I would like to see is that under wl_surface you see a section that says something like wl_scaler:. Listed under that are the methods that are currently listed under wl_surface_scaler. The title wl_scaler: implies all this information: the actual calling convention is to find the global wl_scaler object, call a method called get_scaler_surface with the wl_surface id, and that returns a new object called a wl_scaler_surface, and you call the listed methods on *this* object. The documentation is now where a user can find it, and a lot of boilerplate is elided. One thing that would help a lot is for there to be a standard as to whether a wl_foo global object, when asked to create an object for a wl_bar object, that the object is called wl_foo_bar and not wl_bar_foo. Currently the wl_shell and wl_scaler disagree about this. Since wl_shell_surface is in much more than wl_surface_scaler use I recommend standardizing on putting the global object's type first. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Hi, Again I fear we're drifting massively off-topic, but here we go ... On 3 May 2013 18:50, Bill Spitzak spit...@gmail.com wrote: Daniel Stone wrote: Subsurfaces are being designed for in-process cases, such as a media player inside a browser. Foreign surfaces are intended for cross-process buffer/surface sharing, but I think nested compositors is actually better for the usecase I had in mind, which is WebKit2. So it would be nice if they weren't unnecessarily complex to implement. The final compositor is able to do the fully optimized compositing of surfaces into the output frame buffer, including all the work various people have done to not composite obscured areas. I fail to see how a nested compositor can take advantage of this except by transmitting all the individual surfaces to the final compositor. If that's the goal, then yes. For some usecases however, doing the compositing before and sending one large buffer to the compositor is desirable. This means that buffers and buffer allocation have to be transparently passed through the intermediate compositors. I don't think this has been addressed, and it seems really difficult compared to just reusing the existing single connection to the server. Again, yes, sometimes. For most stacks, passing the allocation through is fairly trivial, as the allocation is client-driven anyway: the client creates the buffer and then just tells the compositor how to access it. EGL implementations can already proxy buffer allocation with zero interference from us, however: they already have a connection to the parent compositor's display, so they can just pass all allocation requests they might get, up the chain. We do need a little bit more API to support this, but nothing too catastrophic, and nothing more implementationally onerous than the stream-of-buffers foreign surface proposal demanded. The proposal where the parent task creates subsurfaces and then does some setup so the child task can draw them seems far more like what webkit is going to want and use. I really don't think it is. I don't think having 'sub-extensions' makes anything more clear: in fact, exactly the opposite. I'm not describing this right. What I want is to be able to read the documentation, and see a list of all the things you can do to a wl_surface. Currently one of the things you can do to a wl_surface is that you can find the wl_shell object and ask it to create a wl_shell_surface object given a wl_surface id, and then you can do things to this wl_shell_surface object. However this is not listed under wl_surface, instead it is listed under two other sections with no links to them (there are backward links but that is pretty useless): wl_shell and wl_shell_surface. I think this is extremely confusing and was probably one of the biggest obstacles I had trying to figure wayland out. What I would like to see is that under wl_surface you see a section that says something like wl_scaler:. Listed under that are the methods that are currently listed under wl_surface_scaler. The title wl_scaler: implies all this information: the actual calling convention is to find the global wl_scaler object, call a method called get_scaler_surface with the wl_surface id, and that returns a new object called a wl_scaler_surface, and you call the listed methods on *this* object. The documentation is now where a user can find it, and a lot of boilerplate is elided. One thing that would help a lot is for there to be a standard as to whether a wl_foo global object, when asked to create an object for a wl_bar object, that the object is called wl_foo_bar and not wl_bar_foo. Currently the wl_shell and wl_scaler disagree about this. Since wl_shell_surface is in much more than wl_surface_scaler use I recommend standardizing on putting the global object's type first. OK, so it pretty much looks like just cosmetic/documentation changes - none of which I'm against at all. Obviously we can't break existing protocol though (such as renaming wl_shell_surface to wl_surface_shell), but we can improve the documentation. I'm sure patches for that would be gratefully accepted. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Pekka Paalanen wrote: What chip do you have in mind, that can do arbitrary matrix-based transforms during an overlay scanout? That just means the surface cannot use the overlay, or the compositor has to use an intermediate image. There are lots of other reasons the surface does not use the overlay, such as not being the top one. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On 05/01/2013 02:16 AM, John Kåre Alsaker wrote: On Tue, Apr 30, 2013 at 10:49 PM, Jason Ekstrand ja...@jlekstrand.net mailto:ja...@jlekstrand.net wrote: On Tue, Apr 30, 2013 at 2:29 PM, Bill Spitzak spit...@gmail.com mailto:spit...@gmail.com wrote: I'm not clear on why Wayland's design requires adding 2 dummy objects to the api (in this case the wl_scalar factory and the per-surface wl_surface_scalar) rather than just adding new methods to the wl_surface object. It seems wasteful for clients and servers to track this whole constellations of id's for the same wl_surface. It also looks too much like X where I had to keep the window, visual, visual id, screen, colormap, gc, and I forget what else, because all the X functions required different arguments types for no discernible reason when they could all be derived from the window id. I'm guessing there is a good reason for compatibility but it seems that just adding more message numbers after the already-used ones would work. To avoid collisions between multiple extensions the server could instead send an event saying the scalar messages to wl_surface start at N and this single number, rather than an per-surface extra object id, is all the client needs to remember. I agree that keeping piles of extra objects around may not be the best solution. However, we cannot simply have the scaler messages to wl_surface start at N type message. This would require substantially altering the wire protocol along with libwayland. I think adding that would be mostly additions to libwayland/wayland-scanner and wouldn't require modifying the wire protocol at all. The protocol allows us to add new requests to a new version of an interface. For instance, the set_buffer_transform request was added to version 2 of the wl_surface interface. I believe the scaled video use case is reason enough to add this straight to wl_surface. Cheers, Ander Even if we did make those changes, I think we would end up right back where we are now having to manage extension proxies so we wouldn't save anything. We would only have to manage an extension object per connection client-side and only one object for all connections server-side, which would be a huge improvement over the current state. Besides, I'm pretty sure one of Kristian's main ideas in the core wayland protocol is everything's an extension so this IS the extension system. It's more like you can add new object types which may or may not be extensions of others, not so much an extension system as a way of doing extensions. We should probably split off any plots for libwayland changes elsewhere... ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Tue, 30 Apr 2013 17:08:49 -0400 Matthias Clasen matthias.cla...@gmail.com wrote: On Tue, Apr 30, 2013 at 8:33 AM, Pekka Paalanen ppaala...@gmail.com wrote: Hi all, below is my first draft for a wl_surface scaling and cropping extension. I called it wl_scaler in the lack of a better name. It is designed similarly to the other wl_surface extensions wl_shell_surface and wl_subsurface. There probably isn't any interesting details to debate, right? ;-) I'd like to have a better name for it, and you might want the set request split into two or three, or not, but otherwise I'm quite satisfied with it. I tried to take into account all existing protocol that interacts with this. Comments? Is the language clear? Can you describe some use cases where you envision using this ? Thats always a good starting point for discussing an api. The only other comment I have is that when you talk about scaling, you really need to say something about the interpolation method. Oh right. The primary use case is for hardware video overlays, again. A particular use case is for video to be (hardware assisted) decoded in its native resolution, sent to a Wayland server, and scaled to fill a window (a sub-surface, actually), again hardware assisted. So it is mostly about the hardware accelerated zero-copy video presentation path. I do not really know about interpolation methods. Defining one would probably leave lots of hardware non-conformant, and I'd rather not do that. Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Tue, 30 Apr 2013 10:54:25 -0500 Jason Ekstrand ja...@jlekstrand.net wrote: On Tue, Apr 30, 2013 at 9:06 AM, John Kåre Alsaker john.kare.alsa...@gmail.com wrote: I'd say we should split the cropping and scaling request into two. How would you suggest doing that? I actually really like the simplicity of this box maps to that box. The boxes approach also does not have rounding issues. The destination size will be exactly what you specify, not some vague number multiplied by number, then rounded somehow. Specifying a scaling with a NULL buffer should still set the surface size, so we can have surfaces with only an input region. I really don't think this is the place for that. I want input-only surfaces too, but I don't think this is the place for that I also don't think this is the place for that. If any... I don't see a way to disable scaling and cropping, could passing 0 as width and height do that? Can't the client just set it to 0, 0, width, height, width, height easily enough? Or just destroy the scaler? Yeah, that's two ways to disable scaling and cropping, should be enough. The first one just has to be used repeatedly, if the buffer size changes. Destroying the surface_scaler returns the original 1:1 mapping. On Tue, Apr 30, 2013 at 2:33 PM, Pekka Paalanen ppaala...@gmail.com wrote: Hi all, below is my first draft for a wl_surface scaling and cropping extension. I called it wl_scaler in the lack of a better name. It is designed similarly to the other wl_surface extensions wl_shell_surface and wl_subsurface. There probably isn't any interesting details to debate, right? ;-) I'd like to have a better name for it, and you might want the set request split into two or three, or not, but otherwise I'm quite satisfied with it. I tried to take into account all existing protocol that interacts with this. Comments? Is the language clear? Thanks, pq ?xml version=1.0 encoding=UTF-8? protocol name=scaler copyright Copyright © 2013 Collabora, Ltd. Permission to use, copy, modify, distribute, and sell this software and its documentation for any purpose is hereby granted without fee, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of the copyright holders not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. The copyright holders make no representations about the suitability of this software for any purpose. It is provided as is without express or implied warranty. THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. /copyright interface name=wl_scaler version=1 description summary=surface cropping and scaling The global interface exposing surface cropping and scaling capabilities is used to instantiate an interface extension for a wl_surface object. This extended interface will then allow cropping and scaling the surface contents, effectively disconnecting the 1:1 relationship between the buffer and the surface size. /description request name=destroy type=destructor description summary=unbind from the cropping and scaling interface Informs the server that the client will not be using this protocol object anymore. This does not affect any other objects, wl_surface_scaler objects included. /description /request enum name=error entry name=scaler_exists value=0 summary=the surface already has a scaler object associated/ /enum request name=get_surface_scaler description summary=extend surface interface for crop and scale Instantiate an interface extension for the given wl_surface to crop and scale its content. If the given wl_surface already has a wl_surface_scaler object associated, the scaler_exists protocol error is raised. /description arg name=id type=new_id interface=wl_surface_scaler summary=the new scaler interface id/ arg name=surface type=object interface=wl_surface summary=the surface/ /request /interface interface name=wl_surface_scaler version=1 description summary=crop and scale interface to a wl_surface An additional interface to a wl_surface
Re: [RFC] wl_surface scale and crop protocol extension
On Tue, 30 Apr 2013 14:21:41 -0700 Bill Spitzak spit...@gmail.com wrote: scalar is often used to identify a single number in linear algebra. A different name would be better. transform might work but that also would cover the subsurface and surface transforms, perhaps that can be moved to this api as well. I never wrote scalar. - pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Tue, 30 Apr 2013 15:49:16 -0500 Jason Ekstrand ja...@jlekstrand.net wrote: On Tue, Apr 30, 2013 at 2:29 PM, Bill Spitzak spit...@gmail.com wrote: I'm not clear on why Wayland's design requires adding 2 dummy objects to the api (in this case the wl_scalar factory and the per-surface wl_surface_scalar) rather than just adding new methods to the wl_surface object. It seems wasteful for clients and servers to track this whole constellations of id's for the same wl_surface. It also looks too much like X where I had to keep the window, visual, visual id, screen, colormap, gc, and I forget what else, because all the X functions required different arguments types for no discernible reason when they could all be derived from the window id. I'm guessing there is a good reason for compatibility but it seems that just adding more message numbers after the already-used ones would work. To avoid collisions between multiple extensions the server could instead send an event saying the scalar messages to wl_surface start at N and this single number, rather than an per-surface extra object id, is all the client needs to remember. I agree that keeping piles of extra objects around may not be the best solution. However, we cannot simply have the scaler messages to wl_surface start at N type message. This would require substantially altering the wire protocol along with libwayland. Even if we did make those changes, I think we would end up right back where we are now having to manage extension proxies so we wouldn't save anything. Besides, I'm pretty sure one of Kristian's main ideas in the core wayland protocol is everything's an extension so this IS the extension system. In the case of the scaling extension, it might be possible to simply have the wl_surface_scaler interface have a single request that takes, as its first argument, a new_id containing the surface to scale. This way it can be called on the surface without the extra object. Well, it would still have the factory object as its first C function call argument, and the wl_surface the second. But if someone wrapped that into, say, a C++ class API anyway, it would all be just WlSurface methods. We just cannot design a protocol like a C API, because oops, it returned an error is equivalent to a segfault, pretty much. If we start returning non-fatal errors, we drown in a swamp of roundtrips. Like you-know-what-11. Really, the protocol bindings are not meant as a nice library API, they are just protocol bindings. Protocol objects are cheap. I think the reason to make it separate is so that it's an optional extension. If we put clipping/scaling in wl_surface, then EVERY compositor will have to support it in order to be a compliant compositor. Indeed. The other alternative, if we added methods directly into wl_surface, would be to add also an extension bitmask event, that would tell which methods are actually implemented. And then extend the bitmask, when the first 32 bits are used... Thanks, pq ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Thu, May 2, 2013 at 5:51 AM, Pekka Paalanen ppaala...@gmail.com wrote: On Tue, 30 Apr 2013 10:54:25 -0500 Jason Ekstrand ja...@jlekstrand.net wrote: On Tue, Apr 30, 2013 at 9:06 AM, John Kåre Alsaker john.kare.alsa...@gmail.com wrote: I'd say we should split the cropping and scaling request into two. How would you suggest doing that? I actually really like the simplicity of this box maps to that box. The boxes approach also does not have rounding issues. The destination size will be exactly what you specify, not some vague number multiplied by number, then rounded somehow. Agreed. Sending transform matrices or the like has HUGE rounding problems. Particularly when we're using wl_fixed which is 24.8. Other methods would require adding rounding conventions etc. Specifying a scaling with a NULL buffer should still set the surface size, so we can have surfaces with only an input region. I really don't think this is the place for that. I want input-only surfaces too, but I don't think this is the place for that I also don't think this is the place for that. If any... I don't see a way to disable scaling and cropping, could passing 0 as width and height do that? Can't the client just set it to 0, 0, width, height, width, height easily enough? Or just destroy the scaler? Yeah, that's two ways to disable scaling and cropping, should be enough. The first one just has to be used repeatedly, if the buffer size changes. Destroying the surface_scaler returns the original 1:1 mapping. On Tue, Apr 30, 2013 at 2:33 PM, Pekka Paalanen ppaala...@gmail.com wrote: Hi all, below is my first draft for a wl_surface scaling and cropping extension. I called it wl_scaler in the lack of a better name. It is designed similarly to the other wl_surface extensions wl_shell_surface and wl_subsurface. There probably isn't any interesting details to debate, right? ;-) I'd like to have a better name for it, and you might want the set request split into two or three, or not, but otherwise I'm quite satisfied with it. I tried to take into account all existing protocol that interacts with this. Comments? Is the language clear? Thanks, pq ?xml version=1.0 encoding=UTF-8? protocol name=scaler copyright Copyright © 2013 Collabora, Ltd. Permission to use, copy, modify, distribute, and sell this software and its documentation for any purpose is hereby granted without fee, provided that the above copyright notice appear in all copies and that both that copyright notice and this permission notice appear in supporting documentation, and that the name of the copyright holders not be used in advertising or publicity pertaining to distribution of the software without specific, written prior permission. The copyright holders make no representations about the suitability of this software for any purpose. It is provided as is without express or implied warranty. THE COPYRIGHT HOLDERS DISCLAIM ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS, IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. /copyright interface name=wl_scaler version=1 description summary=surface cropping and scaling The global interface exposing surface cropping and scaling capabilities is used to instantiate an interface extension for a wl_surface object. This extended interface will then allow cropping and scaling the surface contents, effectively disconnecting the 1:1 relationship between the buffer and the surface size. /description request name=destroy type=destructor description summary=unbind from the cropping and scaling interface Informs the server that the client will not be using this protocol object anymore. This does not affect any other objects, wl_surface_scaler objects included. /description /request enum name=error entry name=scaler_exists value=0 summary=the surface already has a scaler object associated/ /enum request name=get_surface_scaler description summary=extend surface interface for crop and scale Instantiate an interface extension for the given wl_surface to crop and scale its content. If the given wl_surface already has a wl_surface_scaler object associated, the scaler_exists protocol error is raised. /description arg name=id
Re: [RFC] wl_surface scale and crop protocol extension
Hi, On 2 May 2013 19:06, Bill Spitzak spit...@gmail.com wrote: Pekka Paalanen wrote: So it's really a question whether we can require all compositors to unconditionally support cropscale. And that really means *all* Wayland compositors forever, since wl_surface is core protocol. I see no reason not to make this a requirement. You already require 90 degree rotations and ortogonal reflections (since clients can set the buffer transform) and weston uses arbitrary transforms to rotate windows. And you require ARGB32 support even if the hardware does not like it. I also think all of wl_shell should be a core requirement. Not all compositors are user sessions. Think about nested compositors for browsers, or capture, or also very stripped-down usecases where they really don't want to have to deal with this kind of thing. I don't think it's an unreasonable requirement, and really like the design it has at the moment, where attaching the scaler object suppresses the resize-on-attach behaviour, and destroying it reverts to previous. It's pretty elegant, and totally in the vein of wl_shell_surface stacking on top of wl_surface. I don't see how inventing more elaborate extension mechanisms on top of our existing extension mechanism helps anything. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Hi, On 2 May 2013 15:42, Jason Ekstrand ja...@jlekstrand.net wrote: Ok, I see it now. Sorry, but I missed it on my first read-through. Yes, it fixes the problem, but in an extremely confusing way. The reason I say it is confusing is because it inherently mixes buffer and surface coordinate systems. I really think we need to isolate buffer coordinates from surface coordinates more. Perhaps what we need is two requests: set_source_rect and set_dest_rect and completely ignore the x and y from attach. This both provides clarity to the coordinate systems and provides a little separation between crop and scale. Ideally, when wl_surface::commit was added, wl_surface::attach should've been broken out into wl_surface::attach and wl_surface::set_position. Oh well. That being said, we can't ignore the x and y from attach, because that _moves the window_ on screen (think resizing from top left), whereas this is all about how we map the contents of a buffer into that window - totally unrelated to moving. So I think the best way to express that is that this is a transformation that occurs when rendering the buffer - and is thus in buffer co-ordinates - whereas wl_surface::attach is a window management primitive and is thus always in surface co-ordinates. Cheers, Daniel ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Jason Ekstrand wrote: Agreed. Sending transform matrices or the like has HUGE rounding problems. Particularly when we're using wl_fixed which is 24.8. Other methods would require adding rounding conventions etc. I was assuming IEEE 32-bit floating point would be in the api for at least the ABCD of the matrix (the XY could be wl_fixed I suppose). However what you call rounding conventions are needed even for the proposed integer api, and this is also why a source rectangle is needed. It has nothing to do with inaccuracy. It is because filtered needs to know it, since the filter will extend outside the source region (it will for down-scales for any reasonable filter, and even for up-scaling for mitchell/sync style filters). Samples outside this region must be treated specially and this rounding convention must be defined by wayland (almost certainly you want to define it as using the nearest pixel inside the region). So yes a source rectangle must be provided in the api, but not for the reason you think. The destination rectangle is then a nice way to produce rational fractions for scale and also provide a wl_surface size that is different than the buffer size. I would make one change to the proposal because I think arbitrary transforms will need to be considered some day. The source rectangle has to remain orthogonal in source space (otherwise it is useless for filtering), while the destination has to be orthogonal in surface space (since it also controls a lot of other wl_surface details), therefore any future arbitrary transform must be between these. I think the current buffer transform should be considered the start of any future arbitrary transform, so for that reason the source rectangle should be in actual buffer pixels, not in buffer transform space. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
Daniel Stone wrote: I also think all of wl_shell should be a core requirement. Not all compositors are user sessions. Think about nested compositors for browsers, or capture, or also very stripped-down usecases where they really don't want to have to deal with this kind of thing. For simple displays like one that is a single full-screen window always, the api works in that raises determine which window is visible, and attempts to resize or turn off full-screen are ignored. Though nested compositors are an interesting idea, it is clear from how the subsurfaces are being designed that it is not felt that nested compositors are not really useful. The biggest problem I see is that the nested compositor probably loses any high-speed optimizations that only the real compositor can do. I really suspect the only use of a nested compositor will be to test a wayland server inside a window on another one. I don't think it's an unreasonable requirement, and really like the design it has at the moment, where attaching the scaler object suppresses the resize-on-attach behaviour, and destroying it reverts to previous. It's pretty elegant, and totally in the vein of wl_shell_surface stacking on top of wl_surface. I don't see how inventing more elaborate extension mechanisms on top of our existing extension mechanism helps anything. No what I propose is to figure out how to document it so that the wayland api documentation is easier to follow. Make the rules for these sub-api objects match. Then I would expect to see the set-scaling listed under the wl_surface api. There would be a small indication that this api is through the wl_scaler subapi and the rest of the details would be defined in a single section of the wayland documentation. ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
Re: [RFC] wl_surface scale and crop protocol extension
On Thu, May 2, 2013 at 1:22 PM, Daniel Stone dan...@fooishbar.org wrote: Hi, On 2 May 2013 15:42, Jason Ekstrand ja...@jlekstrand.net wrote: Ok, I see it now. Sorry, but I missed it on my first read-through. Yes, it fixes the problem, but in an extremely confusing way. The reason I say it is confusing is because it inherently mixes buffer and surface coordinate systems. I really think we need to isolate buffer coordinates from surface coordinates more. Perhaps what we need is two requests: set_source_rect and set_dest_rect and completely ignore the x and y from attach. This both provides clarity to the coordinate systems and provides a little separation between crop and scale. Ideally, when wl_surface::commit was added, wl_surface::attach should've been broken out into wl_surface::attach and wl_surface::set_position. Oh well. Exactly. That's what my suggestion was trying to fix (at least in the transformed surface case). That being said, we can't ignore the x and y from attach, because that _moves the window_ on screen (think resizing from top left), whereas this is all about how we map the contents of a buffer into that window - totally unrelated to moving. Yes, I realize that the x and y from attach are used for moving or scaling from top or left. My suggestion was to replace the x and y in attach with an x and y in set_dest_rect. We wouldn't be losing functionality, just moving it in a certain case. This way surface coordinates are kept with surfaces and buffer coordinates are kept with buffers. In this case, the defined behavior would be that if you create a scaler for a surface, the x and y in attach are disabled and set_dest_rect takes over. This way older clients can just use attach like they used to and clients that use surface scalers use the destination rectangle. It's not a perfect fix, but I think it moves in the right direction. And, for what it's worth, it doesn't make things significantly more complicated because anything that's going to scale from the top-left will have to mess with both the surface and the scaler anyway. --Jason Ekstrand ___ wayland-devel mailing list wayland-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/wayland-devel
