On Mon, 4 Mar 2024 13:24:56 +0000 Terry Barnaby <ter...@beam.ltd.uk> wrote:
> On 04/03/2024 09:41, Pekka Paalanen wrote: > > On Mon, 4 Mar 2024 08:12:10 +0000 > > Terry Barnaby <ter...@beam.ltd.uk> wrote: > > > >> While I am trying to investigate my issue in the QtWayland arena via the > >> Qt Jira Bug system, I thought I would try taking Qt out of the equation > >> to simplify the application a bit more to try and gain some > >> understanding of what is going on and how this should all work. > >> > >> So I have created a pure GStreamer/Wayland/Weston application to test > >> out how this should work. This is at: > >> https://portal.beam.ltd.uk/public//test022-wayland-video-example.tar.gz > >> > >> This tries to implement a C++ Widget style application using native > >> Wayland. It is rough and could easily be doing things wrong wrt Wayland. > >> However it does work to a reasonable degree. > >> > >> However, I appear to see the same sort of issue I see with my Qt based > >> system in that when a subsurface of a subsurface is used, the Gstreamer > >> video is not seen. > >> > >> This example normally (UseWidgetTop=0) has a top level xdg_toplevel > >> desktop surface (Gui), a subsurface to that (Video) and then waylandsink > >> creates a subsurface to that which it sets to de-sync mode. > >> > >> When this example is run with UseWidgetTop=0 the video frames from > >> gstreamer are only shown shown when the top subsurface is manually > >> committed with gvideo->update() every second, otherwise the video > >> pipeline is stalled. > > This is intentional. From wl_subsurface specification: > > > > Even if a sub-surface is in desynchronized mode, it will behave as > > in synchronized mode, if its parent surface behaves as in > > synchronized mode. This rule is applied recursively throughout the > > tree of surfaces. This means, that one can set a sub-surface into > > synchronized mode, and then assume that all its child and grand-child > > sub-surfaces are synchronized, too, without explicitly setting them. > > > > This is derived from the design decision that a wl_surface and its > > immediate sub-surfaces form a seamlessly integrated unit that works > > like a single wl_surface without sub-surfaces would. wl_subsurface > > state is state in the sub-surface's parent, so that the parent controls > > everything as if there was just a single wl_surface. If the parent sets > > its sub-surface as desynchronized, it explicitly gives the sub-surface > > the permission to update on screen regardless of the parent's updates. > > When the sub-surface is in synchronized mode, the parent surface wants > > to be updated in sync with the sub-surface in an atomic fashion. > > > > When your surface stack looks like: > > > > - main surface A, top-level, root surface (implicitly desynchronized) > > - sub-surface B, synchronized > > - sub-surface C, desynchronized > > > > Updates to surface C are immediately made part of surface B, because > > surface C is in desynchronized mode. If B was the root surface, all C > > updates would simply go through. > > > > However, surface B is a part of surface A, and surface B is in > > synchronized mode. This means that the client wants surface A updates to > > be explicit and atomic. Nothing must change on screen until A is > > explicitly committed itself. So any update to surface B requires a > > commit on surface A to become visible. Surface C does not get to > > override the atomicity requirement of surface A updates. > > > > This has been designed so that software component A can control surface > > A, and delegate a part of surface A to component B which happens to the > > using a sub-surface: surface B. If surface B parts are further > > delegated to another component C, then component A can still be sure > > that nothing updates on surface A until it says so. Component A sets > > surface B to synchronized to ensure that. > > > > That's the rationale behind the Wayland design. > > > > > > Thanks, > > pq > > Ah, thanks for the info, that may be why this is not working even in Qt > then. > > This seems a dropoff in Wayland to me. If a software module wants to > display Video into an area on the screen at its own rate, setting that > surface to de-synced mode is no use in the general case with this > policy. It is of use, if you don't have unnecessary sub-surfaces in synchronized mode in between, or you set all those extra sub-surfaces to desynchronized as well. > I would have thought that if a subsurface was explicitly set to > de-synced mode then that would be honoured. I can't see a usage case for > it to be ignored and its commits synchronised up the tree ? Resizing the window is the main use case. In order to resize surface A, you also need to resize and paint surface B, and for surface B you also need to resize and paint surface C. Then you need to guarantee that all the updates from surface C, B and A are applied atomically on screen. Either you have component APIs good enough to negotiate the stop-resize-paint-resume on your own, or if the sub-components are free-running regardless of frame callbacks, component A can just temporarily set surface B to synchronized, resize and reposition it, and resume. > So is there a way to actually display Video on a subsurface many levels > deep in a surface hierarchy, would setting all of the surfaces up to the > subsurface just below the desktop top level one work (although not ideal > as it would mean overriding other software modules surfaces at the > Wayland level) ? Setting to what? > Or can desynced subsurfaces really only work to one level deep ? You can set your middle sub-surface to desynchronized, too, at least from Wayland perspective. I don't know if Qt let's you. There are implementation issues with nested sub-surfaces in Weston, but this does not seem to be one of them. > If it is just one subsurface level deep that video can be displayed, I > guess I will have to get GStreamers waylandsink to create its subsurface > off the top most surface and add calls to manage its surface from my > app. That should have been the first idea to try. Is Gst waylandsink API the kind that it internally creates a new wl_surface for itself and makes it a sub-surface of the given surface, or is there an option to tell Gst to just push frames into a given wl_surface? If the former, then waylandsink is supposed to somehow give you an API to set the sub-surface position and z-order wrt. its parent and siblings. If the latter, you would create wl_subsurface yourself and keep control of it to set the sub-surface position and z-order. Either way, the optimal result is one top-level wl_surface, with one sub wl_surface drawn by Gst, and no surfaces in between in the hierarchy. > Or maybe get waylandsinks subsurface and manipulate it behind > waylandsinks back. Not sure what this will do to the Qt level though. > Using the QWidgets subsurface as its base should have allowed isolation > to a degree between the Qt and wayland sink sub systems, its a much more > modular approach. You cannot punt between-components integration to Wayland. The sub-surfaces design tried and failed, and as a result the sub-surface protocol is both complex and insufficient. You still need explicit communication between your client-side components when you resize. > There is no documentation on this Wayland restriction in waylandsink or > other places, I can try and feed this back. > > Oh for X11 ! Yeah, the blink of garbage when windows are resized, or the blue colorkey following far behind when a window is moved. Thanks, pq
pgpVVMuYCow6N.pgp
Description: OpenPGP digital signature
