On Wed, 6 Mar 2013 15:01:38 +0100 David Herrmann <[email protected]> wrote:
> Hi Pekka > > On Wed, Mar 6, 2013 at 1:49 PM, Pekka Paalanen <[email protected]> wrote: > > Hi, > > > > I have been pondering about sub-surface nesting, i.e. allowing > > sub-sub-surfaces and so on, and I have hard time figuring out how > > commits should work. > > > > The v2 sub-surface protocol proposal is here: > > http://cgit.collabora.com/git/user/pq/weston.git/tree/protocol/subsurface.xml?h=subsurface-v2 > > > > Adding support for nesting does not require any API changes to the > > protocol. We just need to define, that it is ok to have a sub-surface > > as a parent for another sub-surface, and how commits work. Commit > > behaviour is again the hard part. > > > > Suppose we have a window consisting of the following: > > > > main surface, wl_surface A > > ↳ sub-surface, wl_surface B > > > > Every surface has current state (how it is currently drawn on screen), > > and pending state (to accumulate pieces of state, that will then be > > made current atomically on wl_surface.commit). > > > > Additionally, sub-surfaces have a state cache. When a sub-surface > > itself is explicitly committed, the pending state either becomes > > current, or goes to the cache, depending on the commit mode of the > > sub-surface. The pending state can only be used when a commit request > > was received for the surface itself, never via parent commit. > > > > If the commit mode for the sub-surface B is parent-cached, then > > B.commit() will push the pending state into the cache. The cached state > > is made current on A.commit(), so that all surfaces get updated > > atomically. > > > > If the commit mode for the sub-surface B is independent, then > > B.commit() will make the pending state current. A.commit() does not > > affect B's current state, except for the wl_subsurface.set_position. > > > > The position set by a wl_subsurface.set_position request is always > > stored, and applied only on parent surface commit (A.commit()). This > > way the contents of wl_surface B can be updated independently, but > > positioning it is still tied to the wl_surface A updates. > > > > (I'm not considering the corner cases arising from transitions from one > > commit mode to another at arbitrary times here, but they are solved > > for non-nested sub-surfaces.) > > > > > > Now, add a sub-sub-surface. Suppose we have a window consisting of the > > following: > > > > main surface, wl_surface A > > ↳ sub-surface, wl_surface B, parent-cached > > ↳ sub-sub-surface, wl_surface C, parent-cached > > > > Assume the commit mode of all sub-surfaces is parent-cached, as > > written above. C.commit() will push C's pending state into C's cache. > > B.commit() will push B's pending state into B's cache. But does > > B.commit() do anything to C? > > > > As B is a sub-surface itself, needing a parent commit to change its > > current state, B.commit() certainly cannot make C's cached or pending > > state current. > > > > Suppose we have this sequence in protocol: > > > > C.commit(C1) > > B.commit(B1) > > C.commit(C2) > > A.commit(A1) > > > > What state should be current for each surface? > > For A, it is A1, of course. > > For B, it is B1, trivially. > > For C, should it be C1 which has been "blessed" by B, or C2 which is > > the latest but not "blessed" by B? > > > > If we say C1, then we will need a multi-level cache for each > > sub-surface, with a level of cache for each level of nesting. > > Sub-surface C would need a 2-level cache. C.commit() pushes pending > > state to C.cache-1. B.commit() pushes C.cache-1 to C.cache-2. > > A.commit() makes C.cache-2 as current state of C. > > > > Furthermore, each level of cache may hold a reference to a wl_buffer. > > This means, that the component driving wl_surface C will actually need > > a pool of 4 buffers: > > - a buffer that is currently on screen > > - a buffer that is in cache-2 > > - a buffer that is in cache-1 > > - a buffer that it is rendering to > > > > I do not think this is feasible. So, the answer must be that C2 is the > > current state for C. In other words, whether B.commit is called or not, > > it has no effect on what happens with C. (But does it make sense for > > applications?) > > My first thought was "sure, C1 it is", but considering the huge cache > consumption if we have deeper sub-sub*-surface nesting, this really > doesn't make sense. So I was wondering whether C2 is actually that > bad. > > So lets assume a user uses parent-cached sub-surfaces. This means, > they want the child to be updated only when the parent got updated. > This implicates, that the user has some kind of synchronization > between child and parent. Otherwise using parent-cached just doesn't > make sense as they could use the independent mode instead. > > With this in mind, we know that if we have multi-layer nested > parent-cached sub-surfaces, then there already is a synchronization > between _all_ of them. So saying "parent-cached" means > "top-most-parent-cached" is actually fine for these use-cases, isn't > it?. Of course, "top-most-parent" means the top most parent with only > "parent-cached" in between. > > Ok, but are there situations where parent-cached makes sense but there > is no synchronization in the client already? > I actually cannot think of anything. The thing is, if we have > parent-cached without synchronization in the client, the parent > surface does not know the current state of the client. So it has no > idea which frame the client currently has pending, it just knows that > it is only displayed _together_ with the parent. > So when the parent does the commit, it knows the client is updated > atomically, but it has no idea what frame the client actually drew. So > what is the reason that you actually want the atomic update in this > case? The independent mode would be just fine here. The only drawback > is that the parent no longer controls the frame-rate of the client. > But does that really matter? > > So if we can conclude that the client does synchronization between > parent-cached subsurfaces, we can actually rename parent-cached to > top-most-parent-cached and be fine. > If anyone comes up with a use-case for _real_ parent-cached (that is, > bottom-most-parent-cached), we can always implement that with the > heavy caching, can't we? > > And top-most-parent-cached and independent mode are both trivial to > implement, if I understand you correctly? > > > > > Next a case with mixed commit modes: > > > > main surface, wl_surface A > > ↳ sub-surface, wl_surface B, independent > > ↳ sub-sub-surface, wl_surface C, parent-cached > > > > C.commit() pushes C's pending state to C's cache. > > B.commit() make B's pending state current, and make C's cached state > > current, too. > > A.commit() make A's pending state current, and... does nothing to B, > > since B is independent. Also, does not even look past B, so nothing for C. > > > > Ok, I guess that works. And the case where B is parent-cached and C is > > independent should be obvious, too. > > > > > > > > This leads to the following commit algorithm: > > > > - Main surface commit, and sub-surface commit in independent mode, will > > make own pending state current, and will call recursive commit on all > > its immediate sub-surfaces. > > > > - Sub-surface commit in parent-cached mode will push the pending state > > into the sub-surface's cache. No recursion. > > > > - A recursive commit on a sub-surface with independent commit mode does > > nothing. No recursion. > > > > - A recursive commit on a sub-surface with parent-cached commit mode > > will make the surface's cached state current, and call recursive > > commit on all its immediate sub-surfaces. > > > > And then add all the corner cases when commit modes are changing, and > > e.g. a sub-surface is committed in independent commit mode with some > > state in its cache. > > > > Oh, and I forgot to specify what happens with > > wl_subsurface.set_position. D'oh. > > > > > > So, what are the benefits from nesting sub-surfaces? > > - Each sub-surface's position is relative to its parent, so you don't > > have to add up the offsets in the client. > > - An independent sub-surface can have its own sub-surfaces, and update > > the whole wl_surface sub-tree in sync. > > - ...? > > > > > > Is this all really worth it? > > Does anyone have a good use case for nested sub-surfaces, which would > > be inconvenient to implement with just sibling sub-surfaces without > > nesting? > > > > Unless there is a good use case I can design for, I'm tempted to just > > not bother with this, this time. If we later see, that nesting > > would actually have value, we can add the support by just bumping the > > wl_subcompositor interface version, and without any changes to the > > protocol signature. > > Use-case for sub-sub-surfaces: > > - flash-player embedded in webkit embedded in some application > I often see applications that embed a browser (for instance STEAM). If > that browser embeds other stuff, you want it to be independent of the > parent. > This use case requires top-most-parent-cached or independent. But I > cannot think of a reason why it requires bottom-most-parent-cached, > can you? > > If we don't allow sub-surface nesting, then clients will have to > provide some sub-surface allocation forwarding. So for instance in my > use case described above, the application would provide a sub-surface > allocator to webkit, which would allocate the sub-surface for > flashplayer. But webkit would not be able to allocate the sub-surface > itself from the Wayland compositor as it itself is a sub-surface. > So in this case, we already have the synchronization of the clients > via the sub-surface allocator in the application. So there _is_ a > communication channel between the application and it's > sub-sub-sub-...-surfaces. This allows us to use top-most-parent-cached > instead of (bottom-most-)parent-cached. > > => Implement sub-surface nesting in the compositor to avoid having > each and every application to implement the sub-surface forwarding if > they themselves run as sub-surfaces. > > > Anyway, I really like the proposal. Apart from few typos the protocol > looks really good and I agree that we can implement nesting later. Thank you, and thank you all for the discussion on #wayland, it really gave me some new ideas. I'm not replying to your specific points right now, David, since I have to re-evaluate the design now. I realized the wl_subsurface.set_commit_mode() is not right, if we have nesting. It cannot be per-surface, but it must affect the whole sub-tree of sub-surfaces for atomic resizing to work. That might actually solve a lot of weirdness there. And I was certainly wrong in thinking that adding nesting support as an afterthought would be anything but ugly. So, stay tuned! ;-D Thanks, pq _______________________________________________ wayland-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/wayland-devel
