Hi Jason, Tanibata-san > Hi Jason, > > Thank you very much for feedback. > >> Michale & Nobuhiko, >> >> First of all, thank you for the clarification and thank you for >> sending this to the list and being willing to work with the FOSS >> community to try and make a standard. I'm sorry that this reply is not >> inline. I think that would get disorganized and more confusing but >> I'll try to hit everything I saw. >> >> The first distinction that needs to be made (and I think Pekka was >> trying to hint at this) is what should be standardized. If you look at >> the current [wayland core protocol][1] there is only one shell >> protocol called wl_shell. I have proposed another which will probably >> get called [wl_fullscreen_shell][2]. Both of these have something in >> common: they are purely client-side. There is nothing whatsoever in >> the standard about managing surfaces. I think that we should focus on >> what you have designated ivi_client.
Ok, now I got it. No we do not want to push it to the core protocol. We want to define a IVI-Shell like desktop shell and want to use weston for the reference implementation. The ivi-shell extension should be the minimum subset which shall provided by each ivi-compositor. But in terms of the genivi compliance we are able to define a shell protocol and say this is the minimal subset which is supported, and therefore we need a reference implementation too. Maybe we have mixed something here from our side. We will clarify that >> What about the controller? If you look in the [weston protocol >> folder][3] you will find a number of different protocol files. Some of >> these are for experimental extensions such as subsurfaces which have >> not yet made it into wayland core. However, a number of them such as >> desktop-shell, screenshooter, etc. will *never* be standardized in the >> wayland core. These protocols are completely internal to weston and >> are considered implementation details. The primary example is >> desktop-shell. This protocol exists for the purpose of allowing the >> out-of-process shell controller manage surfaces similar to what you >> propose with ivi_shell. There are other shell plugins for weston >> (hawaii & orbital) that each have their own shell plugin and can have >> their own protocol for talking to an out-of-process controller. Yes this is our focus too, like I have described above. >> How does this impact your proposed protocol? Unless you are convinced >> that every single IVI system manufacturer will want to manage surfaces >> the same way, the controller should be left as a private >> implementation detail. You are free to do it out-of-process and talk >> the wayland protocol to do so (desktop-shell does) but there is no >> need to expose it as part of a standard protocol. By only >> standardizing the client interface you leave app developers (GPS, >> Media players, etc.) free to design their apps however they want and >> you leave IVI system manufacturers free to handle those clients and >> surfaces in whatever way they want. >> >> Ok, now on to actual suggestions. >From this point forward, I am going >> to completely ignore the controller side of things. >> >> First, I would propose to follow the pattern of wl_shell and make two >> interfaces for clients to talk to the compositor. For now, I am going >> to call them wl_ivi_shell and wl_ivi_surface. We can come up with >> different names if you'd like, but those seem reasonable. If we follow >> the pattern of wl_shell, wl_ivi_shell will probably exist for the sole >> purpose of creating wl_ivi_surface objects. This pattern is common in >> the protocol (wl_shell, wl_subcompositor, wl_compositor, etc.). >> >> The main question, then, becomes what to put in wl_ivi_surface. I'm >> not 100% sure what you intend with some of this surface and layer >> stuff, so I'm afraid I don't have a whole lot of specific suggestions >> on that for now. I do, however have some general thoughts and >> questions: >> >> First, I agree with Pekka that you can probably avoid the layers >> thing by simply using subsurfaces. >> > I see. However, we have a use case that several application, different > process share a layer. E.g. Navigation map and Route guidance are > separated into other application. It may kind of grouping parent > surfaces. > >> Second, Why are you specifying pixel formats in ivi_surface? Is the >> compositor supposed to tell the client what format to render in? >> >> Third, concerning the "visibility" flag. The wayland protocol as it >> currently stands tries to avoid telling clients specifically whether >> or not they are visible and where they are on screen. This is because, >> when clients abuse this information, compositors lose the freedom to >> throw surfaces around how they want. Instead of a visibility flag, the >> wl_surface interface provides a "frame" callback that the clients can >> use to know when was the last time they were drawn to the screen. A >> client should throttle rendering based on these frame events. If the >> surface is offscreen and the compositor wants the client to stop >> rendering it simply stop sending it frame events and the client will >> stop drawing. >> > > I have two concerning to use "frame" for realizing invisible. > - To set invisible, application needs to call clear color by itself. I > think it might be overhead for GPU. If we can realize it in shell, it > simply skips it to be composite. Of course, application shall stop > drawing as well. > - Invisible shall be conrolled by cetral controller due to safy > reason.It shall be done in lower part as much as possible. Ideally, if > we can allocate it to another physical plane, it may be best. If Display > contoller doesn't support it, next is compositor. Here I am full inline with the post from Tanibata-san Regards Michael. >> Once again, thank you for mailing the list. I hope my thoughts above >> are helpful and can clear a few things up. >> Thanks, >> --Jason Ekstrand >> >> [1]: >> http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml >> [2] >> [2]: >> http://lists.freedesktop.org/archives/wayland-devel/2013- >> August/010720.html [3] >> [3]: http://cgit.freedesktop.org/wayland/weston/tree/protocol [4] >> On Mon, Sep 9, 2013 at 7:59 PM, Nobuhiko Tanibata >> <[email protected]> wrote: >> > [email protected]> wrote on o: "nobuhiko_tanibata" > <[email protected]>: >> Subject: Re: [PATCH weston 0/6] ivi-shell proposal >> On Sun, 08 Sep 2013 00:13:55 +0900 >> nobuhiko_tanibata <[email protected]> >> wrote: >> >> 2013-09-06 19:16 に Pekka Paalanen さんは書きました: >>> On Fri, 6 Sep 2013 08:39:24 +0900 >>> "Nobuhiko Tanibata" <[email protected]> wrote: >>> >>> "Pekka Paalanen" <[email protected]> wrote on o: "Nobuhiko Tanibata" >>> <[email protected]>: >>>> Subject: Re: [PATCH weston 0/6] ivi-shell proposal >>>> >>>> >>>>> On Wed, 4 Sep 2013 09:08:26 +0900 >>>>> "Nobuhiko Tanibata" <[email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> This series implements ivi-shell to fulfill use cases of In-Vehicle >>>>>> Infotainment, IVI. Such use cases are well overviewed in a project; >>>>>> Genivi IVI layer management. >>>>>> http://projects.genivi.org/ivi-layer-management/node/13 [5] >>>>>> >>>>>> A motivation of this series and basis idea are introduced by Ossama >>>>>> at Automotive Linux Summit 2012 spring. The series implements >>>>>> ivi-shell part. Additionally, GENIVI LM Client Library at slide 20 >>>>>> >> >> is contributed to ivi-layer-management project to support >>>>>> compatible interfaces for Genivi Layer management users. >>>>>> >> >> http://events.linuxfoundation.org/images/stories/pdf/als2012_othman.pdf >> [6] >>>>>> >>>>>> Before I start implementation of ivi-shell, Core members of Genivi >>>>>> IVI layer management defined draft of ivi-shell.xml to fulfill >>>>>> requirements of IVI layer management, inviting Kristian. The series >>>>>> also includes the ivi-shell.xml with updates I faced in actual >>>>>> implementation. >>>>>> >>>>>> Please give me any suggestions. >>>>> >>>>> Hi, >>>>> >>>>> I have understood that the whole design has been cut deep into stone >>>>> >> > a long time ago. What are you able to change now? I do not >>>>> think it is worth commenting on things you can no longer change, so >>>>> what aspects >> > are you looking suggestions for? >>>>> >>>> >>>> Hi, >>>> >>>> I would like to get feedback about that if somebody has a similar >>>> motivation to support ivi as well as desktop and tablet. So it is not >>>> a stone, just a proposal. If somebody has good idea, I would like to >>>> use it or collaborate it. >>> >>> Ok, I just had the understanding that the layer manager simply has to >>> be a separate process and not built into the compositor. If that is >>> not the case, then that is very good news indeed. Everything that >>> manages surfaces, layers, windows, or whatever belongs into the >>> compositor process, where they are much easier to implement and you >>> don't need to introduce interfaces and IPC which are later hard to >>> develop further and cause latencies. Roundtrips and synchronous calls >>> between processes can become a difficult bottle-neck, as X11 has >>> taught us. Also having too many processes becomes a real mess when >>> trying to avoid deadlocks but still keep things coherent and >>> glitch-free (see X11 server vs. window manager vs. ...). So I'm >>> roughly on the same track as Andreas Pokorny. >>> >>> Weston should become the window manager and layer manager, while >>> weston backends deal with the hardware details of compositing. For >>> example, the Raspberry Pi backend of Weston forwards all >>> compositing to the VideoCore firmware, unlike any other backend >>> who actually render a composite themselves. However, the rpi >>> backend does not forward all surfaces to VideoCore all the time, >>> but only the visible ones as needed. And Weston is the only >>> component that *can* know what is visible at any time, since Weston >>> contains the complete scene graph. Weston's internal architecture >>> is also well-suited for *automatically* deciding how to use the >>> limited hardware resources like overlays efficiently, and fall back >>> as necessary, per each output frame. You cannot do that, if you >>> tell applications about specific hardware resources. >>> >> Yes, my understind is the same as yours. >> >> Great. :-) >> And I expects Weston backend >> you mentioned next paragraph is ideally released as reference from SoC >> vender. For exmple, some SoC vendor supports 2D blit engine which can >> effciently composite surfaces and relese weston backend as reference. >> >> Tbh, I doubt vendor's abilities in producing a proper Weston >> backend, and maintaining it as Weston changes. But yeah, something >> like that. >> >>> I suspect there might be some terminology differences here. >>> Something like what IVI calls a "compositor" is a "weston backend" >>> when talking about Weston and Wayland, and IVI layer manager is >>> actually a window manager which is just a shell plugin to Weston. >>> Or am I completely off? >>> >>> >> Yes, you are correct. And I focus on shell part to realize ivi >> requirements. >> >> Shell is usually the part which is primarily used by applications, not >> supporting components of a graphical environment (see wl_shell vs. >> desktop_shell protocol interfaces). I mean the public part of shell, >> like the wl_shell interface for desktop applications. >> >> Which brings me to another question: how likely is it that you >> actually want to support PC desktop applications unmodified >> directly on an IVI system? And I mean on the main compositor of an >> IVI system, which I would imagine to be quite a critical component. >> >> I think native IVI applications could be different enough to PC >> desktop applications, that creating a new shell interface to *replace* >> wl_shell (or whatever shell interface we will be using in the future >> for PC desktop) would be a right choice. >> >> If you actually do want to support PC desktop applications, I could >> see you having another, nested compositor just for supporting PC >> applications, which could run on a more restricted environment and >> maybe access to a more complex GPU which would be too risky for >> the main compositor in IVI. A sort of "untrusted" domain. >> >> Well, Genivi probably has already designed all that, so I'm just >> reinventing the wheel badly here. >> >>> Have you tried to map your IVI concepts of surface/layer/display to >>> Wayland wl_surface, wl_subsurface, and wl_output? I don't really >>> see what kind of interfaces your applications (Wayland clients?) >>> expect to use. >>> >> Yes. As you comment, some use case; visibility and crop/scaling is not >> supported now. So I thought starting new set of protocal to cover ivi >> requiremnts would be better. But I will re-consider them and mail it >> back. >> >> Right. I replied from purely FOSS point of view. You probably have >> time and money deadlines which you must meet while creating a >> self-standing product, and in that case, I do understand going with >> a big re-invention. I understand it, but I'm not happy about it, >> although if you can publish your ad hoc approach like you do here, >> you are contributing valuable experience to the community. >> Especially, if you say something about the shortcomings of the >> design and use experiences. >> >> I'm very happy to see this proposal on the mailing list, even when >> I do not agree with it. >> >>> When I look at the protocol in ivi-shell.xml, I get the feeling that: >>> - Interfaces ivi_layer, ivi_controller_surface, ivi_controller_layer, >>> ivi_controller_screen, and ivi_controller should be internal >>> implementation details inside the weston process, not protocol. Having >>> these as interfaces looks like the X architecture, where the X server >>> process and window manager process continuously struggle to keep each >>> other up-to-date, and carefully try to keep state in sync (and fail), >>> which also makes races and glitches practically unavoidable. >>> >> I have basic question to the above; strugling. wl_subsurface supports >> set_poistion now. >> >> set_position for sub-surfaces is *always* relative to the parent >> surface. The sub-surface position is given as a point on the parent >> surface. You still have no control where on the output any surface >> will appear, because you cannot control where the root surface of a >> tree of sub-surfaces will appear. >> >> If a parent surface is moved on screen, all its sub-surfaces stay >> glued to it automatically without any client interaction. >> I thought it implies positioning from a windowmanger is allowed on >> weston basic concept. >> >>> From a window manager, yes, but this assumes that the window manager >>> is in-process with weston; the window manager must be a compositor >>> plugin. Making it an external process will be a huge amount of >>> trouble and performance loss. >>> >> And the protocol you propose seems to be for an external window >> manager process, from my understanding. >> Each other up-to-date may occurs from window manager and weston >> internl >> decision. >> Or positioning of sub surfaces is out of scope of weston and it just >> composite them according to attributes. >> I may be wrong. Please let me know history. >> >> The first thing is that in Wayland core protocol, there is no >> global positioning system. There are no global coordinates in the >> protocol. All coordinates that clients deal with, are relative to >> some wl_surface. (This also allows the compositor to do arbitrary >> transformations on surfaces, because there is no need for clients >> to know about them, and so no need to express transformations in >> the protocol.) >> >> Sub-surfaces do not change that design. >> >> Another thing is that with sub-surface positioning, the information >> flows strictly into one direction, and we use wl_surface.commit >> request on the parent surface to synchronize everything. That means >> that a client can manipulate a whole tree of sub-surfaces, and >> *guarantee* an atomic, flicker-free, glitch-free update on screen. >> >> If one needed IPC between a compositor and a window manager, you >> would either risk visual glitches as compositor first draws one >> thing before the window manager says otherwise, or jerky compositor >> performance as it needs to wait for the window manager to respond >> before it can draw anything. I don't see any way around that. >> >>> - Interface ivi_client is just a reinvention of wl_compositor and >>> wl_subcompositor. >>> - Interface ivi_surface is a reinvention of wl_surface. >>> >>> Yes, I see there are some details to may want to control like >>> surface opacity, that the current Wayland protocols do not support, >>> but I don't think that replacing everything is a good way to start. >>> >>> It is also very hard to see how objects from all these interfaces >>> are created, and how (if?) they associate to any other protocol >>> objects. >>> >>> Btw. if you need support for surface scaling and cropping, there >>> have been discussion on the Wayland mailing list to bring a crop & >>> scale protocol extension to Wayland. It is actually necessary for >>> efficient video playback etc., so pushing that forward would be >>> nice. >>> >>> >> Thank you. I will check. >> >> Search for "wl_scaler", that was the working name of a proposal. >> >>> After looking through the two links you gave, the ivi-shell.xml, >>> and what you have wrote in the emails, I still have no clue what is >>> the big picture here. >>> >>> >> I will draw a pciture to explain them. I will mail it back later. >> Hi, >> >> I am enclosing a pdf to show relationship compositor, surface, layer, >> and controller. >> >> As Michael Schuldt says in reply, only a process will control >> attribute of surfaces and layers like position, visivility, order, and >> so on. >> The controller process is kind of central one to manage policy of >> layout and status of surfaces and layers by taking account into >> infomation in vehicle. For example, when gear position is rear, rear >> view camera would be set to the top of order and visible. When I see >> your comment, >> I think wl_subsurface can be applied to layer and parent can be >> mapped to layer. Now I am considering it. However, I have one use case >> for ivi to control attribute of wl_surface from outside, e.g set >> invisible to TV screen when speed is e.g 10km/s for safty. it is not >> welcome for users but I have to consider it. Each application can do >> it but on the safty point of view, it shall be handle in cetranl >> controller. >> >> Addtionally, As Michael Schuldt says in reply as well, for debugging >> purpose, controlling wl_surface would be required. >> >> When I see my picture by myself, ivi-shell will be window manager in >> side of weston. So it might not be conflict be concept of shell. >> >> Please give me your feedback. >> >> Best regards, >> Tanibata >> >>> Cool, thanks. >>> >>> I saw the terms surface, layer, etc. in the IVI docs but I didn't >>> really get what they are used for. >>> >>>> - What processes are going to use which interfaces? It looks to me >>>> like some interfaces are not meant for all Wayland clients, but >>>> how is it supposed to work? >>>> >>>> - What components are in a whole IVI system, from the point of view >>>> of Wayland protocol? What are the responsibilities of each component >>>> and how are these distributed into processes? >>>> >>>> - What does a typical IVI application do in terms of Wayland >>>> protocol? Are you using wl_compositor at all? Or any other >>>> Wayland core interfaces? >>>> >>>> These are just few questions to get you oriented on what kind of >>>> things puzzle me here. Obviously, I have never been in touch with >>>> Genivi stuff before, and I would assume most here have not either. >>>> >>>> The protocol you propose seems to have many references to >>>> "id_native" and "native content", what is all this "native" stuff >>>> about? Or all the integer id's you seem to be sending back and >>>> forth, why can't you use real protocol objects to refer to those? >>>> >>>> >>>> The above is the first impression from someone, who does not know >>>> anything about the IVI architecture, but is fairly familiar with >>>> Wayland. Sorry if it came out harsh, but I feel like I totally >>>> missed the whole background of this proposal: why design it like >>>> that? >>>> >>> >>> Thank you again for good feedback. They are very helpfull for me. >>> >>> Thank you, >>> pq >>> _______________________________________________ >>> wayland-devel mailing list >>> [email protected] >>> http://lists.freedesktop.org/mailman/listinfo/wayland-devel [1] >>> >> _______________________________________________ >> wayland-devel mailing list >> [email protected] >> http://lists.freedesktop.org/mailman/listinfo/wayland-devel [1] >> >> >> Links: ------ [1] >> http://lists.freedesktop.org/mailman/listinfo/wayland-devel [2] >> http://cgit.freedesktop.org/wayland/wayland/tree/protocol/wayland.xml >> [3] http://lists.freedesktop.org/archives/wayland-devel/2013- >> August/010720.html [4] >> http://cgit.freedesktop.org/wayland/weston/tree/protocol [5] >> http://projects.genivi.org/ivi-layer-management/node/13 [6] >> > http://events.linuxfoundation.org/images/stories/pdf/als2012_othman.pdf > _______________________________________________ wayland-devel mailing > list [email protected] > http://lists.freedesktop.org/mailman/listinfo/wayland-devel ------------------------------------------------------- BMW Group Michael Schuldt Plattformtechnol. Infotainment-Kompon. Max-Diamand-Straße 13 80788 München Mailto: [email protected] URL: http://www.bmwgroup.com ------------------------------------------------------- Bayerische Motoren Werke Aktiengesellschaft Vorstand: Norbert Reithofer, Vorsitzender, Milagros Caiña Carreiro-Andree, Herbert Diess, Klaus Draeger, Friedrich Eichiner, Harald Krüger, Ian Robertson, Peter Schwarzenbauer. Vorsitzender des Aufsichtsrats: Joachim Milberg Sitz und Registergericht: München HRB 42243 -------------------------------------------------------- _______________________________________________ wayland-devel mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/wayland-devel
