How HW resources are managed is an implementation choice (ODP and HW design/implementation). What should the ODP architecture specify?
ODP applications really only need low-overhead I/O access (sending data to and from HW blocks). Control access could normally accept higher overhead and potentially be done through the kernel. With control access I mean resource allocation/deallocation and configuration.This would allow reuse of (existing) kernel drivers and also make the kernel aware of resource usage (to facility cleanup, e.g. when an application terminates willingly or unwillingly). There are some use cases we need to discuss and agree on in order to be able to define the requirements on resource management. Start by assuming one OS instance on the SoC. Application means ODP application. 1) An application (repeatedly) reconfigures itself (e.g. to adapt to changing use cases or capacity requirements) and allocates and frees HW resources. A freed HW resource can then be allocated and used by some other application. 2) An application terminates (exit or crash) without properly cleaning up HW resources it had allocated. The same or a different application is launched and allocates the same HW resources. 3) Multiple applications run concurrently, without interfering with each other. 4) Application(s) run in a VM and there may be one or multiple VM's on the same SoC. Possibly the hypervisor is using some HW resources as well, this must be protected from all VM's and their applications. 5) Resource cleanup after VM termination so that HW resources can be allocated to other VM. 6) VM swapping, VM migration... I expect that SoC specific HW limitations will make some of these use cases impossible to implement robustly (e.g. lack of SMMU/IOMMU will be quite limiting). We probably need to define specific feature levels and then individual ODP implementations will specify which feature levels they support. -- Ola On 18 August 2014 08:36, Alexandru Badicioiu <[email protected] > wrote: > HW resources are not exclusively managed by the OS/HV; most likely > resource management is split between kernel and user-space. User-space > drivers are usually libraries and ODP implementation is on top of those > libraries so the ODP must provide the clean-up routines. > > > On 14 August 2014 18:01, Ola Liljedahl <[email protected]> wrote: > >> (Catching up with all emails exchanged during my vacation). >> >> Whenever an API has a create call, there should be a corresponding >> destroy call that reverses the effects of the create call. >> >> Resetting and freeing up HW resources ought to the be responsibility of >> the OS and the hypervisor (if any). But if an application uses the destroy >> calls in an appropriate way, it should be possible for the application to >> reverse all resource allocation and initialisation, thus enabling the >> application to restart (indefinitely) without OS/HV intervention. Is this >> required behaviour? Should we have a test case for this? >> >> >> >> On 15 July 2014 21:35, Bill Fischofer <[email protected]> wrote: >> >>> Thanks Mike. I've assigned the card to me and will take an initial stab >>> at summarizing the issues and proposals in arch doc format. >>> >>> Bill >>> >>> >>> On Tue, Jul 15, 2014 at 1:09 PM, Mike Holmes <[email protected]> >>> wrote: >>> >>>> After todays call it was clear that at the very least the ODP >>>> architecture document needs to spell out ODPs role in teardown. >>>> >>>> The issues were: >>>> The context within which ODP apis may be called. >>>> The graceful and abnormal termination cases. >>>> Define ODPs role, vs that of the application, vendor support and the HW >>>> in the tear down cases. >>>> >>>> I created a Linaro JIRA card https://cards.linaro.org/browse/CARD-1563 >>>> to capture this work item. >>>> >>>> >>>> On 11 July 2014 08:22, Bill Fischofer <[email protected]> >>>> wrote: >>>> >>>>> I can certainly add it to the agenda so we can get wider feedback on >>>>> this subject. >>>>> >>>>> Bill >>>>> >>>>> >>>>> On Fri, Jul 11, 2014 at 5:40 AM, Mike Holmes <[email protected]> >>>>> wrote: >>>>> >>>>>> What is the outcome of this thread - do we need to look at cleanup >>>>>> APIs ? >>>>>> >>>>>> Are there any more opinions on the list ? >>>>>> My feeling is we have a partial use case discussion and a general >>>>>> feeling that it is a good idea from a few others, I propose we bring this >>>>>> up in the Tuesday Call if possible. >>>>>> >>>>>> Mike >>>>>> >>>>>> >>>>>> On 9 July 2014 04:19, Maxim Uvarov <[email protected]> wrote: >>>>>> >>>>>>> On 07/09/2014 11:58 AM, Alexandru Badicioiu wrote: >>>>>>> >>>>>>>> In addition to "close" calls for queues, pktios, buffer pools, etc >>>>>>>> there might be hardware resources requested from the specific device >>>>>>>> drivers during global and local initialization and required for queues, >>>>>>>> pktios, etc. >>>>>>>> These have to be returned to the drivers in order to avoid leaks >>>>>>>> and support application restart, for example. >>>>>>>> To handle this kind of cleanup I'm proposing to have explicit calls >>>>>>>> : odp_finish_local() and odp_finish_global() as counterparts for init >>>>>>>> calls. >>>>>>>> >>>>>>>> Alex >>>>>>>> >>>>>>> >>>>>>> +1 for odp_finish_xx() >>>>>>> >>>>>>> Maxim. >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 8 July 2014 19:42, Bill Fischofer <[email protected] >>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> The problem with an innocent-looking cleanup call is that is >>>>>>>> makes >>>>>>>> it easy to assume that ODP is keeping track of a lot of things >>>>>>>> that clearly it isn't. Does ODP know how many buffer pools have >>>>>>>> been allocated so that they'll be released at cleanup? Does ODP >>>>>>>> track which queues have been allocated or contain pending events >>>>>>>> that need to be flushed? Or threads? Or any number of other >>>>>>>> resources? Do we want ODP to be doing that sort of tracking? >>>>>>>> Clearly not. >>>>>>>> >>>>>>>> Graceful cleanup in general is a lot more complex than >>>>>>>> initialization because initialization starts from and idle >>>>>>>> system >>>>>>>> with nothing going on. By contrast, once a system is up and >>>>>>>> running there are all sorts of loosely-tracked things "in >>>>>>>> motion" >>>>>>>> that need to be stopped and recovered for an orderly return to >>>>>>>> an >>>>>>>> idle state (from which a new init can be started) can be >>>>>>>> successful. >>>>>>>> >>>>>>>> Abnormal termination processing doesn't have to worry about this >>>>>>>> because it makes no pretense of being surgical. I just don't >>>>>>>> see >>>>>>>> a real need for this sort of thing (and the resources needed to >>>>>>>> develop and test it) being high on the priority list for ODP >>>>>>>> given >>>>>>>> it's intended use. >>>>>>>> >>>>>>>> Whether or not we want to pursue this, we certainly need to add >>>>>>>> the "bracketing" close calls for things like queues, pktio, etc. >>>>>>>> first since they'd be needed as a precursor to any general >>>>>>>> cleanup. So I'd start with these. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Jul 8, 2014 at 10:35 AM, Mike Holmes >>>>>>>> <[email protected] <mailto:[email protected]>> wrote: >>>>>>>> >>>>>>>> I would couple of points >>>>>>>> >>>>>>>> 1. A clean up hook imposes no overhead if not needed, it >>>>>>>> can >>>>>>>> >>>>>>>> be compiled out transparently if there is no content >>>>>>>> for a >>>>>>>> given platform. >>>>>>>> 2. I find symmetry is generally beneficial, being able to >>>>>>>> >>>>>>>> undo something can simplify code and provide a contract >>>>>>>> with the application that it things are available for >>>>>>>> reuse at a defined point. >>>>>>>> 3. SoCs are used in complex systems where "hooks" can often >>>>>>>> >>>>>>>> allow use cases or "silicon patches" that expand on the >>>>>>>> originally imaged utility of a device. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 8 July 2014 10:25, Santosh Shukla >>>>>>>> <[email protected] <mailto:santosh.shukla@linaro. >>>>>>>> org>> >>>>>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>> On 8 July 2014 19:54, Bill Fischofer >>>>>>>> <[email protected] >>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> > By discarded I mean the old VM is terminated. All >>>>>>>> hypervisors automatically >>>>>>>> > clean up terminated VMs the same way that Linux cleans >>>>>>>> up terminated >>>>>>>> > processes. This does not involve the cooperation of >>>>>>>> the >>>>>>>> entity being >>>>>>>> > terminated since it may be in an error state and >>>>>>>> non-functional anyway. >>>>>>>> > >>>>>>>> >>>>>>>> Now, It make sense to me, thanks :) >>>>>>>> > >>>>>>>> > On Tue, Jul 8, 2014 at 9:21 AM, Santosh Shukla >>>>>>>> <[email protected] <mailto: >>>>>>>> [email protected]>> >>>>>>>> >>>>>>>> > wrote: >>>>>>>> >> >>>>>>>> >> On 8 July 2014 19:34, Bill Fischofer >>>>>>>> <[email protected] >>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> >> > VM migration is transparent to the VM, no? >>>>>>>> Quiescing >>>>>>>> operation (an >>>>>>>> >> > application function) is a different matter. In >>>>>>>> the >>>>>>>> case of NFV I'd >>>>>>>> >> > expect >>>>>>>> >> > instance A to be quiesced, (new) instance B to be >>>>>>>> started, and then >>>>>>>> >> > quiesced >>>>>>>> >> > flows to be restarted against instance B. >>>>>>>> Instance A >>>>>>>> is then discarded >>>>>>>> >> > so >>>>>>>> >> >>>>>>>> >> I am with till here except "discarded" - If you >>>>>>>> discard >>>>>>>> resource then >>>>>>>> >> those resource could be IO, Memory wont be reclaimed >>>>>>>> to >>>>>>>> hypervisor for >>>>>>>> >> use to another VM right? Or I am confused in >>>>>>>> understand >>>>>>>> discarded vs >>>>>>>> >> termination, I believe termination something mean to >>>>>>>> source VM >>>>>>>> >> explicitly free resources. >>>>>>>> >> >>>>>>>> >> > termination is a non-issue. >>>>>>>> >> >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > On Tue, Jul 8, 2014 at 8:58 AM, Lide, David >>>>>>>> <[email protected] <mailto:[email protected]>> wrote: >>>>>>>> >> >> >>>>>>>> >> >> VM migration for SDN use case (where data plane >>>>>>>> may >>>>>>>> get moved) would be >>>>>>>> >> >> a >>>>>>>> >> >> use case where a cleanup API would be required. >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> regards, >>>>>>>> >> >> >>>>>>>> >> >> dave >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> From: [email protected] >>>>>>>> <mailto:[email protected]> >>>>>>>> >> >> [mailto:[email protected] >>>>>>>> >>>>>>>> <mailto:[email protected]>] On Behalf Of >>>>>>>> Bill Fischofer >>>>>>>> >> >> Sent: Tuesday, July 08, 2014 9:52 AM >>>>>>>> >> >> To: Alexandru Badicioiu >>>>>>>> >> >> Cc: [email protected] >>>>>>>> <mailto:[email protected]> >>>>>>>> >>>>>>>> >> >> Subject: Re: [lng-odp] ODP cleanup API >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> I'd like to understand the use case for such an >>>>>>>> API. >>>>>>>> Generally >>>>>>>> >> >> embedded >>>>>>>> >> >> systems (and that's what data plane applications >>>>>>>> are--switches, >>>>>>>> >> >> routers, >>>>>>>> >> >> etc.) are designed for continuous operation once >>>>>>>> they are initialized. >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> On Tue, Jul 8, 2014 at 8:40 AM, Alexandru >>>>>>>> Badicioiu >>>>>>>> >> >> <[email protected] >>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>> >> >> >>>>>>>> >> >> Hi, >>>>>>>> >> >> >>>>>>>> >> >> I would like to get your comments related to >>>>>>>> having >>>>>>>> ODP cleanup API - >>>>>>>> >> >> the >>>>>>>> >> >> counterpart of init API. Perhaps not required for >>>>>>>> linux-generic >>>>>>>> >> >> implementation where all process memory, sockets, >>>>>>>> file handles are >>>>>>>> >> >> eventually returned to the OS, I think they are >>>>>>>> required to properly >>>>>>>> >> >> cleanup >>>>>>>> >> >> any allocation of HW resources. >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> Thanks, >>>>>>>> >> >> >>>>>>>> >> >> Alex >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> >> _______________________________________________ >>>>>>>> >> >> lng-odp mailing list >>>>>>>> >> >> [email protected] >>>>>>>> <mailto:[email protected]> >>>>>>>> >>>>>>>> >> >> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>>>> >> >> >>>>>>>> >> >> >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > >>>>>>>> >> > _______________________________________________ >>>>>>>> >> > lng-odp mailing list >>>>>>>> >> > [email protected] >>>>>>>> <mailto:[email protected]> >>>>>>>> >>>>>>>> >> > http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>>>> >> > >>>>>>>> > >>>>>>>> > >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> lng-odp mailing list >>>>>>>> [email protected] <mailto:[email protected]. >>>>>>>> org> >>>>>>>> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- *Mike Holmes* >>>>>>>> >>>>>>>> Linaro Technical Manager / Lead >>>>>>>> LNG - ODP >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> lng-odp mailing list >>>>>>>> [email protected] <mailto:[email protected]> >>>>>>>> >>>>>>>> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> lng-odp mailing list >>>>>>>> [email protected] >>>>>>>> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> lng-odp mailing list >>>>>>> [email protected] >>>>>>> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Mike Holmes* >>>>>> Linaro Technical Manager / Lead >>>>>> LNG - ODP >>>>>> >>>>>> _______________________________________________ >>>>>> lng-odp mailing list >>>>>> [email protected] >>>>>> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> *Mike Holmes* >>>> Linaro Technical Manager / Lead >>>> LNG - ODP >>>> >>> >>> >>> _______________________________________________ >>> lng-odp mailing list >>> [email protected] >>> http://lists.linaro.org/mailman/listinfo/lng-odp >>> >>> >> >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> http://lists.linaro.org/mailman/listinfo/lng-odp >> >> >
_______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
