On 25 May 2016 at 13:11, Christophe Milard <[email protected]> wrote:
> > > On 25 May 2016 at 12:18, Ola Liljedahl <[email protected]> wrote: > >> I have attempted to summarise and comment on the discussion. This is >> partly >> intended for an external audience so sorry if I seem to repeat things >> already "solved". Unfortunately I couldn't attend yesterdays public call >> but I read the notes. >> >> Does ODP define the threading model? The ODP documentation does refer to >> “ODP threads” and the ODP API specifies thread identifiers (small >> integers, >> starting from 1 I think) and defines “thread masks” which define which >> threads participate in certain processing (e.g. scheduler groups). >> >> The - mostly accepted answer - is no. ODP does not define the thread >> model. >> This is instead inherited from the execution environment (which the ODP >> implementation in question is designed for). ODP will not have any thread >> API’s (e.g. for setting CPU affinity for a “ODP” thread), the application >> will have to use any native API but there might also be ODP helpers for >> this (there are some new CPPU pinning ODP helper functions for e.g. >> pthreads and Linux processes). Such helpers are not proper ODP calls >> because they might not be meaningful (or available) in some execution >> environment (e.g. CPU pinning would probably work very differently in a >> bare metal environment where thread equals CPU, helpers wouldn't be >> necessary and could only serve to confuse the ODP implementation). >> >> There are two major flavours of the thread model: multiple threads in a >> single memory space (“process” in Linux) - "single-process” - and multiple >> threads in different memory spaces - “multi-process”. How can ODP >> resources >> be shared by ODP threads in different thread models and execution >> environments? >> >> ODP handles are global and can be used by all ODP threads in the same >> application (regardless of thread model). ODP shared memory regions are >> also shared by all ODP threads. As ODP shared memory is really an OS >> concept, the underlying OS may allow even wider sharing (e.g. sharing of >> shared memory regions between different (ODP) applications). >> >> (Virtual) addresses derived from ODP handles (e.g. address for shared >> memory region, address for buffer or packet data) may not be valid for all >> threads. The ODP implementation specifies whether addresses derived from >> ODP objects can be shared by threads. >> > > I do not agree fully with that: I would rather say that the OS rules > applies for theses addresses: I don't think the linux-generic odp > implementation should have to specify more than saying that pointers are > sharable withing the linux rules,(i.e. between linux threads and "before > fork". It boils down to the same, probably. > The rules and limitations of memory and pointer sharability derive from the execution environment (e.g. the OS). And ODP implementation is designed for some specific execution environment (e.g. Linux user space) and inherits those rules and limitations. > Having said that, we will surely have to specify on what objects the OS > rules apply: if process A creates a pool P and then forks B and C, it is > likely that addresses retrieved from packets allocated from P will be > shareable as P was created before the fork. ODP will have to specify these > kind of behaviour as nothing says when the memory is actually allocated. > Do we want this rules to be common over all ODP implementations (e.g. to > be able to say that on any ODP, the OS rules apply to the pool rather than > individual packets)? > I think we should make strong recommendations for the behaviour we see as vital to ODP's success. An ODP implementation may still not follow those recommendations but they better have good reasons for doing so (i.e. not mainstream HW architecture or SW execution environment). We will not be able to depend on such ODP implementations for ODP's mainstream success. I didn't follow you here with regards to pools vs. packets. Both pools and packets have ODP handles which should be globally sharable so I see no difference between them. > >> There is a recommendation (from ODP public call on 20160524) that "for >> best >> portability applications should not assume that such addresses have >> validity beyond thread scope". I don’t agree with this recommendation as I >> think it pushes application writers towards complicated designs were >> pointers to data structures (which may be allocated from e.g. ODP shared >> memory regions or buffer pools, what else are shared memory regions for?) >> cannot be shared directly (e.g. as a pointer in “C”), every memory >> reference would have to be a composite of some ODP handle and an offset or >> similar. Porting and writing applications to ODP will become very >> difficult. >> >> What I think is also missing is a strong recommendation for ODP >> implementations to support sharing of addresses between threads in at >> least >> the single-process model (perhaps under Linux as an additional >> constraint). >> How the multi-process model (if supported) works would be implementation >> specific (possibly depending on the capabilities of the execution >> environment). There could exist implementations that only support the >> multiprocess model (e.g. because the underlying HW does not provide >> coherent shared memory and symmetric multiprocessing), I don't expect such >> platforms or ODP implementations to be mainstream, SMP HW has won. Such >> ODP >> implementation can still implement the ODP API but will require >> applications to understand the platform-specific limitations to e.g. >> thread >> and memory models. Absolute portability of ODP applications to any ODP >> implementation (however unusual execution environment or memory model) is >> not an absolute goal and can be sacrificed (and was never guaranteed as >> different ODP implementations will support different number of queues, >> classification rules etc). Applications will most likely also be using >> API's and facilities beyond ODP so automatic portability cannot be >> guaranteed in the general case. >> >> If ODP implementations do not support sharing memory *and* corresponding >> addresses between threads, I think they (we) are making ODP application >> design dangerously complicated and thus much less likely that there will >> actually be useful ODP applications written. And without applications, >> there won’t be any open ODP ecosystem. >> >> > I think we are getting closer to some kind of agreement :-) > > >> Den som gapar över för mycket mister ofta hela stycket. >> >> -- Ola >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> https://lists.linaro.org/mailman/listinfo/lng-odp >> > > _______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
