On 05/09 16:54:26, Bill Fischofer wrote: > The purpose of this thread is to summarize the discussions we've had over > the past week on this topic and to foster discussion. The goal here is to > reach consensus on a complete specification for ODP in this area that can > be added to the User Guide for Monarch, as well as to form the basis for > any expansion work needed in this area for Tiger Moth. > > Background > ========= > > An ODP instance is bounded by everything between an odp_init_global() > through a matching odp_term_global() call. Applications are expected to > make use of a single ODP instance, though systems may support multiple > concurrent applications, each of which have their own independent ODP > instances. Communication between different applications is via independent > of whether either of them are making use of ODP and is expected to be via > standard OS or I/O mechanisms. The ODP Pktio IPC mechanism is experimental > and not enabled by default in Monarch. > > Threads and Processes > ================== > > ODP itself does not define a threading model or provide APIs for creating > or destroying threads. The odp_thread_t type is intended to be an > abstraction of whatever underlying threading model is available to the > application via OS or other means outside the scope of ODP. ODP provides a > limited number of thread APIs for basic identification purposes. These > include: > > - odp_thread_id() - to obtain a unique thread ID for the current thread > - odp_thread_count() - to determine the number of threads that are > running in the current ODP instance > - odp_thread_count_max() - to determine the maximum number of threads > that can be supported by the current ODP instance > - odp_thread_type - to determine whether the current thread is a worker > or control thread > > In particular, ODP does not specify whether threads are implemented within > a single address space or separate address spaces. > > To facilitate operation in Linux environments, ODP provides a series of > "helper" APIs that layer on top of Linux pthreads. These are found in > helper/include/odp/helper/linux.h and include: > > - odph_linux_pthread_create() - to create a pthread > - odph_linux_pthread_join() - to wait for pthreads to exit > - odph_linux_process_fork() - to fork a linux process > - odph_linux_process_fork_n() - to fork a number of linux processes in a > single call > - odph_linux_ process_wait_n() - to wait for a number of linux processes > to exit > > These helper functions provide a means of setting pthread attributes and > process affinity flags as part of their operation, but again being helpers > and not ODP APIs they do not determine ODP API semantics. > > Issues > ===== > > Several issues have been identified as a result of the above. The main > issue is what, if anything, does ODP have to say about whether addresses > derived from handles returned by ODP APIs may or may not be shared among > threads? The consensus seems to be that ODP itself is silent on this > subject. While it is recognized that it is convenient for applications to > share addresses between threads, whether or not they can do so safely is a > function of both application design and the underlying ODP implementation. > > For example, if an application consists of multiple threads prior to the > odp_global_init() call, then it is undefined whether handles, let alone > addresses, derived from that ODP instance are sharable with other > application threads. If the other application threads are pthreads sharing > the same address space, then perhaps yes, but if they are separate > processes then from the perspective of the ODP instance they are > effectively separate applications and hence things like ODP handles would > have no meaning within them. > > It seems that ODP should define the scope of an ODP instance (i.e., which > threads ODP handles derived from that instance are valid) to be the thread > that calls odp_init_global() and its direct descendants. > > Second, regarding addresses derived from ODP handles (e.g., those returned > by odp_shm_addr(), odp_packet_data(), etc.) again it seems that these can > be shared between threads only under two conditions: > > 1. The threads are within the same ODP instance > 2. The threads occupy a single shared address space or are forked from > the the same root thread that created the handle from which the returned > address is derived. > > Point 2 is again a function of both application design and the underlying > implementation. If the application creates all of its resources (pools, > shms, etc. during initialization prior to launching threads then these two > criteria should be satisfied). However, if the application starts creating > threads (particularly those that are not sharing the same address space) > and then those child threads are the ones that create pools, shms, etc., > then in general it does not seem safe to assume that addresses derived from > these handles have meaning to any other thread that is not a direct > descendent of the creating thread. > > From a portability and best practices standpoint it therefore seems that we > should encourage all global resource allocation to be performed by the > application during initialization after it has called odp_init_global() but > prior to creating other threads. > > Please comment and raise other cases that should be considered.
> _______________________________________________ > lng-odp mailing list > [email protected] > https://lists.linaro.org/mailman/listinfo/lng-odp Are we trying to do anything with 'ODP thread' or 'ODP shmem' which is beyond what is already provided by an operating system, or a C library, or a coroutine or 'green thread' library or higher level language runtime? If so, please do explain! If not, or just not today, perhaps these things are simply a matter of implementation detail rather than exposed as an API to the application. If apps must link in libodphelper in addition to libodp it is an equivalent dependency regardless of whether it is considered part of the API or not. There are a few other interesting findings when poking around with ldd after building and running the tests. Perhaps ODP example applications may demonstrate how to construct applications which make use of ODP thread-safe libraries including multi-process environments. This may have a larger effect on implementation details than specific changes to function declarations exposed to applications. TLS, test framework, and other assumptions in today's reference implementation may need to change. A reference implementation is one from which other implementations are derived or used to port. A 2-3kloc patch series for running _all_ tests where a helper is used for concurrent programming and then modified to fork processes instead of pthreads is a catalyst for driving change. If certain objects need to reside in shared memory between two processes _and_ the backing memory _must_ be mapped into the same VA space in each process, is it acceptible for an ODP implementation to use a fixed range in the VA space somewhere below the TASK_UNMAPPED_BASE address? This address is compiled into the Linux kernel and used when mapping things when an address is unspecified. Or, the application may provide the virtual address to ODP. If a fixed VA range in the process is too strict of a requirement, please explain the application use case. _______________________________________________ lng-odp mailing list [email protected] https://lists.linaro.org/mailman/listinfo/lng-odp
