On 04/28/2016 10:50 AM, Barry Spinney wrote:
Inter-process dynamic shared memory support.
This API provides a convenient method for multiple processes to share memory
using a persistent filesystem-based arena that is automatically mapped at the
same, fixed address in all processes sharing the arena. The application chooses
an address for the arena to be located at and a maximum size to which it can
grow, and the system manages coordinating access to memory mapped from the file
among the processes. Since the address is fixed, absolute pointer values, etc.,
may be safely stored into the arena.
The files are opened such that they are automatically closed if the process
calls exec() to start a new executable.
Thanks for copying me on this subject. As Christophe had mentioned in
an earlier email, in order to achieve pointer coherency across process
boundaries, the 'client' processes must be forked from the process which
does the initial allocation in order to inherit a copy of the initial
allocator's address map and then must avoid invalidating that address
map via any form of exec*() call. While this is a similar scenario to
that employed with the use of pthreads, I personally feel this is an
undesirable direction to take within ODP itself. It might make a good
addition to a helper suite.
As I understand it the primary reason for ODP is to promote application
portability and reuse. To my thinking this means ODP proper should make
no assumptions of address space layout across ODP threads.
While POSIX compliant systems may support these assumptions, there may
be other platforms which do not. Indeed there may be platforms in which
the combination of underlying hardware and OS do not support shared
memory at all. Therefore I regard it as a non-portable design if the
application assumes pointer validity across threads of execution which
may or may not have overlapping or even compatible address spaces. If
this is bad practice at the application level where the designer may
deliberately eschew portability in favor of performance, it is most
certainly bad practice to embed such assumptions in an underlying
foundational library.
Furthermore, I believe that when implementing a foundational library it
is extremely important to ensure that the library behaves consistently
with best practices in the rest of the platform environment. As a
general rule POSIX systems make no guarantees of pointer coherency
across process boundaries - so I do not favor a design which assumes
some pointers can be shared safely while others can not. I believe this
invites confusion and maintenance nightmares for the application designer.
In contrast, if the reference platform implementation of ODP shared
memory makes no assumptions about the address space layout of one ODP
thread versus another, it encourages application design which is
consistent with typical POSIX practice and available documentation. This
makes the library more intuitive to use and less prone to
misunderstanding and erroneous usage. I realize this might require
rethinking of some existing code, but I believe this will result in a
more robust and portable reference platform which will better serve as
an example for best implementation practices.
If my opinion regarding the importance of portability versus performance
is not shared then I question why the reference implementation should
bother to support process-based ODP threads at all.
The main reasons for using processes instead of pthreads include address
space isolation (or reconfiguration in a NUMA environment, for example)
or the lack of support for pthreads.
Address space reconfiguration invalidates pointer coherency across
processes, and obviously Linux has pthread support - so what would be
the use case for process-based ODP threads if they have to have pointer
coherency among themselves?
If process-based ODP threads are important then I believe for the sake
of consistency they should regard all pointers and memory map
configurations as thread-specific and not shareable. Any ODP data
constructs expected to be shared or inherited across ODP threads should
comply with this restriction regardless of whether the underlying
threads are implemented as processes or pthreads.
By the time odp_init_global() and odp_init_local() have completed
execution, there should be no differences in the ODP environment
inherited by processes as opposed to pthreads. Any shared data should
be resolved to process-local pointers. In the case of pthread-based ODP
threads executing with a data address map shared with the parent
process, this is a wash... in the case of shared memory objects pointers
might be resolved to local addresses the first time a descendant thread
needs to access a given shmem object. This could work the same way
whether the descendant ODP thread was a forked process with its own
address space or a cloned pthread sharing the address space of its
parent process. This approach also lends itself more readily to
file-based data sharing on platforms which lack memory-mapped file or
shared memory support.
With that I'll close this novel-length diatribe and wait for the flames
:-) If you stuck with it for this long, thanks for devoting so much
attention.
Gary Robertson
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
