Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Sylvain Jeaugey
On Fri, 11 Jun 2010, Jeff Squyres wrote: On Jun 11, 2010, at 5:43 AM, Paul H. Hargrove wrote: Interesting. Do you think this behavior of the linux kernel would change if the file was unlink()ed after attach ? After a little talk with kernel guys, it seems that unlinking wouldn't change

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Paul H. Hargrove
Sylvain Jeaugey wrote: On Thu, 10 Jun 2010, Paul H. Hargrove wrote: [snip] As for why mmap is slower. When the file is on a real (not tmpfs or other ramdisk) I am 95% certain that this is an artifact of the Linux swapper/pager behavior which is thinking it is being smart by "swapping

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Paul H. Hargrove
Chris, I think that "reclaiming pages from the page cache" is the PROBLEM, not the solution. If I understand you correctly a lower value of "swappiness" means that the ANONYMOUS pages of an application's stack and heap are less likely to be subject to swap I/O. However, the concern here

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-11 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/06/10 18:43, Paul H. Hargrove wrote: > When the file is on a real (not tmpfs or other ramdisk) I am 95% certain > that this is an artifact of the Linux swapper/pager behavior which is > thinking it is being smart by "swapping ahead". Even when

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-10 Thread Sylvain Jeaugey
On Thu, 10 Jun 2010, Jeff Squyres wrote: Sam -- if the shmat stuff fails because the limits are too low, it'll (silently) fall back to the mmap module, right? From my experience, it completely disabled the sm component. Having a nice fallback would be indeed a very Good thing. Sylvain

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-10 Thread Jeff Squyres
On Jun 10, 2010, at 4:57 AM, Sylvain Jeaugey wrote: > > As for why mmap is slower. When the file is on a real (not tmpfs or other > > ramdisk) I am 95% certain that this is an artifact of the Linux > > swapper/pager > > behavior which is thinking it is being smart by "swapping ahead". Even

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-10 Thread Jeff Squyres
On Jun 10, 2010, at 4:43 AM, Paul H. Hargrove wrote: > One should not ignore the option of POSIX shared memory: shm_open() and > shm_unlink(). When present this mechanism usually does not suffer from > the small (eg 32MB) limits of SysV, and uses a "filename" (in an > abstract namespace) which

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-10 Thread Sylvain Jeaugey
On Thu, 10 Jun 2010, Paul H. Hargrove wrote: One should not ignore the option of POSIX shared memory: shm_open() and shm_unlink(). When present this mechanism usually does not suffer from the small (eg 32MB) limits of SysV, and uses a "filename" (in an abstract namespace) which can portably

Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing

2010-06-10 Thread Sylvain Jeaugey
On Wed, 9 Jun 2010, Jeff Squyres wrote: On Jun 9, 2010, at 3:26 PM, Samuel K. Gutierrez wrote: System V shared memory cleanup is a concern only if a process dies in between shmat and shmctl IPC_RMID. Shared memory segment cleanup should happen automagically in most cases, including abnormal