Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-20 Thread Gleb Natapov
On Fri, May 18, 2007 at 06:04:07PM -0400, Patrick Geoffray wrote: > Hi Gleb, > > Gleb Natapov wrote: > > new madvice flag was implemented that allows userspace to mark certain > > memory to not be copied to a child process. This memory is not mapped in > > a child at all, no even VMA created for i

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-18 Thread Patrick Geoffray
Hi Gleb, Gleb Natapov wrote: new madvice flag was implemented that allows userspace to mark certain memory to not be copied to a child process. This memory is not mapped in a child at all, no even VMA created for it. In the parent this memory is Ah, that explain your previous mention of segfau

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-18 Thread Gleb Natapov
On Thu, May 17, 2007 at 02:57:22PM -0400, Patrick Geoffray wrote: > gshipman wrote: > >> The fork() problem is due to memory registration aggravated by > >> registration cache. Memory registration in itself is a hack from > >> the OS > >> point of view, and you already know a lot about the variou

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-18 Thread Gleb Natapov
On Thu, May 17, 2007 at 02:35:02PM -0400, Patrick Geoffray wrote: > Brian Barrett wrote: > > On the other hand, since the MPI standard explicitly says you're not > > allowed to call fork() or system() during the MPI application and > > Does it ? The MPI spec says that you should not access buf

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-18 Thread Gleb Natapov
On Thu, May 17, 2007 at 12:30:51PM -0400, Patrick Geoffray wrote: > Jeff Squyres wrote: > > Moving to devel; this question seems worthwhile to push out to the > > general development community. > > > > I've been coming across an increasing number of customers and other > > random OMPI users wh

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-18 Thread Gleb Natapov
On Thu, May 17, 2007 at 10:20:35AM -0600, Brian Barrett wrote: > On the other hand, since the MPI standard explicitly says you're not > allowed to call fork() or system() during the MPI application and > sense the network should really cope with this in some way, if it > further complicates t

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-17 Thread Patrick Geoffray
gshipman wrote: The fork() problem is due to memory registration aggravated by registration cache. Memory registration in itself is a hack from the OS point of view, and you already know a lot about the various problems related to registration cache. So Gleb is indicating that this is a probl

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-17 Thread gshipman
The fork() problem is due to memory registration aggravated by registration cache. Memory registration in itself is a hack from the OS point of view, and you already know a lot about the various problems related to registration cache. So Gleb is indicating that this is a problem in the pipeli

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-17 Thread Patrick Geoffray
Brian Barrett wrote: On the other hand, since the MPI standard explicitly says you're not allowed to call fork() or system() during the MPI application and Does it ? The MPI spec says that you should not access buffers that have been committed to MPI (pending asynchronous send or recv buffe

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-17 Thread Patrick Geoffray
Jeff Squyres wrote: Moving to devel; this question seems worthwhile to push out to the general development community. I've been coming across an increasing number of customers and other random OMPI users who use system(). So if there's zero impact on performance and it doesn't make the co

Re: [OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-17 Thread Brian Barrett
On the other hand, since the MPI standard explicitly says you're not allowed to call fork() or system() during the MPI application and sense the network should really cope with this in some way, if it further complicates the code *at all*, I'm strongly against it. Especially since it won't

[OMPI devel] [RFC] Send data from the end of a buffer during pipeline proto

2007-05-17 Thread Jeff Squyres
Moving to devel; this question seems worthwhile to push out to the general development community. I've been coming across an increasing number of customers and other random OMPI users who use system(). So if there's zero impact on performance and it doesn't make the code [more] incredibly