In that scenario, you need to set the session directories to point somewhere other than /tmp. I believe you will find that in our FAQs as this has been a recurring problem. The shared memory backing file resides in the session directory tree, so if that is NFS mounted, your performance will stink.
People with that setup generally point the session dir at a ramdisk area, but anywhere in ram will do. On Apr 12, 2010, at 9:10 AM, Oliver Geisler wrote: > > Quoting Ashley Pittman <ash...@pittman.co.uk>: > >> >> On 10 Apr 2010, at 04:51, Eugene Loh wrote: >> >>> Why is shared-memory performance about four orders of magnitude slower than >>> it should be? The processes are communicating via memory that's shared by >>> having the processes all mmap the same file into their address spaces. Is >>> it possible that with the newer kernels, operations to that shared file are >>> going all the way out to disk? Maybe you don't know the answer, but >>> hopefully someone on this mail list can provide some insight. >> >> Is the /tmp filesystem on NFS by any chance? >> > > Yes, /tmp is on NFS .. those are diskless nodes all without disks and no swap > space mounted. > > Maybe I should setup one of the nodes with a disk, so I could try the > difference. > > (Sorry, but I may return results next week since, I am out of office right > now) > > Thanks > oli > > > >> Ashley, >> >> -- >> >> Ashley Pittman, Bath, UK. >> >> Padb - A parallel job inspection tool for cluster computing >> http://padb.pittman.org.uk >> >> >> _______________________________________________ >> devel mailing list >> de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/devel >> >> -- >> This message has been scanned for viruses and >> dangerous content by MailScanner, and is >> believed to be clean. >> >> > > > > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
