On Fri, 5 Nov 2010, Rob Latham wrote: ... > Let's consider two process A and B who are doing non-contiguous > writes. A writes to the even bytes and B writes to the odd bytes. > Data sieving means that both processes will read a big chunk of the > file, modify the local copy of the buffer and then write it. OK, so > Lustre will ensure that either all of A's write or all of B's write > will show up, but that still means one process stomps all over the > other. > > so you need something to serialize the read-modify-write. > > Note that if you use *collective* i/o then much of this no longer > becomes a problem. Because it's collective the processes can use MPI > to coordinate and eliminate this false sharing problem. > > Note also that if you never ever use a shared file, then I guess you > can use local locks. But if you are always doing one file per > processor instead of a few shared files, you will end up with at the > very least a mess of files. ...
Brilliant, very informative - thanks Rob. Hopefully this will help aid other people in my situation; this question has cropped-up a few times on this list and never seems to make it to a consensus - so different people have chosen different options. Best wishes, Mark -- ----------------------------------------------------------------- Mark Dixon Email : [email protected] HPC/Grid Systems Support Tel (int): 35429 Information Systems Services Tel (ext): +44(0)113 343 5429 University of Leeds, LS2 9JT, UK ----------------------------------------------------------------- _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
