On Fri, Nov 05, 2010 at 05:11:47PM +0000, Mark Dixon wrote: > Assuming I do not disable data sieving, which of the following > options will most likely give me correct behaviour? > > 1) Enable Lustre's cluster-wide coherent fcntl locks. > I think you're going to have to go with this approach
> 2) Cheat, and enable Lustre's (cheaper) fcntl locks that are only > coherent on an individual computer, on the assumption that Lustre's > own internal locking mechanisms will "do the right thing". Let's consider two process A and B who are doing non-contiguous writes. A writes to the even bytes and B writes to the odd bytes. Data sieving means that both processes will read a big chunk of the file, modify the local copy of the buffer and then write it. OK, so Lustre will ensure that either all of A's write or all of B's write will show up, but that still means one process stomps all over the other. so you need something to serialize the read-modify-write. Note that if you use *collective* i/o then much of this no longer becomes a problem. Because it's collective the processes can use MPI to coordinate and eliminate this false sharing problem. Note also that if you never ever use a shared file, then I guess you can use local locks. But if you are always doing one file per processor instead of a few shared files, you will end up with at the very least a mess of files. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
