On Fri, 8 Dec 2006, Martin Pokorny wrote:
I'm trying to determine whether multiple processes on multiple nodes can
simultaneously mmap a common file on a lustre file system, write to it,
and produce a coherent result (I'm using OpenMPI to spawn the processes
and provide synchronization barriers). In my tests, each process is
writing a 10,000 byte segment of the file, but is memory mapping the
whole file.
With my very limited understanding of its internals, I would expect Lustre
to provide it's best results with stripe-aligned, or at least page-aligned
write areas. With your test, Lustre's internal locking may well be under
high stress.
What I'm seeing is that if I use 40 processes or less, the file is
(usually) produced correctly. However, when I try my test with 50 or 100
processes, I rarely get a good result; in fact, the tests seem to hang.
What I've found is that, when the test fails, there are processes
remaining on the lustre client nodes that are using up all the CPU, but
never seem to finish. I have no trouble interrupting the running
processes in this case.
While I'm not entirely sure of the result I should expect in these
tests, I certainly would expect the test to finish. Does anyone have any
comments or ideas?
Maybe some locks are ping-ponging between clients? Or it could be a real
deadlock too.
CFS engineers will probably suggest you to turn on certain debugging flags
and post the resulting logs, which only them can analyze. ;-)
Cheers,
--
Jean-Marc Saffroy - [EMAIL PROTECTED]
_______________________________________________
Lustre-discuss mailing list
[email protected]
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
