On Jun 22, 2007 18:01 -0300, Balagopal Pillai wrote: > Looks like the problem is solved. I just had to bump up > RxDescriptors for e1000 > to prevent the dropping frames issue. After that i could run two > bonnie++ from two nodes in parallel without issues -
Great. > Fergal McCarthy wrote: > >Also when you do the ls, are you doing it from a login window or from a > >script? If from a login window be aware that most GNI/Linux environments > >setup an alias for ls that turns on colourisation of files by default, > >based on file type, and that means that the ls command is going to be > >stat()ing every file which means that a simple ls does the same amount > >of work as ls -l. This is true. > >And if you didn't already know ls -l (i.e. stat()) is > >a very expensive operation in Lustre, especially if any of the files in > >a directory are being modified concurrently with the ls being invoked... > >If you stat() a file you have to talk to the MDT to get the general file > >attributes, and then query every OST that holds data objects for the > >file to get the size of each to be able to calculate the total file size > >to be able to return the necessary results to the stat() call... And if > >any of the objects are being modified it is worse because the OST must > >ask the clients to flush their pending changes to the objects so that an > >accurate size can be determined, which means that a ls -l of a directory > >can cause heavy I/O as clients are forced to flush dirty data to the > >servers. This part isn't true for a very long time (since Lustre 1.0 or something). Since "ls -l" information can already be out-of-date by the time it is printed to the screen (especially because GNU ls always buffers the output even if you ask it not to sort) there is no way to have 100% accurate size for "ls -l". As a result, Lustre does a "glimpse" of the size on the client which is holding the "end of file" lock (i.e. what is the current size of the file, including unwritten data in the client cache) and returns this to the client doing the "ls -l". That avoids having to flush the lock on the client doing the write, and still allows for "impatient user doing ls -l to see file size change". Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
