Hi, To explain things, I'm working on Kerrighed (www.kerrighed.org) and I am trying to find a system "better than nfs" for 4 to 256 clients (atm, maybe more later). The fs should : - client cache coherent - the most POSIX compatible - the most efficient in "general use".
Lustre seems indeed pretty good for client cache coherency and (it seems) POSIX compliance (ie: I can't fine other fs with same properties). > I don't think it is realistic to expect that a cache-coherent distributed > filesystem that can scale to 10000's of clients is also performing as fast as > a single client on a local filesystem. That Lustre is completing in 4:19 vs. > 3:50 on the local filesystem (12% slower) is a pretty good result for Lustre, > I think. To be more exact, lustre on 4 nodes is completing on 4:19 vs 3:50 on one node :) I know a filesystem can't be good for everything, my question could be understood as "is there some basic (or a bit more complex) tunning to improve compilation/ I/O on small files". Lustre seems, as I said, good for cache coherency and POSIX compliance (maybe note the best, but definitively not the worst). > We're of course working on improving Lustre performance for this kind of > situation, but it isn't really a priority for most of our customers. I don't > want to discourage you from using Lustre, and of course I'd also like Lustre > to be faster than even the local filesystem but you should also look at the > other benefits. If you know some other filesystem with such properties, I can try them, but I did not find any of them (it also must be free software). > > A more fair test might be to do the local-node compile, and then copy the > kernel and all the modules to each of the client nodes, since Lustre is also > making the output files available on all of the clients. It is also > worthwhile (if you have the time) to determine whether your "make -j 16" is > CPU bound, or IO bound on the local filesystem? You might try pre-staging > all of the input files on the client nodes, and have the compiler output go > into a separate directory (not sure if this is possible with linux kernel > compiles) so that the output files created during the run do not invalidate > the directory caches. I will try this. > For comparison, on the same two systems (local fs vs. Lustre) try writing > 32*10GB files from 32 clients (use rsh or NFS or whatever you want to > transport data from clients to local filesystem) and see how performance > compares. :-) I would try to find some more "practical" tests, maybe something video related on some "big" videos, or some parrallel work on big images, or ... Thanks for your answers, Regards, Maxence -- Maxence DUNNEWIND Contact : [email protected] Site : http://www.dunnewind.net 06 32 39 39 93 GPG : 18AE 61E4 D0B0 1C7C AAC9 E40D 4D39 68DB 0D2E B533
signature.asc
Description: Digital signature
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
