On Sep 17, 2009 13:28 +0200, Lukas Hejtmanek wrote: > is it possible to optimize Lustre so that is supports really large directories > (with 30k small files in it)? We have 8 physical clients which process jpeg > files stored on Lustre volume and I get sooner or later client freezes - ls in > Lustre directory waits forever. I there something I could do to improve > performance?
We regularly test directories with 1M files in them, so I don't consider 30k files to be a large directory. If you are always using small files there are different things you can do to optimize performance, such as using RAID-1+0 instead of RAID-5/6 on the OSTs. > The lustre server is Build Version: > 1.8.0-19700101010000-PRISTINE-.usr.src.lustre-prod.linux-2.6.22.19-2.6.22.19 > > The lustre client is Build Version: > 1.6.7.1-19700101010000-PRISTINE-.scratch.xhejtman.suse-2.6.22.17-0.1-2.6.22.17-0.1-xen-lustre Using a more common kernel (e.g. RHEL5.3) means a lot more people are testing the same code as you are. > Lustre: stable-MDT0000-mdc-ffff8802855b7800: Connection to service > stable-MDT0000 via nid x.x....@tcp was lost; in progress operations using > this service will wait for recovery to complete. As other's mentioned, this seems like a network or server problem. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
