On Nov 26, 2007 3:32 PM, Robin Humble <[EMAIL PROTECTED]> wrote: > >> I'm seeing what can only be described as dismal striped write > >> performance from lustre 1.6.3 clients :-/ > >> 1.6.2 and 1.6.1 clients are fine. 1.6.4rc3 clients (from cvs a couple > >> of days ago) are also terrible.
I have 3 OSTs capable to deliver 300+ MB/sec each for large streaming writes with 1M blocksize. On one client, with one OST I can see almost all this bandwidth over Infiniband. If I run three processes in parallel on this very client, each writing into a separate OST, I arrive to 520 MB/sec aggregate (3 streams at approx 170+ MB/sec each). If I try to stripe over these three OSTs on this client, performance of one stream drops to 60+ MB/sec. Changing stripesize to a smaller one (1/3 MB) makes things worse. Writing with larger block sizes (9M, 30M) does not improve things. Increasing the stripesize to 25 MB allows to approach the speed of a single OST, as one would expect (blocks are round robined over all three OSTs). But never more. Zeroing checksums on the client does not help. Will now be downgrading the client to 1.6.2 to see if this helps. Andrei. _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
