On May 22, 2009 09:58 -0700, Hayes, Robert N wrote: > We have been troubleshooting a major performance issue with out > Lustre systems when switching from 1.6.4.3 to 1.6.5. Using a > patchless client and running a single thread, single file (50GB) dd > copy, the 1.6.4.3 client wrote up to 1.3GB/s. Using the same system and > changing only the client to 1.6.5, write speed went down to 700MB/s. > > We found one configuration option different by default between the > two clients. In 1.6.4.3, checksum is disabled by default. In 1.6.5, > checksum is enabled by default. Disabling checksum on the 1.6.5 client > resulted in a performance increase to 1.1GB/s. Question - are there > other option changes between 1.6.4.3 and 1.6.5 that might account for > the remaining 200MB/s loss in performance?
I believe that the checksumming change is the major one. This mostly affects the single-threaded high-bandwidth case as you found out. With most HPC jobs the IO is multi-threaded and all of the cores on the client can contribute. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
