On Sat, Nov 19, 2005 at 10:49:07AM -0500, Phillip Susi wrote: > I don't see why the filesystem's cluster size should have a thing to do > with the buffer size used to copy files. For optimal performance, the > larger the buffer, the better. Diminishing returns applies of course, > so at some point the increase in buffer size results in little to no > further increase in performance, so that's the size you should use. I > believe that the optimal size is about 64 KB.
In local file systems, i'm sure you are correct. If you are working with a remote file system, however, the optimal size is on the order of megabytes, not kilobytes. For a specific example, consider the PVFS2 file system, where the plateau in "blocksize vs. bandwitdh" is two orders of magnitude larger than 64 KB. PVFS2 is a parallel file system for linux clusters. I am not nearly as familiar with Lustre, GPFS, or GFS, but I suspect those filesystems too would benefit from block sizes larger than 64 KB. Are you taking umbrage at the idea of using st_blksize to direct how large the transfer size should be for I/O? I don't know what other purpose st_blksize should have, nor are there any other fields which are remotely valid for that purpose. Thanks for your feedback. ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Labs, IL USA B29D F333 664A 4280 315B _______________________________________________ Bug-coreutils mailing list [email protected] http://lists.gnu.org/mailman/listinfo/bug-coreutils
