On Wed, Feb 22, 2006 at 02:30:34PM -0500, Andrew Pochinsky wrote: > 3. Kill pvfs2-client-core > # ps auxww | grep pvfs2 > .... > # kill 45673 > # time cp /data/P1/some-file /dev/null > real 0m38.448s > user 0m0.023s > sys 0m0.652s > > -- faster than before, but still a lot slower than pvfs2-cp: > > 4. Run pvfs2-cp on the same file: > # pvfs2-cp -t /data/P1/some-file /dev/null > Wrote 169869851 bytes in 1.708348 seconds. 94.828753 MB/seconds > > Do you want me to do the third test on your list and remove the kernel > module?
It looks like that 3rd test won't be necessary yet. There are two issues here: One issue is that /bin/cp uses the blocksize of the destination when deciding how big a buffer to use for each system call. /dev/null has a 4k blksize, so /bin/cp will read from pvfs2 in 4k pieces. Pretty inneficient as you've seen. pvfs2-cp defaults to a 4 MB blocksize and you get the much better performance you observed. (This has been addressed in the development version of coreutils and will be part of coreutils-6.0 whenever it is released) You observed about 4 MB/sec through the VFS. We'd expect reading from PVFS2 to be slower but that is *really* slow. To verify, try comparing pvfs2-cp and /bin/cp to write data to pvfs2. I suspect pvfs2-cp will be faster than /bin/cp, but if things are OK, both *should* be in the same ballpark. The second (and more tricky) issue is what causes the VFS interface to get into such a bad state? It's encouraging that restarting pvfs2-client-core helps. If you can give me an idea of what sort of workload puts PVFS2 into a bad state, I can try to reproduce it here. ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Labs, IL USA B29D F333 664A 4280 315B _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
