On Wed, Jan 17, 2007 at 11:47:37AM -0800, Chris Halstead wrote:
> Copy tests were done by simply timing 'cp' from the mounted PVFS volume to
> local /dev/shm (to avoid filesystem latency).
>
> When I strace the pvfs server process, I see the server writing to the TCP
> socket in chunks no larger than ~4k:
>
> writev(11, [{"\277\312\0\0\4\0\0\0\213^\0\0\0\0\0\0(\20\0\0\0\0\0\0", 24},
> {"\274\v\0\0\2\0\0\0!\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0\0>"..., 4136}], 2) =
> 4160
What you are seeing there is that the cp(1) program uses the blocksize
of the destination file system when deciding how big a buffer to use.
About a year ago we got the coreutils maintainers to take into account
the source and destination file systems, but that's only available in
coreutils-6.0 and newer (so it might take a while longer to show up in
distributions).
> I've tried tuning kernel-level TCP settings and
> TCPBufferSend/TCPBufferReceive in the pvfs config, but I never see a change
> in the size of the writes to the TCP socket. Is this normal? Is there a
> tuning parameter that I'm missing that will influence the size of the
> writes?
Two things to try: 1: you can upgrade your coreutils or 2: you can use
pvfs2-cp, which will bypass the kernel VFS and use a 4 MB buffer size
by default.
==rob
--
Rob Latham
Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF
Argonne National Lab, IL USA B29D F333 664A 4280 315B
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
