Ah, of course. I wasn't even thinking of that. Using pvfs2-cp copies the
file in under 3sec consistently. I'll make sure that my automation scripts
use pvfs2-cp isntead of cp.
Thanks for the help!
-chris
On 1/17/07 12:09 PM, "Robert Latham" <[EMAIL PROTECTED]> wrote:
> On Wed, Jan 17, 2007 at 11:47:37AM -0800, Chris Halstead wrote:
>> Copy tests were done by simply timing 'cp' from the mounted PVFS volume to
>> local /dev/shm (to avoid filesystem latency).
>>
>> When I strace the pvfs server process, I see the server writing to the TCP
>> socket in chunks no larger than ~4k:
>>
>> writev(11, [{"\277\312\0\0\4\0\0\0\213^\0\0\0\0\0\0(\20\0\0\0\0\0\0", 24},
>> {"\274\v\0\0\2\0\0\0!\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0\0>"..., 4136}], 2) =
>> 4160
>
> What you are seeing there is that the cp(1) program uses the blocksize
> of the destination file system when deciding how big a buffer to use.
> About a year ago we got the coreutils maintainers to take into account
> the source and destination file systems, but that's only available in
> coreutils-6.0 and newer (so it might take a while longer to show up in
> distributions).
>
>> I've tried tuning kernel-level TCP settings and
>> TCPBufferSend/TCPBufferReceive in the pvfs config, but I never see a change
>> in the size of the writes to the TCP socket. Is this normal? Is there a
>> tuning parameter that I'm missing that will influence the size of the
>> writes?
>
> Two things to try: 1: you can upgrade your coreutils or 2: you can use
> pvfs2-cp, which will bypass the kernel VFS and use a 4 MB buffer size
> by default.
>
> ==rob
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
