Ok, I think I've bashed cpdup's new features into shape on HEAD.
Beware that the updated cpdup must be running on both sides of the
link to use the new feature and there is no endian conversion.
The new feature is '-pN', e.g. -p16, which parallelizes operations
when the source and/or destination is a remote host specification.
This should significantly speed up operations over slow links, and
even over fast links.
I originally didn't intend to implement this sort of feature but
at the moment my offsite backup box has 100ms of latency due to a
routing snafu well beyond my control. Needless to say, 10 files
checked per second is a bit too slow.
One final item not yet solved is that apparently cpdup's individual
write()'s to its pipe to ssh is resulting in ssh (which turns off nagle)
to send out one TCP packet per request. haven't found a way to tell
ssh to leave nagle on for the batch link, since interactive response is
not really needed when using the -pN option. This is resulting in
fairly expensive and unnecessary packet overhead. If anyone has any
ideas on how to fix ssh I'm all ears. I'd rather not gang the writes
in cpdup, it would be kinda messy to do that.
Once this has been tested well enough I will MFC it to 1.12.
-Matt
