Drew-
There is a build of GZip
(under Debian I think) that has an rsync
friendly option. I have posted a patch
for zlib to this newsgroup with code that
makes zlib rsync friendly (along with some
extra optimizations over the GZip implementation).
Take a look in the list
archives and y
Gary-
Shachar Shemesh
has created an AES encryption module that
generates RSync friendly encrypted files.
Google his name and you should be able to
find it.
- K
Original Message
>
I am using a SSH tunnel for transit, so just
the backup siteGH+-
Shachar-
I think Wayne is mostly
pointing you to the correct location here.
If you look at the code where the checksum
is computer, you'll find that the rolling
checksum actually consists of two parts -
one is the sum of all of the byte values
in the window, the other is the offset weig
e two rolling
checksum components, instead of just appending
one to the other...
Just some comments that
may help!
- K
Original Message
>
Kevin Day wrote:> Shachar->
> True enough - with one additional
thought - if the block size is set >
to b
nsfer is
the biggest cause of elapsed time for big
files that don't change much.
It may be that the square
root of file size for block size isn't appropriate
for files as big as you are working with...
- K
Original Message
>
Kevin Day wrote:> I
Shachar,It does use a hash table.
rsync adds the two components of the
rolling checksum together to come up with
a 16 bit hash, and performs a table lookup.
The table contains an offset into the
sorted rsync checksum table (which contains
both weak and strong checksums), and does
a line
Hi all-
My test results so far indicate a pretty decent improvement in overall rsync
performance when using a slightly more sophisticated checksum calculation.
The attached patch has the required changes (in hindsight, I should have
compressed this using zlib with the new algorithm :-) ).
Some
RSync-
Sorry about that. I accidentally
reversed the default values I had specified
for the window size and the reset block size.
This only effects the outcome if you accept
all default values, and don't initialize
with your own.
The correct default values in the deflate.c
file (line
Per my comment yesterday:
> However, I just kicked off another parametric study varying the rsync block
> size and the block mask size, while holding the window size constant. If
> there is an optimization avaialble for rsync block size, then we should see a
> dependency in these
results.
T
dent on the rsync block size, but that the curve was
relatively flat. This is true for nice, uncompressed data. However, when we
start looking at this behavior for compressed data streams (rsync friendly,of
course), then the curve is nowhere near as flat. So optimization of the rsync
block siz
Hi, all -
I've been digging into the question of using rsync to sync up compressed data,
and read with interest the conversations regarding the rsync friendly patch
available for gzip (I belive Rusty provided this patch). For anyone
interested, the message thread is archived at
http://lists.
11 matches
Mail list logo