rsync already has a memory-hogging issue. Imagine having it search your
entire directory tree, checksumming all files, storing and sending them
all, comparing both lists looking for matching date/time/checksums to
guess where you've moved files to. You'd be better off to use a wrapper
the tools you move files with, keeping a replayable log, and have your
mirrors retrieve and replay that log, before doing the rsync.
Tim Conway
[EMAIL PROTECTED]
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
Phil Howard <[EMAIL PROTECTED]>
Sent by: [EMAIL PROTECTED]
12/03/2001 09:04 AM
To: [EMAIL PROTECTED]
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject: Re: Why does one of there work and the other doesn't
Classification:
On Mon, Dec 03, 2001 at 12:09:16AM +1100, Martin Pool wrote:
| On 30 Nov 2001, Randy Kramer <[EMAIL PROTECTED]> wrote:
|
| > I am not sure which end the 100 bytes per file applies to, and I guess
| > that is the RAM memory footprint?. Does rsync need 100 bytes for each
| > file that might be transferred during a session (all files in the
| > specified directory(ies)), or does it need only 100 bytes as it does
one
| > file at a time?
|
| At the moment that is 100B for all files to be transferred in the
| whole session. This is a big limit to scalability at the moment, and
| a goal of mine is to reduce it to at most holding file information
| from a single directory in memory.
It would still be nice to have an option to gather all files at once,
but this will be of value if it also gathers all the checksums and
syncronizes files moves that have happened on the source end by
doing the syncronization of the moved file to the new location using
the old (checksum matched) file on the destination end. Right now
if a file gets moved from one location to another (especially in a
different directory, which is often the case with a re-organization)
things get retransferred even though most every file already exists
somewhere on the destination.
--
-----------------------------------------------------------------
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
-----------------------------------------------------------------
