You could use find to build a filter to use with rsync, then update the filter every few days if it takes too long to create.

I have used a script to build a filter on the source server to exclude anything over 5 days old, invoked when the sync starts, but it only parses around 2000 files per run.


On 2/07/2015 2:34 a.m., Ken Chase wrote:
What is taking time, scanning inodes on the destination, or recopying the entire
backup because of either source read speed, target write speed or a slow 
between them?

Do you keep a full new backup every day, or are you just overwriting the target


On Wed, Jul 01, 2015 at 10:06:57AM +0200, Dirk van Deun said:
   >> If your goal is to reduce storage, and scanning inodes doesnt matter,
   >> use --link-dest for targets. However, that'll keep a backup for every
   >> time that you run it, by link-desting yesterday's copy.
   >The goal was not to reduce storage, it was to reduce work.  A full
   >rsync takes more than the whole night, and the destination server is
   >almost unusable for anything else when it is doing its rsyncs.  I
   >am sorry if this was unclear.  I just want to give rsync a hint that
   >comparing files and directories that are older than one week on
   >the source side is a waste of time and effort, as the rsync is done
   >every day, so they can safely be assumed to be in sync already.
   >Dirk van Deun
   >Ceterum censeo Redmond delendum

Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options:
Before posting, read:

Reply via email to