On Sat, 2007-12-22 at 17:02 -0600, [EMAIL PROTECTED] wrote: > One further aspect of this that would be nice to solve too, is that > using that technique in this case means that rsync has to check > something like 550,000 (literally) messages to get thru a sync. > > That number will dwindle to a fraction of that eventually but it will > be a while, and of course I need dayly backups of the other stuff. > > My test run has been running since I read mail reply.. probably 15 > minutes so far. > > I'm guessing there isn't much of any way around that... and just have > schedule the backups accordingly.
Right, there's certainly no way to avoid considering all the source files, and no easy way to avoid considering excluded files (to discover that they are excluded). But do try the incremental recursion mode in development versions of rsync 3.0.0 if you haven't already; it will reduce memory usage and help streamline the process. Incidentally, if the sender could fill the socket with file data and then go incrementally wading through the excluded files while it is waiting on the network, that might *really* help. This enhancement would be easier to make if the complex I/O control flow in each process were refactored into coroutines that trade control at defined points when they block. I wonder if there is a good coroutine library for C that could handle such things as selecting on the combined set of fds when all coroutines block and resuming the coroutine whose fd became ready. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
