Re: [PATCH] Batch-mode rewrite

2004-07-23 Thread Wayne Davison
On Thu, Jul 22, 2004 at 07:27:28PM -0400, Chris Shoemaker wrote: I see now that the hard-link post-processing looks pretty firmly attached to the generator, with the 4 hard_link_check() calls in recv_generator(). I'll look at this some more later. Though the current code generates the

Re: [PATCH] Batch-mode rewrite

2004-07-23 Thread Wayne Davison
On Thu, Jul 22, 2004 at 07:01:12PM -0400, Chris Shoemaker wrote: BTW, your index notification scheme isn't in CVS, is it? It is. It's just not the one that was in the g2r-basis-filename.diff file (where the extra pipe from the generator conveyed the index info back to the receiver in addition

Re: [PATCH] Batch-mode rewrite

2004-07-22 Thread Chris Shoemaker
Wayne, I took a crack a batch-mode test case, to try avoid batch-mode regressions. Does this look reasonable? Also, I came across a confusing typo (I think.) Also attached. -chris Index: testsuite/README.testsuite === RCS file:

Re: [PATCH] Batch-mode rewrite

2004-07-22 Thread Wayne Davison
On Wed, Jul 21, 2004 at 03:54:11PM -0400, Chris Shoemaker wrote: What data exactly? I thought: 1) all recv-to-gen communications went through the error_pipe[] fds. Yes, that became true when I got rid of the extra pipe that used to separate the redo values from the error messages.

Re: [PATCH] Batch-mode rewrite

2004-07-22 Thread Wayne Davison
On Thu, Jul 22, 2004 at 04:10:41PM -0400, Chris Shoemaker wrote: I took a crack a batch-mode test case, to try avoid batch-mode regressions. Does this look reasonable? Yes, that looks good. I made some minor changes and checked it in. Also, I came across a confusing typo (I think.) Also

Re: [PATCH] Batch-mode rewrite

2004-07-22 Thread Chris Shoemaker
On Thu, Jul 22, 2004 at 06:36:27PM -0700, Wayne Davison wrote: On Wed, Jul 21, 2004 at 03:54:11PM -0400, Chris Shoemaker wrote: What data exactly? I thought: 1) all recv-to-gen communications went through the error_pipe[] fds. Yes, that became true when I got rid of the extra pipe

Re: [PATCH] Batch-mode rewrite

2004-07-22 Thread Chris Shoemaker
On Thu, Jul 22, 2004 at 07:01:12PM -0400, Chris Shoemaker wrote: On Thu, Jul 22, 2004 at 06:36:27PM -0700, Wayne Davison wrote: And don't forget the hard-link post-processing -- it would also need to happen after the receiver finished its job. Ok, it's late and this new patch seems like

Re: [PATCH] Batch-mode rewrite

2004-07-21 Thread Chris Shoemaker
On Tue, Jul 20, 2004 at 09:10:22AM -0700, Wayne Davison wrote: On Mon, Jul 19, 2004 at 06:18:49PM -0400, Chris Shoemaker wrote: Ok, how about this: Instead of index notification, run the generator and receiver serially. I had wondered about that too, but the problem is that the generator

Re: [PATCH] Batch-mode rewrite

2004-07-21 Thread Chris Shoemaker
Geesh, I forgot the attachment, (again). On Wed, Jul 21, 2004 at 03:54:11PM -0400, Chris Shoemaker wrote: On Tue, Jul 20, 2004 at 09:10:22AM -0700, Wayne Davison wrote: On Mon, Jul 19, 2004 at 06:18:49PM -0400, Chris Shoemaker wrote: avoid this and also to separate the post-generator

Re: [PATCH] Batch-mode rewrite

2004-07-20 Thread Wayne Davison
On Mon, Jul 19, 2004 at 06:18:49PM -0400, Chris Shoemaker wrote: Ok, how about this: Instead of index notification, run the generator and receiver serially. I had wondered about that too, but the problem is that the generator expects data from the receiver, so we'd need to add special code to

Re: [PATCH] Batch-mode rewrite

2004-07-19 Thread Chris Shoemaker
On Sun, Jul 18, 2004 at 08:58:43PM -0700, Wayne Davison wrote: On Sun, Jul 18, 2004 at 06:20:59PM -0400, Chris Shoemaker wrote: On Sun, Jul 18, 2004 at 05:25:18PM -0700, Wayne Davison wrote: So, perhaps we should go ahead and save off the exclude list in the batch file and force

Re: [PATCH] Batch-mode rewrite

2004-07-19 Thread Chris Shoemaker
On Sun, Jul 18, 2004 at 08:37:03PM -0700, Wayne Davison wrote: On Sun, Jul 18, 2004 at 06:20:59PM -0400, Chris Shoemaker wrote: Could a simplified version of this index notification take place over the existing error-pipe pair? The data is traveling in the opposite direction for what we

Re: [PATCH] Batch-mode rewrite

2004-07-19 Thread Chris Shoemaker
helps to attach, eh? On Mon, Jul 19, 2004 at 06:18:49PM -0400, Chris Shoemaker wrote: On Sun, Jul 18, 2004 at 08:37:03PM -0700, Wayne Davison wrote: On Sun, Jul 18, 2004 at 06:20:59PM -0400, Chris Shoemaker wrote: Could a simplified version of this index notification take place over the

Re: [PATCH] Batch-mode rewrite

2004-07-18 Thread Wayne Davison
On Fri, Jul 16, 2004 at 04:40:41PM -0400, Chris Shoemaker wrote: If I understand your changes, the files-from stuff you're skipping is only the flagging of the fd, no actual communication. Yes, you're right there -- my comment wasn't accurate. I was worried about the files-from data getting in

Re: [PATCH] Batch-mode rewrite

2004-07-18 Thread Chris Shoemaker
On Sun, Jul 18, 2004 at 05:25:18PM -0700, Wayne Davison wrote: Seems like the two choices we have are: (1) Force the excludes into the batch file and read them in the local- to-local batch-reading transfer. (2) Require the user to re-specify the excludes if they want the same update

Re: [PATCH] Batch-mode rewrite

2004-07-18 Thread Wayne Davison
On Sun, Jul 18, 2004 at 06:20:59PM -0400, Chris Shoemaker wrote: Could a simplified version of this index notification take place over the existing error-pipe pair? The data is traveling in the opposite direction for what we need (and it's not bidirectional). ..wayne.. -- To unsubscribe or

Re: [PATCH] Batch-mode rewrite

2004-07-18 Thread Wayne Davison
On Sun, Jul 18, 2004 at 06:20:59PM -0400, Chris Shoemaker wrote: On Sun, Jul 18, 2004 at 05:25:18PM -0700, Wayne Davison wrote: So, perhaps we should go ahead and save off the exclude list in the batch file and force read_batch mode to read them? I'm leaning in this direction. I was too

Re: [PATCH] Batch-mode rewrite

2004-07-17 Thread Wayne Davison
On Fri, Jul 16, 2004 at 04:48:55PM -0400, Chris Shoemaker wrote: what's this part about standard input? It reads the batch data from stdin (the current manpage says list where it should say batch data). This is quite useful, as seen in the example in the manpage: $ rsync

Re: [PATCH] Batch-mode rewrite

2004-07-15 Thread Wayne Davison
On Thu, Jul 15, 2004 at 08:04:56PM +0300, Eran Tromer wrote: About the batch-mode rewrite: perhaps this would be a good opportunity to make checksum_seed random also in batch mode, and store it in the batch file, instead of using a fixed checksum_seed? This has been done, though the random

Re: [PATCH] Batch-mode rewrite

2004-07-14 Thread Chris Shoemaker
There it goes... On Wed, Jul 14, 2004 at 12:16:45AM -0700, Wayne Davison wrote: On Tue, Jul 13, 2004 at 04:40:39PM -0400, Chris Shoemaker wrote: Do you see any reason to keep FIXED_CHECKSUM_SEED around? It doesn't hurt anthing, but I don't see a use for it. You're right -- the new

Re: [PATCH] Batch-mode rewrite, update to man page

2004-07-14 Thread Chris Shoemaker
I've attached an update to the man page regarding batch mode. I didn't change the statement about batch mode being experimental, but maybe we should consider modifying it. It did serve well to manage my expectations when I first tried batch mode and found that it didn't work at all for

Re: [PATCH] Batch-mode rewrite

2004-07-14 Thread Wayne Davison
I did some work refining your patch a little, and liked the result so much I went ahead and checked it into CVS. I'd appreciate it if you could give my changes a look to see if I messed anything up. The most important changes I made were: - Delay the start of batch writing until after any

Re: [PATCH] Batch-mode rewrite

2004-07-13 Thread Chris Shoemaker
Wayne, Do you see any reason to keep FIXED_CHECKSUM_SEED around? It doesn't hurt anthing, but I don't see a use for it. -chris -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Re: [PATCH] Batch-mode rewrite

2004-07-13 Thread Craig Barratt
Chris Shoemaker writes: Do you see any reason to keep FIXED_CHECKSUM_SEED around? It doesn't hurt anthing, but I don't see a use for it. So long as the --checksum-seed=N option remains, I'm ok getting rid of FIXED_CHECKSUM_SEED. Craig -- To unsubscribe or change options:

Re: [PATCH] Batch-mode rewrite

2004-07-12 Thread Wayne Davison
First, a summary of my thoughts: This looks to be a much simpler way to integrate batch support into rsync than what we currently have. I'm quite interested to see this refined further. Nice work! Some other comments: On Sun, Jul 11, 2004 at 06:08:04PM -0400, Chris Shoemaker wrote: 1) I

Re: [PATCH] Batch-mode rewrite

2004-07-12 Thread Wayne Davison
On Mon, Jul 12, 2004 at 02:47:30PM -0400, Chris Shoemaker wrote: On Mon, Jul 12, 2004 at 12:34:38PM -0700, Wayne Davison wrote: rsync -av --read-batch=foo localhost:/path/bar Hmm, that case wasn't in my testing. Did I introduce this limitation or has it always been so? The old batch

Re: [PATCH] Batch-mode rewrite

2004-07-12 Thread Wayne Davison
On Mon, Jul 12, 2004 at 02:47:30PM -0400, Chris Shoemaker wrote: On Mon, Jul 12, 2004 at 12:34:38PM -0700, Wayne Davison wrote: Another thing I noticed was that a local --write-batch copy behaved as if --whole-file had been specified. Hmm, I forgot about that. Q: Shouldn't this be set in

Re: [PATCH] Batch-mode rewrite

2004-07-12 Thread Chris Shoemaker
Wayne, A couple more thoughts: On Mon, Jul 12, 2004 at 12:34:38PM -0700, Wayne Davison wrote: First, a summary of my thoughts: One thought here: would it make things simpler to separate the option- parsing variables (read_batch write_batch) from a set of variables that would

Re: [PATCH] Batch-mode rewrite

2004-07-12 Thread Chris Shoemaker
On Mon, Jul 12, 2004 at 07:11:04PM -0700, Wayne Davison wrote: On Mon, Jul 12, 2004 at 02:47:30PM -0400, Chris Shoemaker wrote: On Mon, Jul 12, 2004 at 12:34:38PM -0700, Wayne Davison wrote: Another thing I noticed was that a local --write-batch copy behaved as if --whole-file had been

Re: [PATCH] Batch-mode rewrite

2004-07-12 Thread Chris Shoemaker
On Mon, Jul 12, 2004 at 02:47:30PM -0400, Chris Shoemaker wrote: Ok. diff -cu it is. I used -b because the auto-tab feature in emacs sometimes causes noisy whitespace changes in the diff. I'll incorporate your comments and rediff. Ok, actually it seems diff -cu isn't right. Instead I

[PATCH] Batch-mode rewrite

2004-07-11 Thread Chris Shoemaker
Wayne, Please consider the attached patch. This applies to the current CVS, and is independant of patches/local-batch.diff. As a matter of fact, I'm sure it would conflict heavily with local-batch.diff. This version of batch mode has a couple distinguishing features: Write-batch