Re: Batch mode scenario (use case)
Matt wrote: Suresh Govindachar wrote: How about the following scheme? open /e/cmds/foo do till end-of-file: begin-continue discard lines till there is a match to some/path/projects/c_a keep lines till there is a match to a directory outside c_a Note that the lines being kept could be data/commands corresponding to files inside c_a end-continue end-do write kept lines to new batch file This is a clever idea, but I think it would be very difficult to subset the batch file correctly since the rsync protocol is quite complex (and binary). For one thing, the new batch file would need to contain the protocol initialization and at least the relevant portion of the file list. You might be able to accomplish this by keeping everything until the first file transfer is logged, but that won't work if the sender used incremental recursion because the relevant portion of the file list will be randomly interspersed with file transfers. If you need to restore a subtree from a batch file, another thing you could do is make a modified copy of rsync just for the job with a strncmp thrown in to skip all file transfers except the ones in the subtree. Also, instead of a batch file, you might consider copying to a new destination with --compare-dest to the original; that gives you a tree of changed files from which you can easily restore files or subtrees. Thanks for pointing out --compare-dest: I verified it on a test directory and this feature does indeed offer a resolution to the situation described in the original post. (I have a follow up question which belongs to another thread.) --Suresh -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Batch mode scenario (use case)
On Sun, Sep 09, 2007 at 11:17:05PM +, Suresh Govindachar wrote: Responding to the question of how to use the batch file /e/cmds/foo created via the command: rsync -a --only-write-batch=/e/cmds/foo /c/home/wer/work/ /e/gold to selectively restore a subdirectory of work/ such as some/path/projects/c_a into a new location such as /f/new_home/wer/work which has a copy of some/path/projects/c_a gotten from /e/gold but is otherwise empty, Matt wrote: Rsync currently doesn't provide a good way to do this. It would be great if sender filters could be used to control which file-list entries from the batch file are processed. For now, a hackish way to quickly recover a subdirectory is to --read-batch to a destination that has been set up specially so that the user cannot write to anything in the destination except the subdirectory to be recovered. Is there a specification for the format of the batch file? The batch file format is simply a dump of rsync's network protocol. That way, replaying the batch file is quite similar to performing the original sync, except for the bytes come from the file instead of from the socket. At least, that's the way it worked last time I looked, which was several years ago. -chris Might it be possible to write a script to read /e/cmds/foo and create a new batch file that would correspond to having been created while rsync'ing --only-write-batch of work/path/c_a with gold/path/c_a? Thanks, --Suresh -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Batch mode scenario (use case)
On 9/9/07, Suresh Govindachar [EMAIL PROTECTED] wrote: How about the following scheme? open /e/cmds/foo do till end-of-file: begin-continue discard lines till there is a match to some/path/projects/c_a keep lines till there is a match to a directory outside c_a Note that the lines being kept could be data/commands corresponding to files inside c_a end-continue end-do write kept lines to new batch file This is a clever idea, but I think it would be very difficult to subset the batch file correctly since the rsync protocol is quite complex (and binary). For one thing, the new batch file would need to contain the protocol initialization and at least the relevant portion of the file list. You might be able to accomplish this by keeping everything until the first file transfer is logged, but that won't work if the sender used incremental recursion because the relevant portion of the file list will be randomly interspersed with file transfers. If you need to restore a subtree from a batch file, another thing you could do is make a modified copy of rsync just for the job with a strncmp thrown in to skip all file transfers except the ones in the subtree. Also, instead of a batch file, you might consider copying to a new destination with --compare-dest to the original; that gives you a tree of changed files from which you can easily restore files or subtrees. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Batch mode scenario (use case)
Hello, I came up with this scenario of the use of batch mode while thinking of back-up schemes to use for myself. However, it could be that the last step needed in this scenario is not supported by rsync! Here's the scenario: At one time, /c/home/wer/work and /e/gold had identical content and were really huge (say, 200 GBytes). After some complex, intricate work, Mr. Wer did: rsync -a --only-write-batch=/e/cmds/foo /c/home/wer/work/ /e/gold Very soon after that, /c dies; and Wer needs to provide his Customer_A stuff that used to be in /c/home/wer/work/some/path/projects/c_a He needs this stuff immediately, meaning, Wer cannot wait to rebuild everything from /e/gold. However, gold/some/path/projects/c_a is merely 5 Megs -- but it is not the latest version. Wer starts by creating the directory /f/new_home/wer/work, and then uses -R as follows: rsync -aR /e/gold/./some/path/projects/c_a /f/new_home/wer/work which gives him an old version of c_a inside /f/new_home/wer/work/some/path/projects Question: How can Wer use the batch file /e/cmds/foo to recover c_a in a timely manner (recall /e/gold is 200 Gigs and Wer cannot wait to rebuild the entire /f/new_home/wer/work; and he needs c_a immediately)? Thanks, --Suresh -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Batch mode scenario (use case)
On 9/9/07, Suresh Govindachar [EMAIL PROTECTED] wrote: Question: How can Wer use the batch file /e/cmds/foo to recover c_a in a timely manner (recall /e/gold is 200 Gigs and Wer cannot wait to rebuild the entire /f/new_home/wer/work; and he needs c_a immediately)? Rsync currently doesn't provide a good way to do this. It would be great if sender filters could be used to control which file-list entries from the batch file are processed. For now, a hackish way to quickly recover a subdirectory is to --read-batch to a destination that has been set up specially so that the user cannot write to anything in the destination except the subdirectory to be recovered. Matt -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Batch mode scenario (use case)
Responding to the question of how to use the batch file /e/cmds/foo created via the command: rsync -a --only-write-batch=/e/cmds/foo /c/home/wer/work/ /e/gold to selectively restore a subdirectory of work/ such as some/path/projects/c_a into a new location such as /f/new_home/wer/work which has a copy of some/path/projects/c_a gotten from /e/gold but is otherwise empty, Matt wrote: Rsync currently doesn't provide a good way to do this. It would be great if sender filters could be used to control which file-list entries from the batch file are processed. For now, a hackish way to quickly recover a subdirectory is to --read-batch to a destination that has been set up specially so that the user cannot write to anything in the destination except the subdirectory to be recovered. Is there a specification for the format of the batch file? Might it be possible to write a script to read /e/cmds/foo and create a new batch file that would correspond to having been created while rsync'ing --only-write-batch of work/path/c_a with gold/path/c_a? Thanks, --Suresh -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
