Re: Rsync sequence
On Mon 19 Jul 2010, dschuett wrote: allow for a more efficient backup. I know I could just run them at different times in the same night, but I don't want to chance one not completing before the other starts. Can I just list them one after the other like so in crontab -e: # m h dom mon dow command 00 4 * * 7 rsync -auv --delete /Godfather/Music /Backup rsync -auv --delete /Godfather/Documents /Backup rsync -auv --delete /Godfather/Setups /Backup rsync -auv --delete /Godfather/Pictures /Backup rsync -auv --delete /Godfather/Backups /Backup rsync -auv --delete /Godfather/Videos /Backup This is hardly an rsync question... I would put the commands in a separate shell script file: #!/bin/sh rsync -auv --delete /Godfather/Music /Backup rsync -auv --delete /Godfather/Documents /Backup rsync -auv --delete /Godfather/Setups /Backup rsync -auv --delete /Godfather/Pictures /Backup rsync -auv --delete /Godfather/Backups /Backup rsync -auv --delete /Godfather/Videos /Backup Then make the file executable (chmod +x) and then put the name of the script in the crontab. Paul -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
RE: Rsync sequence
I have one script that has multiple rsync to a backup server. I do not want them all running at the same time but I want to be sure that they all complete. What I do is use the sh wait command link rsync . rsync . Wait rsync . rsync . wait And so on It will run the rsyncs in the background but will wait until both have finished until it runs the next two rsync. Robert -Original Message- From: rsync-boun...@lists.samba.org [mailto:rsync- boun...@lists.samba.org] On Behalf Of Paul Slootman Sent: Tuesday, July 20, 2010 5:19 AM To: dschuett Cc: rsync@lists.samba.org Subject: Re: Rsync sequence On Mon 19 Jul 2010, dschuett wrote: allow for a more efficient backup. I know I could just run them at different times in the same night, but I don't want to chance one not completing before the other starts. Can I just list them one after the other like so in crontab -e: # m h dom mon dow command 00 4 * * 7 rsync -auv --delete /Godfather/Music /Backup rsync -auv --delete /Godfather/Documents /Backup rsync -auv --delete /Godfather/Setups /Backup rsync -auv --delete /Godfather/Pictures /Backup rsync -auv --delete /Godfather/Backups /Backup rsync -auv --delete /Godfather/Videos /Backup This is hardly an rsync question... I would put the commands in a separate shell script file: #!/bin/sh rsync -auv --delete /Godfather/Music /Backup rsync -auv --delete /Godfather/Documents /Backup rsync -auv --delete /Godfather/Setups /Backup rsync -auv --delete /Godfather/Pictures /Backup rsync -auv --delete /Godfather/Backups /Backup rsync -auv --delete /Godfather/Videos /Backup Then make the file executable (chmod +x) and then put the name of the script in the crontab. Paul -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart- questions.html -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Rsync sequence
Can I just list them one after the other like so in crontab -e You may wish to also consider using a single rsync command : rsync -auv --delete \ /Godfather/Documents \ /Godfather/Setups \ /Godfather/Pictures \ /Godfather/Backups \ /Godfather/Videos \ /Backup # backup destination With this approach I am not sure which order the source directories will be copied? If the execution order is critical then you are probably best to stick to a separate command for each source directory in a script as has been previously suggested. Finally, I recommend excessing extreme caution when using the --delete option. I hope this helps. --- This email is protected by LBackup, an open source backup solution. http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --compare-dest weirdness
Hi Henri Thanks very much. LBackup looks good, but i need a solution which only copies files which have changed, and does not link the others, as the folder is destined for a tar archive. Also, the copied files must be fully functional, i.e. must be whole files, not just differences like rdiff. this is why im using rsync. Using checksum works, however with over 600gb to backup it is not a viable option. the -i shows that most are being copied due to time differences, so in theory -t should work? This does in fact work on a little test setup on my work laptop, i will test it properly when i get home tonight. Thanks Regards Kevin Murray On 20 July 2010 12:40, Henri Shustak henri.shus...@gmail.com wrote: interestingly, i tried to see if something was wrong with my statments by doing: mkdir ~/rsynctest/dir1 mkdir ~/rsynctest/dir2 mkdir ~/rsynctest/dir3 nano ~/rsynctest/dir1/file1 (wrote the line hello world and saved) nano ~/rsynctest/dir1/file2 (wrote the line hello and saved) cp ~/rsynctest/dir1/file* ~/rsynctest/dir2/ checked md5sums of both files in both dirs to ensure they were identical the did rsync -rvu --compare-dest=/home/kevin/rsynctest/dir2/ /home/kevin/rsynctest/dir1/ /home/kevin/rsynctest/dir3/ the two files from dir1 were copied to dir3, even though identical copies, with the same names, existed in dir2. i also tried this again with the compare dir relative to the dest dir, i.e.: rsync -rvu --compare-dest=../dir2/ /home/kevin/rsynctest/dir1/ /home/kevin/rsynctest/dir3/ again with same result I thought about this further and I think the problem you are having is that the modification times are not being preserved and you are not using the --checksum option. As such, I suggest that you add the --times or --checksum option and see if that helps. I suspect that this will solve your problem. Let me know =:^) - This email is protected by LBackup, an open source backup solution. Free as in freedom; LBackup is licensed under the GNU GPL http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
How to block this fiel name from being transfered via rsync?
1272574453.H315376P30969.dwhs38.dwhs.net,S=1859 I tried *.*.dwhs38.dwhs.net,* *.*.dwhs38.dwhs.net But it still transfers. Also if there is a tutorial on blocking rsync files please let me know. Thanks, Charles www.ezlinuxadmin.com - Chaze http://www.charlesyarbrough.com My Blog | DWHS http://www.dwhs.net Web Hosting -- View this message in context: http://old.nabble.com/How-to-block-this-fiel-name-from-being-transfered-via-rsync--tp29221007p29221007.html Sent from the Samba - rsync mailing list archive at Nabble.com. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync algorithm
Hi, I am learning rsync mechanism. My understanding is that rsync reads a file into multiple chunks and calculates every byte for MD5 and rolling checksum for each chunk.. On the other side of file system, it compares two checksums for each chunk in the same file to decide if two files are equal or not. Is it correct? If it is correct, should I worry the calculation time if I have thousands large files each contains 10 MB? Please correct me. Thank you. Kind regards, hce -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: --compare-dest weirdness
the -i shows that most are being copied due to time differences, so in theory -t should work? This does in fact work on a little test setup on my work laptop, i will test it properly when i get home tonight. I am glad that the -t option is working with your test. If you decide to use LBackup and have a full hard linked backup then a possible option is to have a post action script which passes a list of files and directories which have been updated or created to tar for archive creation. To enable rsync session logging add the following line to the LBackup backup configuration file : enable_rsync_session_log=YES Then using tar with the -n option you would be able to feed in a list of newly created or modified files. Depending upon what kind of information you want to preserve something like the following could do the trick. However, please keep in mind that you may need to add or remove tar various options. tar -c -n -z -v -f /tmp/new_archive.tar.gz -T /tmp/list_to_archive.txt If you opted for such an approach you could potentially stop using the --compare-dest option and rely upon the output from rsync for a list of files you would like to archive. All the best with your backup script. With any luck the rsync -t option will work with you backup as well as it did for your testing. --- This email is protected by LBackup, an open source backup solution. http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
