Minimize log file: get rid of .d...p..... send
I'm just starting with rsync, googled and experienced a lot, but get stuck with minimizing the log-file: I only want to see the real changes: new, updated and deleted files/directories in the log-file. The LOG-file always contains a lot of lines like: 2010/08/31 20:19:01 [24573] .d...p. send home/hans/Desktop 2010/08/31 20:19:05 [24573] .d...p. send home/hans/Documents/PCPa I did not change anything, also not the permissions. So these directories are unchanged! Question: how to get rid of these lines, while keeping lines like: 2010/08/31 20:19:01 [24573] f.stp. send home/hans/2bedited.txt 2010/08/31 20:19:01 [24573] f+ send home/hans/Documents/newfile.txt Want to use rscync for a manual backup (now and then) and my situation is: Source: workstation with PCLinuxOS 2010. Destination: a NAS (LaCie Ethernet Disk mini) with a share called Backup, mounted as /mnt/backup: [h...@localhost mnt]$ ls -l ... drwxr-xr-x 4 hans hans0 2010-08-31 20:18 backup/ ... [h...@localhost mnt]$ ls -l backup total 0 drwxr-xr-x 2 hans hans 0 2010-08-31 19:39 deleted/ drwxr-xr-x 9 hans hans 0 2010-08-31 20:18 hans/ drwxr-xr-x 2 hans hans 0 2010-08-31 20:18 updated/ [h...@localhost mnt]$ the rsync command I use is: rsync -axhhX --stats --delete --delete-excluded --backup --backup-dir-dels=deleted/`date +%Y-%m-%d` --backup-dir=updated/`date +%Y-%m-%d` --suffix=.`date +%H%M` --log-file=/home/hans/rsync-files/rsync-log.`date +%Y%m%d-%H%M` --log-file-format=%i %o %f --exclude-from=/home/hans/rsync-files/exclude.txt /home/hans /mnt/backup NB: also the old mailing list thread rsync 2.6.4 - directory always updated from John Stoffel in 2005 did not help me (I'm not that good in *nix file system stuff) Best regards, Hans Troost -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
On Thu, 2010-09-02 at 11:38 +0200, hans.tro...@telfort.nl wrote: I'm just starting with rsync, googled and experienced a lot, but get stuck with minimizing the log-file: I only want to see the real changes: new, updated and deleted files/directories in the log-file. The LOG-file always contains a lot of lines like: 2010/08/31 20:19:01 [24573] .d...p. send home/hans/Desktop 2010/08/31 20:19:05 [24573] .d...p. send home/hans/Documents/PCPa I did not change anything, also not the permissions. So these directories are unchanged! Yes, but are they the same on source and destination? The behavior you observe could happen if the destination filesystem is unable to store permissions and silently ignores chmod(2) calls. In that case, you should disable the -p option. Question: how to get rid of these lines, while keeping lines like: 2010/08/31 20:19:01 [24573] f.stp. send home/hans/2bedited.txt 2010/08/31 20:19:01 [24573] f+ send home/hans/Documents/newfile.txt To get rid of lines with only attribute changes, you can set a log format that does not contain %i. But I would suggest first looking into why they are being printed. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: OSX 10.6.4 error with -R option
I have had reports of problems with the -R option on OSX 10.6.4. Just tested it myself and found this odd result: When I run this dtruss -f path/to/rsync -aHAXNR --fileflags --force-change --protect-decmpfs --stats -v /Users/astrid/Documents/main.m /Users/astrid/Desktop/rrr it produces the expected results with the relative folder paths in place /Users/astrid/Desktop/rrr/Users/astrid/Desktop/main.m but copying file from Desktop itself dtruss -f path/to/rsync -aHAXNR --fileflags --force-change --protect-decmpfs --stats -v /Users/astrid/Desktop/main.m /Users/astrid/Desktop/rrr /Users/astrid/Desktop/rrr/Users/astrid/ Problem solved Sorry. I didn't even think to check permissions on the Desktop folder, the first logical thing to check. Somehow they had been changed. Fixed them and -R works fine now. I believe other users are likely having permissions issues too. I have made the same mistake myself and have had the issue reported to me on multiple occasions in the past. This is why LBackup now checks the permissions are enabled on the destination volume. Perhaps when rsync runs on OS X systems it should default to a mode where a permissions are enabled check is performed on the destination volume and then a warning could be reported if they are not enabled? Just a possibility to avoid many more people having this same issue? Quoted below is an example of the output from LBackup when the permissions on a destination drive is not enabled : WARNING! : Permissions are disabled on backup destination volume. It is recommended that permissions on the destination volume are enabled. Failure to enable permissions will most likely result in the hard link system failing. Permissions may be enabled by issuing the following command as root : /usr/sbin/vsdbutil -a /Volumes/volume_name Perhaps adding something like this directly into rsync is a good idea for Mac OS X users? Any comments? - This email is protected by LBackup, an open source backup solution. http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: compare a directory with itself -- differences found
well, when I tell rsync to compare my home directory with itself, it reports many differences. I'm using rsync version 3.0.7 protocol version 30. I was hoping to use it to verify my backup. What kind of differences are reported. Also are you using the the --check-sum option for verification based upon file check sums of files rather than the modification times? - This email is protected by LBackup, an open source backup solution. http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: compare a directory with itself -- differences found
What kind of differences are reported. here's a small sample:[I said something like rsync ~ ~] cd+ /Users/ cd+ /Users/pen/ f+ /Users/pen/A f+ /Users/pen/B f+ /Users/pen/Desktop DB f+ /Users/pen/Desktop DF f+ /Users/pen/backup_excludes.txt f+ /Users/pen/correct f+ /Users/pen/doit f+ /Users/pen/pen.lisp f+ /Users/pen/rsync-new-executable f+ /Users/pen/rsync-tiger f+ /Users/pen/temp f+ /Users/pen/verify-backup f+ /Users/pen/zap-DOT f+ /Users/pen/zapit Also are you using the the --check-sum option for verification based upon file check sums of files rather than the modification times? Yes, what I'm doing is: ~/rsync-new-executable --dry-run --checksum --recursive --ignore-times --itemize-changes --xattrs -R --exclude-from ~/backup_excludes.txt $* I'm on OS X 10.4.11, but I'm using a current rsync, not the one that was included with OS X. I greatly appreciate the help! -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: compare a directory with itself -- differences found
I think an important question is what kind of differences do you need to check with regards integrity of your copy / backup? In addition, I suggest that you actually use rsync to copy the data to a separate directory and then compare two different directories. The reason I suggest this is because it is not clear that you are doing this from your rsync command. The following your rsync commands could be used achieve the results I believe you are trying to achieve? # Make a copy to the /tmp directory : ~/rsync-new-executable --recursive --ignore-times --itemize-changes --xattrs -R --exclude-from ~/backup_excludes.txt /Users/pen /tmp/pen # Check that the copy has completed successfully using checksums and dry run : ~/rsync-new-executable --dry-run --checksum --recursive --ignore-times --itemize-changes --xattrs -R --exclude-from ~/backup_excludes.txt Users/pen /tmp/pen Another possibility is to use a tool which is designed to compare copies / backups on Mac OS X. A great tool which I would highly recommend is backup-bouncer : http://www.n8gray.org/code/backup-bouncer/ If you are just after checksums then another possibility is something like fingerprint or the various other checksumming / tripwire tools (it really depends on the kind of attributes you wish to verify : http://www.lucidsystems.org/tools/fingerprint/ I hope this helps. - This email is protected by LBackup, an open source backup solution. http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: compare a directory with itself -- differences found
On Thu, 2010-09-02 at 16:22 -0700, Pen Helm wrote: What kind of differences are reported. here's a small sample:[I said something like rsync ~ ~] cd+ /Users/ cd+ /Users/pen/ f+ /Users/pen/A f+ /Users/pen/B f+ /Users/pen/Desktop DB f+ /Users/pen/Desktop DF f+ /Users/pen/backup_excludes.txt f+ /Users/pen/correct f+ /Users/pen/doit f+ /Users/pen/pen.lisp f+ /Users/pen/rsync-new-executable f+ /Users/pen/rsync-tiger f+ /Users/pen/temp f+ /Users/pen/verify-backup f+ /Users/pen/zap-DOT f+ /Users/pen/zapit [...] Rsync is showing that none of the source files have counterparts on the destination. That most likely indicates that your source and destination arguments do not match up correctly. ~/rsync-new-executable --dry-run --checksum --recursive --ignore-times --itemize-changes --xattrs -R --exclude-from ~/backup_excludes.txt $* Unfortunately, this doesn't show the actual source and destination arguments. But assuming they were /Users/pen /Users/pen, then rsync would copy /Users/pen to /Users/pen/Users/pen according to the -R option. You probably want to remove -R and use a trailing slash on the source path. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
