Re: Minimize log file: get rid of .d...p..... send
On Fri, 2010-09-03 at 17:05 +0200, Hans Troost wrote: and indeed - although I specified rsync -vvv (most verbose output) - rsync silently ignores chmod(2) calls with emphasis on silently: 2010/09/03 16:45:49 [7882] send_files(15, /home/hans/rsync-files/rsync-test/New Folder) 2010/09/03 16:45:49 [7882] cd+ send home/hans/rsync-files/rsync-test/New Folder 2010/09/03 16:45:49 [7882] set modtime of rsync-test/New Folder to (1283525117) Fri Sep 3 16:45:17 2010 2010/09/03 16:45:49 [7882] delete_in_dir(rsync-test/New Folder) Rsync is calling chmod, there just isn't a log message for it even on high verbosity. Run rsync under strace if you want to see all the system calls it is making. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
Thanks Matt, Tested again a bit and indeed: permissions on source and destination are different: [h...@localhost rsync-test]$ cd ~/rsync-files/rsync-test/ [h...@localhost rsync-test]$ ls -l total 24 drwxrwxr-x 2 hans hans 4096 2010-09-02 18:24 2bchanged/ drwxrwxr-x 2 hans hans 4096 2010-09-03 16:45 New Folder/ drwxrwxr-x 2 hans hans 4096 2010-09-03 16:42 New Folder (2)/ [h...@localhost rsync-test]$ cd /mnt/backup//rsync-test/ [h...@localhost rsync-test]$ ls -l total 8 drwxr-xr-x 1 hans hans 0 2010-09-02 18:24 2bchanged/ drwxr-xr-x 1 hans hans 0 2010-09-03 16:45 New Folder/ drwxr-xr-x 1 hans hans 0 2010-09-03 16:42 New Folder (2)/ and indeed - although I specified rsync -vvv (most verbose output) - rsync silently ignores chmod(2) calls with emphasis on silently: 2010/09/03 16:45:49 [7882] send_files(15, /home/hans/rsync-files/rsync-test/New Folder) 2010/09/03 16:45:49 [7882] cd+ send home/hans/rsync-files/rsync-test/New Folder 2010/09/03 16:45:49 [7882] set modtime of rsync-test/New Folder to (1283525117) Fri Sep 3 16:45:17 2010 2010/09/03 16:45:49 [7882] delete_in_dir(rsync-test/New Folder) Since it is just a backup of my own personal workstation I decided to to use rsync -a --no-p : when restoring I just have to re-apply the correct permissions (if required at all). I decided to also remove the --log-file-format at all. This gives the log-file I prefer. Thanks for your comments and recommendations, Kind regards, Hans Troost -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
Matt,
Took a while to get out that I didn't have strace, installed and learned it
a bit.
!!! Even with strace I do not get a chmod call shown. !!!
I do not worry, since my problem is solved (thanks), but still am curious.
Please don't spend time if this is not interesting for you and others as
well. This is just to inform you!
Created a new folder new-rsync-with-strace ran my script again:
#!/bin/bash
RawDate=`date +%Y%m%d`
NiceDate=`date +%Y-%m-%d`
Now=`date +%H%M`
strace -o /home/hans/strace.log -s 80 rsync -axhhX --no-p --stats --delete
--delete-excluded --backup --backup-dir-dels=$NiceDate/deleted --suffix-
dels=.$Now --backup-dir=$NiceDate/updated --suffix=.$Now --log-
file=/home/hans/rsync-files/rsync-log.$RawDate-$Now --exclude-
from=/home/hans/rsync-files/exclude.txt /home/hans/rsync-files/rsync-test
/mnt/backup
folder listing (source):
drwxrwxr-x 2 hans hans 4096 2010-09-03 18:06 new-rsync-with-strace
(destination)
drwxrwxr-x 2 hans hans 4096 2010-09-03 18:06 new-rsync-with-strace
and got this output (search for : new-rsync-with-strace):
execve(/usr/bin/rsync, [rsync, -axhhX, --no-p, --stats, --
delete, --delete-excluded, --backup, --backup-dir-
dels=2010-09-03/deleted, --suffix-dels=.1806, --backup-
dir=2010-09-03/updated, --suffix=.1806, --log-file=/home/hans/rsync-
files/rsync-log.20100903-1806, --exclude-from=/home/hans/rsync-
files/exclude.txt, /home/hans/rsync-files/rsync-test, /mnt/backup], [/*
76 vars */]) = 0
brk(0) = 0x9a62000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb77fb000
access(/etc/ld.so.preload, R_OK) = -1 ENOENT (No such file or
directory)
open(/etc/ld.so.cache, O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=122494, ...}) = 0
mmap2(NULL, 122494, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb77dd000
close(3)= 0
open(/lib/libacl.so.1, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\24\0\0004\0\0\0\0l\0\0\0\0\0\0004\0
\0\5\0(\0\32\0\31\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\310h\0\0\310..., 512)
= 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=28688, ...}) = 0
mmap2(NULL, 31556, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb77d5000
mmap2(0xb77dc000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x6) = 0xb77dc000
close(3)= 0
open(/usr/lib/libpopt.so.0, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\224l\0\0\0\0\0\0004\0
\0\5\0(\0\31\0\30\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\314h\0\0\314..., 512)
= 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=28796, ...}) = 0
mmap2(NULL, 31724, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb77cd000
mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x6) = 0xb77d4000
close(3)= 0
open(/lib/i686/libc.so.6, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220k\1\0004\0\0\0\324\326\25\0\0\0\0\0004\0
\0\n\0(\0d\0c\0\6\0\0\0004\0\0\0004\0\0\0004\0\...@\1\0\0@..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=1433972, ...}) = 0
mmap2(NULL, 1444104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
= 0xb766c000
mprotect(0xb77c6000, 4096, PROT_NONE) = 0
mmap2(0xb77c7000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x15a) = 0xb77c7000
mmap2(0xb77ca000, 10504, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_ANONYMOUS, -1, 0) = 0xb77ca000
close(3)= 0
open(/lib/libattr.so.1, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20\16\0\0004\0\0\0\250=\0\0\0\0\0\0004\0
\0\5\0(\0\32\0\31\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20;\0\0\20..., 512) =
512
fstat64(3, {st_mode=S_IFREG|0644, st_size=16824, ...}) = 0
mmap2(NULL, 19676, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7667000
mmap2(0xb766b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x3) = 0xb766b000
close(3)= 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7666000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7665000
set_thread_area({entry_number:-1 - 6, base_addr:0xb76656c0, limit:1048575,
seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1,
seg_not_present:0, useable:1}) = 0
mprotect(0xb77c7000, 8192, PROT_READ) = 0
mprotect(0xb7818000, 4096, PROT_READ) = 0
munmap(0xb77dd000, 122494) = 0
rt_sigaction(SIGUSR1, {0x8063790, [], SA_NOCLDSTOP}, NULL, 8) = 0
rt_sigaction(SIGUSR2, {0x8064100, [], SA_NOCLDSTOP}, NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8064150, [], SA_NOCLDSTOP}, NULL, 8) = 0
time(NULL) = 1283530015
geteuid32() = 500
umask(0)= 02
brk(0) = 0x9a62000
Re: Minimize log file: get rid of .d...p..... send
On Fri 03 Sep 2010, Hans Troost wrote: !!! Even with strace I do not get a chmod call shown. !!! strace -o /home/hans/strace.log -s 80 rsync -axhhX --no-p --stats --delete If you're doing --no-p you wonder why it's not setting permissions? Paul -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
OK, thanks Paul: stupid me: forgot to remove --No-p.
Now I did, but still
Now made it:
#!/bin/bash
RawDate=`date +%Y%m%d`
NiceDate=`date +%Y-%m-%d`
Now=`date +%H%M`
#echo RawDate: $RawDate
#echo NiceDate: $NiceDate
#echo Now: $Now
strace -o /home/hans/strace.log -s 78 rsync -axhhX --perms --stats --delete
--delete-excluded --backup --backup-dir-dels=$NiceDate/deleted --suffix-
dels=.$Now --backup-dir=$NiceDate/updated --suffix=.$Now --log-
file=/home/hans/rsync-files/rsync-log.$RawDate-$Now --exclude-
from=/home/hans/rsync-files/exclude.txt /home/hans/rsync-files/rsync-test
/mnt/backup
created a new directory (test03).
Source:
drwxrwxr-x 2 hans hans 4096 2010-09-03 18:49 test03
Dest:
drwxr-xr-x 1 hans hans 0 2010-09-03 18:49 test03
strace.log (still no chmod found!!)
execve(/usr/bin/rsync, [rsync, -axhhX, --perms, --stats, --
delete, --delete-excluded, --backup, --backup-dir-
dels=2010-09-03/deleted, --suffix-dels=.1850, --backup-
dir=2010-09-03/updated, --suffix=.1850, --log-file=/home/hans/rsync-
files/rsync-log.20100903-1850, --exclude-from=/home/hans/rsync-
files/exclude.txt, /home/hans/rsync-files/rsync-test, /mnt/backup], [/*
76 vars */]) = 0
brk(0) = 0x9b35000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb78c6000
access(/etc/ld.so.preload, R_OK) = -1 ENOENT (No such file or
directory)
open(/etc/ld.so.cache, O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=122494, ...}) = 0
mmap2(NULL, 122494, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb78a8000
close(3)= 0
open(/lib/libacl.so.1, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\24\0\0004\0\0\0\0l\0\0\0\0\0\0004\0
\0\5\0(\0\32\0\31\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\310h\0..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=28688, ...}) = 0
mmap2(NULL, 31556, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb78a
mmap2(0xb78a7000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x6) = 0xb78a7000
close(3)= 0
open(/usr/lib/libpopt.so.0, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\224l\0\0\0\0\0\0004\0
\0\5\0(\0\31\0\30\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\314h\0\0..., 512) =
512
fstat64(3, {st_mode=S_IFREG|0755, st_size=28796, ...}) = 0
mmap2(NULL, 31724, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7898000
mmap2(0xb789f000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x6) = 0xb789f000
close(3)= 0
open(/lib/i686/libc.so.6, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220k\1\0004\0\0\0\324\326\25\0\0\0\0\0004\0
\0\n\0(\0d\0c\0\6\0\0\0004\0\0\0004\0\0\0004\0\...@\1\0..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=1433972, ...}) = 0
mmap2(NULL, 1444104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
= 0xb7737000
mprotect(0xb7891000, 4096, PROT_NONE) = 0
mmap2(0xb7892000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x15a) = 0xb7892000
mmap2(0xb7895000, 10504, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_ANONYMOUS, -1, 0) = 0xb7895000
close(3)= 0
open(/lib/libattr.so.1, O_RDONLY) = 3
read(3,
\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\20\16\0\0004\0\0\0\250=\0\0\0\0\0\0004\0
\0\5\0(\0\32\0\31\0\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\20;\0..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=16824, ...}) = 0
mmap2(NULL, 19676, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) =
0xb7732000
mmap2(0xb7736000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|
MAP_DENYWRITE, 3, 0x3) = 0xb7736000
close(3)= 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb7731000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0xb773
set_thread_area({entry_number:-1 - 6, base_addr:0xb77306c0, limit:1048575,
seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1,
seg_not_present:0, useable:1}) = 0
mprotect(0xb7892000, 8192, PROT_READ) = 0
mprotect(0xb78e3000, 4096, PROT_READ) = 0
munmap(0xb78a8000, 122494) = 0
rt_sigaction(SIGUSR1, {0x8063790, [], SA_NOCLDSTOP}, NULL, 8) = 0
rt_sigaction(SIGUSR2, {0x8064100, [], SA_NOCLDSTOP}, NULL, 8) = 0
rt_sigaction(SIGCHLD, {0x8064150, [], SA_NOCLDSTOP}, NULL, 8) = 0
time(NULL) = 1283532600
geteuid32() = 500
umask(0)= 02
brk(0) = 0x9b35000
brk(0x9b56000) = 0x9b56000
open(/usr/share/locale/locale-archive, O_RDONLY|O_LARGEFILE) = -1 ENOENT
(No such file or directory)
open(/usr/share/locale/locale.alias, O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=2570, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE,
Re: Minimize log file: get rid of .d...p..... send
On Fri, 2010-09-03 at 18:18 +0200, Hans Troost wrote: Took a while to get out that I didn't have strace, installed and learned it a bit. !!! Even with strace I do not get a chmod call shown. !!! strace -o /home/hans/strace.log -s 80 rsync [...] That is just tracing the first rsync process, which is the sender. The chmod calls would be in the generator or the receiver, which are forked by the sender for a local run. Use the -f option to strace to trace all three processes. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
On Fri, 2010-09-03 at 20:49 +0200, Hans Troost wrote: Indeed, strace -f option helps: the chmod status unfinished shows up several times now. no real warnings or errors, but nevertheless: information about what's going wrong. Unfinished does not indicate that anything is wrong. It just means that something happened in another process between when the call was issued and when it returned, so the output had to be broken up. The problem is that the chmod calls return 0 (success) without actually setting the permissions. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
Thanks Matt, That clarifies it for me. Completely happy with all your support: learned a lot, now made up my mind about my backups and know how to achieve this. Regards, Hans Troost -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Minimize log file: get rid of .d...p..... send
On Thu, 2010-09-02 at 11:38 +0200, hans.tro...@telfort.nl wrote: I'm just starting with rsync, googled and experienced a lot, but get stuck with minimizing the log-file: I only want to see the real changes: new, updated and deleted files/directories in the log-file. The LOG-file always contains a lot of lines like: 2010/08/31 20:19:01 [24573] .d...p. send home/hans/Desktop 2010/08/31 20:19:05 [24573] .d...p. send home/hans/Documents/PCPa I did not change anything, also not the permissions. So these directories are unchanged! Yes, but are they the same on source and destination? The behavior you observe could happen if the destination filesystem is unable to store permissions and silently ignores chmod(2) calls. In that case, you should disable the -p option. Question: how to get rid of these lines, while keeping lines like: 2010/08/31 20:19:01 [24573] f.stp. send home/hans/2bedited.txt 2010/08/31 20:19:01 [24573] f+ send home/hans/Documents/newfile.txt To get rid of lines with only attribute changes, you can set a log format that does not contain %i. But I would suggest first looking into why they are being printed. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
