DO NOT REPLY [Bug 7618] symlinks and --link-dest
https://bugzilla.samba.org/show_bug.cgi?id=7618 --- Comment #17 from the_ma...@seznam.cz 2010-08-15 03:58 CST --- (In reply to comment #16) I just noticed the no symtimes (i.e., HAVE_LUTIMES off) in the rsync --version output for the client in comment #0. Consistent with that, rsync does not attempt to set the symlink's mtime in the first log in comment #12. So the problem must be elsewhere. Next theory: It looks like symlinks on HP-UX can have permissions other than 777, according to a discussion (http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1281845702588+28353475threadId=956122) I found by searching Google for hp-ux symlink permission. And it looks like rsync (in unchanged_attrs) will consider a difference in permissions of a symlink to disqualify it for hard-linking, even if rsync is unable to set symlink permissions (HAVE_LCHMOD off). What are the permissions of home/zumrova/.tsm on the HP-UX source? hpuh:/home/zumrova # ls -l .tsm lrwxr-xr-x 1 root sys 13 Mar 23 2006 .tsm - /usr/tsm/.tsm (and for info hpuh:/home/zumrova # ls -l /usr/tsm/.tsm -r--r--r-- 1 hajek sys 4253 Mar 6 2001 /usr/tsm/.tsm ) But then I would expect rsync to consider the symlink changed every time, not only when running with --link-dest. Mike -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 7618] symlinks and --link-dest
https://bugzilla.samba.org/show_bug.cgi?id=7618 --- Comment #18 from the_ma...@seznam.cz 2010-08-15 04:05 CST --- And on hp-ux chmod blahblah /home/zumrova/.tsm changes permissions on the real file, the symlink permissions stays unchanged. Mike -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Performance and simultaneous connections over SSH
Hi all, I have about 15 LDAP servers that will be using Rsync over SSH to sync up their access logs to a centralized server for further processing. The estimated log volume is around 12 Gigs / days total. On the centralized server, the directory where the logs are being synced is an enterprise-grade NAS. So it looks like this: LDAP servers -- rsync over SSH -- NFS mount on log repo server -- NAS device I have 4 servers currently configured and everything appeared to work fine at first. I do not have any visibility to the LDAP servers but I assume their clocks are synced. The cron is running at the same times: HH:00, HH:10, 20, 30, 40 and 50. I just started to notice some 'sync sputtering': Sometime, all 4 server's latest access log will have the same timestamp e.g. 09:30, while at other times I would see something like this: It is 09:35 and I have 2 servers at 09:20 and 2 at 09:30 i.e. they could not all sync themselves during the last round. How does rsync handle mutliple simultaneous connections over SSH. I am guessing that it is up to the log repo server to allocate 4 separate SSH sessions, and that within each of these, it will use rsync to sync up the logs. Is that correct? If so, then the only issue would be the log repo server requesting a lot of information from the NAS device at the same time, hence causing the operation to possibly fail. The actual rsync sommand looks like this: /bin/rsync --timeout=300 --rsync-path=/usr/local/bin/rsync -avz -e /usr/local/bin/ssh -i key `find $LDAP_LOGS_LOCAL/access* -mtime -5 -type f` flmu...@$preparser_server:$LDAP_LOGS_REMOTE Timeout is set to 300 and logs being checked and synced are at most 100 megs in size (they rotate once they hit 100 megs). Cheers, François -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 7618] Symlink permissions disqualify --link-dest even when they can't be set
https://bugzilla.samba.org/show_bug.cgi?id=7618 m...@mattmccutchen.net changed: What|Removed |Added Summary|symlinks and --link-dest|Symlink permissions ||disqualify --link-dest even ||when they can't be set --- Comment #19 from m...@mattmccutchen.net 2010-08-15 12:37 CST --- (In reply to comment #17) hpuh:/home/zumrova # ls -l .tsm lrwxr-xr-x 1 root sys 13 Mar 23 2006 .tsm - /usr/tsm/.tsm So that's it. Unfortunately, the only workarounds I can offer are to exclude the symlink or recreate it on the source with 777 permissions by setting your umask to 0. But then I would expect rsync to consider the symlink changed every time, not only when running with --link-dest. I should have been clearer. Treating differences in symlink permissions as significant even on systems where they can't be set is not the expected behavior, it is a bug specific to --link-dest. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 7622] New: Factor out common logic from itemize and unchanged_attrs functions
https://bugzilla.samba.org/show_bug.cgi?id=7622 Summary: Factor out common logic from itemize and unchanged_attrs functions Product: rsync Version: 3.1.0 Platform: All OS/Version: All Status: NEW Severity: enhancement Priority: P3 Component: core AssignedTo: way...@samba.org ReportedBy: m...@mattmccutchen.net QAContact: rsync...@samba.org There is a lot of logic duplicated between these two functions. Factoring it out would make it easier to add support for preserving new attributes and would cut down on the number of bugs in one function but not the other, such as: https://lists.samba.org/archive/rsync/2008-March/020276.html bug 7618 -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 7618] Symlink permissions disqualify --link-dest even when they can't be set
https://bugzilla.samba.org/show_bug.cgi?id=7618 --- Comment #20 from the_ma...@seznam.cz 2010-08-15 13:42 CST --- (In reply to comment #19) (In reply to comment #17) hpuh:/home/zumrova # ls -l .tsm lrwxr-xr-x 1 root sys 13 Mar 23 2006 .tsm - /usr/tsm/.tsm So that's it. Unfortunately, the only workarounds I can offer are to exclude the symlink or recreate it on the source with 777 permissions by setting your umask to 0. But then I would expect rsync to consider the symlink changed every time, not only when running with --link-dest. I should have been clearer. Treating differences in symlink permissions as significant even on systems where they can't be set is not the expected behavior, it is a bug specific to --link-dest. That's my opinion too :-) Is it a known bug? Will it be fixed in new version? Very thanks, Mike -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
DO NOT REPLY [Bug 7618] Symlink permissions disqualify --link-dest even when they can't be set
https://bugzilla.samba.org/show_bug.cgi?id=7618 --- Comment #21 from m...@mattmccutchen.net 2010-08-15 14:14 CST --- (In reply to comment #20) Is it a known bug? It is a known bug now, by virtue of you having reported it and me having diagnosed it. Will it be fixed in new version? Probably, but I cannot make any guarantees. -- Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email --- You are receiving this mail because: --- You are the QA contact for the bug, or are watching the QA contact. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Performance and simultaneous connections over SSH
On Sun, 2010-08-15 at 10:50 -0600, Francois Begin wrote: I just started to notice some 'sync sputtering': Sometime, all 4 server's latest access log will have the same timestamp e.g. 09:30, while at other times I would see something like this: It is 09:35 and I have 2 servers at 09:20 and 2 at 09:30 i.e. they could not all sync themselves during the last round. I would suggest enabling logging with --log-file on either the client or the server side to get more information about what is happening. How does rsync handle mutliple simultaneous connections over SSH. I am guessing that it is up to the log repo server to allocate 4 separate SSH sessions, and that within each of these, it will use rsync to sync up the logs. Is that correct? Correct. The four rsync instances on the log repo server will work independently. -- Matt -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Scripting rsync against a filesystem with a quota
On another note I'm getting an error: copyfile(.._..v6AxMh,./., COPYFILE_UNPACK) failed:13 When trying to run with -E. Upstream rsync has no such error message. If you're using the Apple-modified rsync, we don't support it here. You may wish to compile a version of rsync from source. Below is a link with instructions on compiling rsync (and directions on applying some Mac OS X patches). http://www.lbackup.org/developer/rsync_hfs To keep things simple you may wish to skip over the steps associated with the patches. To do this just skip the two commands quoted below : patch -p1 patches/fileflags.diff patch -p1 patches/crtimes.diff Hope this helps. -- This message is protected by LBackup http://www.lbackup.org -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
