Hi Alex,
Thank you very much indeed for that link. I was actually considering
using find -0 together with --files-from, and making find to list only
files that were changed some time ago -- but first, this would probably
break hardlinks, and second, despite most of the files in question are
temporary files, a user can still delete something old enough right in
the middle of an rsync.
However, looking at rsync problems that we have more closely, I now
realize that most of them are related to time setting issues -- part of
these similar to bug https://bugzilla.samba.org/show_bug.cgi?id=4977 and
the like. But I think these problems can and shall be addressed )
I will write more on this when I will finish testing.
Kind regards, George
On 08/04/17 00:01, Axel Kittenberger wrote:
With this two options on a very live system you may need to take into
account this bug as well I reported a while ago:
https://bugzilla.samba.org/show_bug.cgi?id=12569
<https://protect-au.mimecast.com/s/5vYEBwtZLRbJcm?domain=bugzilla.samba.org>
Due to this I'm currently not using --ignore-missing-args /
--delete-missing but rather -exclude=* -include=- and a autocreated
hierachical filter rule list on stdin.
PS: Yes to simply ignore code 24 would be correct. However, it creates
exit code 2 in these cases.
On Fri, Apr 7, 2017 at 2:58 PM, Georgy Fedorov via rsync
<rsync@lists.samba.org <mailto:rsync@lists.samba.org>> wrote:
Dear All,
We sometimes have to replicate large "live" filesystems with many
( sometimes millions, up to few hundred millions ) files on them.
( Copying actively used files is of course a bad idea, but it
really helps to keep the delta small, so one final transfer can
later save the day. )
The problem, as one may guess, is that some files may disappear
during the process, so rsync finishes with an error code 23.
The good news is that since version 3.1.0 rsync has two options to
address this problem: --ignore-missing-args and
--delete-missing-args .
The bad news, however, is that even with any of these, when the
file disappears, rsync handles the file transfer properly, but
then tries to set times or attributes on these files nevertheless.
That fails with errno 22 ( EINVAL ) and still leads to exit code
23, which is a bit of an annoyance.
I am currently trying to come up with a patch to address this
issue, since I'd like to have exit code 0 for either of
--ignore-missing-args or --delete-missing-args, when the files
are, well, missing.
Probably it is not the right way to address this problem, but in
the same vein as --ignore-missing-args are implemented, the patch
can go as follows:
https://gist.github.com/anonymous/96ba8bf10f864a93fd9203f75c43ffe9
<https://gist.github.com/anonymous/96ba8bf10f864a93fd9203f75c43ffe9> .
Basically we need to do two changes:
(1) in options.c, make sure that the "missing_args" value is
transferred when set ; and
(2) in rsync.c, in the "missing_args" case, replace FERROR_XFER
with something else, since apparently any log message with logcode
FERROR_XFER sets the flag 'got_xfer_error' in log.c, and that
finally leads to exit code RERR_PARTIAL (23), which is what we are
trying to avoid.
I am currently testing this on fairly big datasets to see if
there's something missing, and will write more when I see how it
goes ( as I said, the data sets are fairly large ).
There shall be a better way to address that, but this is all I can
do with a very shallow acquaintance with rsync source code.
Nevertheless, it would be great if that could be fixed in the
trunk one way or another )
Kind regards, George
--
George Fedorov
Senior Systems Specialist
Melbourne School of Engineering
The University of Melbourne, Victoria 3010, Australia
phone: *****
email: *****
http://www.eng.unimelb.edu.au/
--
Please use reply-all for most replies to avoid omitting the
mailing list.
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
<https://protect-au.mimecast.com/s/gn98BxSeXmW9fJ?domain=lists.samba.org>
Before posting, read:
http://www.catb.org/~esr/faqs/smart-questions.html
<https://protect-au.mimecast.com/s/oDLrBgcVgWZDt8?domain=catb.org>
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
