Re: rsync exit code 23 (partial transfer due to errors): List of possible reasons and how to ignore some?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Unfortunately, exit 23 litterally just means something else went wrong and might have scrolled off of the screen if you have rsync listing files (--verbose or --itemize_changes). Essentially, it is anything that doesn't have its own exit code. I just ignore it. The most common reaosn is file vanished. - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone:(407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. [email protected] (work) Orlando, [email protected] (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., On Thu, 14 Dec 2023, rsync--- via rsync wrote: Date: Thu, 14 Dec 2023 19:20:15 +0100 From: rsync--- via rsync To: [email protected] Subject: rsync exit code 23 (partial transfer due to errors): List of possible reasons and how to ignore some? I am trying to find a solution for the open source Linux software "Back In Time" (https://github.com/bit-team/backintime) where we evaluate the rsync exit code when taking a backup via rsync and inform the user that an error has occured. Questions: 1. Is there full list of possible reasons available that make rsync exit with the return value 23? 'rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]' I want to decide for each reason if we treat it as an error or warning. 2. I want to ignore some reasons for exit code 23 by only logging it, mainly: symlink has no referent: "/home/user/Documents/dead-link" Is there a way to - either prevent that this error leads to exit code 23 (if no other reason occurs) - or to prevent this specific check at all (but eg. copy the symlink "as is")? THX a lot! PS 1: The typcially used rsync command line looks like this: rsync --recursive --times --devices --specials --hard-links --human-readable -s --copy-links --acls --perms --executability --group --owner -- info=progress2 --no-inc-recursive -l --delete --delete-excluded -v -i --out-format=BACKINTIME: %i %n%L --link-dest=../../20231214-143359-584/backup -- chmod=Du+wx --exclude=/home/username/temp/testBAK_profil1 --exclude=/home/username/.local/share/backintime --exclude=.local/share/backintime/mnt -- include=/home/username/Documents/ --include=/home/username/ --include=/home/ --include=/home/username/temp/deleted_folder/ -- include=/home/username/temp/ --include=/home/username/.mozilla/ --exclude=.gvfs --exclude=.cache/* --exclude=.thumbnails* -- exclude=.local/share/[Tt]rash* --exclude=*.backup* --exclude=*~ --exclude=.dropbox* --exclude=/proc/* --exclude=/sys/* --exclude=/dev/* -- exclude=/run/* --exclude=/etc/mtab --exclude=/var/cache/apt/archives/*.deb --exclude=lost+found/* --exclude=/tmp/* --exclude=/var/tmp/* -- exclude=/var/backups/* --exclude=.Private --exclude=lock --exclude=root_only_file.txt --include=/home/username/Documents/** -- include=/home/username/temp/deleted_folder/** --include=/home/username/.mozilla/** --exclude=* / /home/username/temp/testBAK_profil1/backintime/computer1/username/1/new_snapshot/backup PS 2: You can find more details or even answer in our Github issue too: https://github.com/bit-team/backintime/issues/1587 -BEGIN PGP SIGNATURE- iF0EARECAB0WIQSHERqysePm7S8yuR9UoLWOVtABBwUCZXtTzQAKCRBUoLWOVtAB B2CnAJ9YGQ/gXTPP2Ntg3arHHDC11cRnfgCeJVpDOnGhTH0OreZfj8pIbnsL3SI= =Eqei -END PGP SIGNATURE- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync exit code 23 (partial transfer due to errors): List of possible reasons and how to ignore some?
On Thu, 2023-12-14 at 14:09 -0500, Kevin Korb wrote:
> Unfortunately, exit 23 litterally just means something else went wrong
> and might have scrolled off of the screen if you have rsync listing
> files (--verbose or --itemize_changes). Essentially, it is anything
> that doesn't have its own exit code. I just ignore it. The most common
> reaosn is file vanished.
THX for sharing your experiences and knowledge :-)
I have just tried to "reverse engineer" the possible reasons from the source
code
and have found 21 reasons that I hope will never happen ;-)
So ignoring (or treating as a warning only) sound as best option so far.
-
Looking into the rsync source code I can see only one location where exit code
23 is set:
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/errcode.h#L42
#define RERR_PARTIAL23 /* partial transfer */
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/log.c#L97
{ RERR_PARTIAL, "some files/attrs were not transferred (see previous
errors)" },
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/cleanup.c#L217-L218
if (io_error & IOERR_GENERAL || got_xfer_error)
exit_code = RERR_PARTIAL;
So the question is which reasons cause
- IOERR_GENERAL
- got_xfer_error
to be true?
IOERR_GENERAL is set for different reasons (first line is the log output format
string):
1. receive_sums failed [what is that at all?]:
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/sender.c#L345-L348
2. send_files failed to open %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/sender.c#L358-L362
3. fstat failed
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/sender.c#L373C33-L373C45
4. read errors mapping %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/sender.c#L433-L436
5. change_dir %s failed
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L366-L369
6. skipping overly long name: %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1245-L1247
7. symlink has no referent: %s
See the source code comments there when symlinks are checked:
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1271C1-L1282C28
8. readlink_stat(%s) failed
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1293-L1295
9. skipping file with bogus (zero) st_mode: %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1298-L1302
10. skipping symlink with 0-length value: %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1565-L1566
11. [%s] cannot convert filename: %s (%s)
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1595-L1601
12. [%s] cannot convert symlink data for: %s (%s)
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1609-L1614
13. get_acl(fname, &sx) < 0 // with no explicit error message!
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1628-L1632
14. get_xattr(fname, &sx) < 0 // with no explicit error message!
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1637-L1642
15. link_stat %s failed
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1809-L1811
16. opendir %s failed
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1841-L1843
17. filename overflows max-path len by %u: %s/%s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1863-L1871
18. cannot send file with empty name in %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1873-L1877
19. readdir(%s)
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L1886-L1889
20. link_stat %s failed
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/flist.c#L2396-L2399
21. cannot add local filter rules in long-named directory: %s
https://github.com/WayneD/rsync/blob/2f9b963abaa52e44891180fe6c0d1c2219f6686d/exclude.c#L815-L818
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
