[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #41 from Mihnea-Costin Grigore --- The discussion about file systems like ZFS/BTRFS/etc. and their various snapshot mechanisms is off-topic relative to this feature request, since they are very different technologies used for different purposes. rsync is used commonly to synchronise at the *file level* between very different operating systems -- from Linux to Windows, from macOS to Linux, etc. It also has multiple features to allow filtering and selecting files within the source and destination directories, thus only synchronising a subset of files. We need a solution that works with the flexibility of rsync itself, and snapshots (useful as they are) do not fit that at all. This would be a very useful feature to have as part of rsync, made evident by the many requests both here and on other discussion forums over the past almost 20 (!) years. Can we rather discuss what the blockers are for the existing patches? What stops them from inclusion -- is it quality, functionality, compatibility, test coverage, something else? Working to fix that and making the already existing patches acceptable for inclusion would appear to be the most constructive course of action, rather than deflecting the request. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 6741] 'deleting' messages show up in improper places
https://bugzilla.samba.org/show_bug.cgi?id=6741 --- Comment #5 from Marc Aurèle La France --- Created attachment 18263 --> https://bugzilla.samba.org/attachment.cgi?id=18263=edit rsync stdout filter Just something I've come up with to work around this issue. Not perfect but does the job. See usage information in the comments. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15585] New: rsync ends still with error 22 when try to deleting many files
https://bugzilla.samba.org/show_bug.cgi?id=15585 Bug ID: 15585 Summary: rsync ends still with error 22 when try to deleting many files Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: f...@fkn-systems.de QA Contact: rsync...@samba.org Target Milestone: --- Problem: When syncing Souce/Target and must deleting many files, rsync stop still quiet with Error-Code: "22 - Error allocating core memory buffers" Desc: When more than ~5-10 million files to delete, rsync ends still quiet, in case of deleting process. Rsync reads (with --delete-before) the complete Source and breaks than after deleting many (>1000) files on Target. Else, (withOUT --delete-before), he does some sync on Target and some deletes, some sync ... and breaks than after delete some files. In case of this, Backups are not synced, and may space overflow after next backup run. Use of ‐‐max‐delete=100.000.000 are not functional to solve this Command: rsync -vaxHSPAX --delete [--delete-before] $SOURCE $TARGET # $SOURCE, $TARGET are real directorys or remote (ramote:dir...) Versions: It tested fail with: * rsync version 3.2.3 protocol version 31 * rsync version 3.2.7 protocol version 31 -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #39 from andy --- > This feature request is so old it has lost relavence because btrfs/zfs/etc > are more optimal backup solutions than rsync. Funny I am doing exactly this, but I came to rsync looking for a backup for when ZFS fails. Many consider zfs/btrfs/snapshots as "not a backup". There are many things that can go wrong that you will need real backups to save you: - accidentally deleted pool/datasets/snapshots - bug in replication tool - user error - ZFS bug It should be considered a certainty that one of those will happen at some point, and the ZFS snapshots won't save you. > With zfs one can even have encrypted backups without the backup server ever > seeing the key or un-encrypted data. I love this idea, but in practice I'm finding there is significant risk with the state of ZFS encryption. There are so many active bugs related to encryption. I'm in the middle of implementing a replication system based on raw encrypted snapshot replication between multiple systems, trusted and untrusted. But the new bugs I've run into along the way, along with the previously known ones, makes me really feel the need for a solid non-ZFS filesystem backup. And also a low complexity tool, not dependent on complicated replication tools/scripts. In looking for rename/move solutions with rsync, one issue I can foresee with inode tracking is that I find it is very common to cross filesystem boundaries. Anything tracking inodes would need to track the device as well, though the device number from the stat struct doesn't seem to be enough in the case of ZFS to trace back to what filesystem it actually comes from. Reading the unison documentation, it seems that for linux they track the combo of inode & last modification time to detect moves/renames. I wonder if some kind of collision is possible under a rare multi-filesystem edge case. Inodes aren't unique across multiple filesystems. Restic is another option that handles moves/renames/dedup automatically, just at the cost of CPU time for encrypting/hashing. Probably worth considering borg and friends at that point. Well, maybe the cost of rsync's inefficiency here is worth it's simplicity. But it would be a great feature to have. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15546] New: disable of sorting when files to transfer is fed via --files-from
https://bugzilla.samba.org/show_bug.cgi?id=15546 Bug ID: 15546 Summary: disable of sorting when files to transfer is fed via --files-from Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: devz...@web.de QA Contact: rsync...@samba.org Target Milestone: --- i need to transfer files in correct order so they are serialized on disk in a special way to optimize access time. apparently, files fed via --files-from automatically getting ordered. it would be nice if this could be discabled. i could help myself with tar trough a pipe, but i would use my favourite and most trusted file transfer tool to handle such a job, too. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.
https://bugzilla.samba.org/show_bug.cgi?id=12806 --- Comment #7 from Aryo Da --- I think this a severe bug for all backup use cases of rsync that take a full snapshot with permissions (--perms) by creating hardlinks to unchanged files + copies of changed files (--link-dest): -> Whenever an old snapshot is deleted with `rsync --delete` the permissions of the hardlinked files change and trigger another copy of the files in a new snapshot (= waste of storage space and backup run-time). -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12806] Deleting in a row of hardlinked snapshots resets file permissions.
https://bugzilla.samba.org/show_bug.cgi?id=12806 --- Comment #6 from Aryo Da --- Created attachment 18117 --> https://bugzilla.samba.org/attachment.cgi?id=18117=edit MRE (minimal reproducible example) as bash script to reproduce the bug Rename to "setup.sh" and make it executable... -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 7809] I/O errors other than IOERR_GENERAL should not suppress deletion
https://bugzilla.samba.org/show_bug.cgi?id=7809 --- Comment #6 from Jeffrey Simon --- 3. The script does not work from launchd running as root. I should have given the failure mode, which is the following: rsync: opendir "/Volumes/Backup1/." failed: Operation not permitted (1) rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996) [sender=2.6.9] -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 7809] I/O errors other than IOERR_GENERAL should not suppress deletion
https://bugzilla.samba.org/show_bug.cgi?id=7809 --- Comment #5 from Jeffrey Simon --- "Excludes are relative to the source dir". Are you saying that the excludes should be --exclude=.DocumentRevisions-V100 --exclude=.TemporaryItems --exclude=.Trashes? That is a rhetorical question, because I no longer get an issue with with excludes. After I added sudo, those errors no longer occur. However, I am still having a problem, as follows: 1. The plain rsync command (with sudo and excludes) works from the command line. 2. The same rsync command (with sudo and excludes) works as part of a script from the command line. 3. The script does not work from launchd running as root. Because rsync appears to be working properly, my issues are most likely with the environmental differences between command line and launchd. So except for the oddities that these issues only showed up on macOS Ventura, but did not happen on macOS Monterey, I believe the issue can be considered somewhat resolved. I say "somewhat" because it is not clear to me why sudo should be necessary from the command line, but it is. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 7809] I/O errors other than IOERR_GENERAL should not suppress deletion
https://bugzilla.samba.org/show_bug.cgi?id=7809 --- Comment #4 from Kevin Korb --- Your excludes aren't working because excludes are relative to the source dir not /. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 7809] I/O errors other than IOERR_GENERAL should not suppress deletion
https://bugzilla.samba.org/show_bug.cgi?id=7809 --- Comment #3 from Jeffrey Simon --- One other point to follow up my first post of 2023-07-21: None of these issues occurred on macOS Monterey 12.4.x -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 7809] I/O errors other than IOERR_GENERAL should not suppress deletion
https://bugzilla.samba.org/show_bug.cgi?id=7809 --- Comment #2 from Jeffrey Simon --- This is an old ticket, but I am getting the same or similar problem in 2023 with rsync on macOS Ventura 13.4.1. Here is the first attempt and partial results: rsync -av --delete /Volumes/Backup1/ /Volumes/Backup2/ building file list ... rsync: opendir "/Volumes/Backup1/.DocumentRevisions-V100" failed: Permission denied (13) rsync: opendir "/Volumes/Backup1/.TemporaryItems" failed: Permission denied (13) rsync: opendir "/Volumes/Backup1/.Trashes" failed: Permission denied (13) done IO error encountered -- skipping file deletion ... With the following command that adds the exclusions the same results occur: rsync -av --delete --exclude=/Volumes/Backup1/.DocumentRevisions-V100 --exclude=/Volumes/Backup1/.TemporaryItems --exclude=/Volumes/Backup1/.Trashes /Volumes/Backup1/ /Volumes/Backup2/ building file list ... rsync: opendir "/Volumes/Backup1/.DocumentRevisions-V100" failed: Permission denied (13) rsync: opendir "/Volumes/Backup1/.TemporaryItems" failed: Permission denied (13) rsync: opendir "/Volumes/Backup1/.Trashes" failed: Permission denied (13) done IO error encountered -- skipping file deletion VM Archive/.DS_Store sent 1570115 bytes received 42 bytes 1046771.33 bytes/sec total size is 568948981975 speedup is 362351.65 rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/c2cb9645-dafc-11ed-aa26-6ec1e3b3f7b3/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996) [sender=2.6.9] The only resolution to that issue is to use add the use of sudo: sudo rsync -av --delete --exclude=/Volumes/Backup1/.DocumentRevisions-V100 --exclude=/Volumes/Backup1/.TemporaryItems --exclude=/Volumes/Backup1/.Trashes /Volumes/Backup1/ /Volumes/Backup2/ building file list ... done .DocumentRevisions-V100/ .DocumentRevisions-V100/.cs/ .DocumentRevisions-V100/AllUIDs/ .DocumentRevisions-V100/AllUIDs/1/ .DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/ .DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/3C2D59C9-44A2-471D-866B-282495F0718D.txt .DocumentRevisions-V100/AllUIDs/1/com.apple.documentVersions/9876415A-EA39-4A70-843F-7F9C8FFE75BF.txt .DocumentRevisions-V100/db-V1/ .DocumentRevisions-V100/db-V1/db.sqlite-wal .DocumentRevisions-V100/purgatory/ .DocumentRevisions-V100/staging/ .DocumentRevisions-V100/staging/501-60852-3Yny2b0q/ .TemporaryItems/ .TemporaryItems/folders.0/ .TemporaryItems/folders.0/TemporaryItems/ .TemporaryItems/folders.501/ .TemporaryItems/folders.501/TemporaryItems/ 1P-Vault/ Download Archive/Windows/ZTree/ RH-archive/20230507/ VM Archive/W10-upgr-from-W7.pvm/ VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/ VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/ VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/MacOS/ VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/Resources/ VM Archive/W10-upgr-from-W7.pvm/W10-upgr-from-W7.app/Contents/Resources/English.lproj/ VM Archive/W10-upgr-from-W7.pvm/Windows 7-0.hdd/ VM Archive/W10-upgr-from-W7.pvm/Windows Disks/ sent 183 bytes received 740 bytes 666969.20 bytes/sec total size is 568949268052 speedup is 341214.72 The following are my conclusions: 1. The bug in this ticket still occurs in my environment. 2. Using the --exclude parameter would seem to be a workaround, but it is not. 3. Adding sudo along with the --exclude parameters and the command works as expected from the command line. However, the intended use case is to have this command run from launchd. For some reason running the launchd command as root does not solve this, and even adding sudo as part of the command does not solve it. This final aspect might have nothing to do with rsync, but could rather be in my configuration or usage. I am including this information just to be complete. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15393] New: rsync attempts to set extended attributes while in dry-run
https://bugzilla.samba.org/show_bug.cgi?id=15393
Bug ID: 15393
Summary: rsync attempts to set extended attributes while in
dry-run
Product: rsync
Version: 3.2.0
Hardware: All
OS: All
Status: NEW
Severity: normal
Priority: P5
Component: core
Assignee: wa...@opencoder.net
Reporter: miguelangel.pros...@gmail.com
QA Contact: rsync...@samba.org
Target Milestone: ---
Using the following system:
Debian 11 with rsync 3.2.3
Fedora 38 with rsync 3.2.7
Creating the following environment:
mkdir -p src/x cmp/x dst/
setfattr -n user.foo -v bar cmp/x
Executing the following command:
rsync --recursive --xattrs --dry-run --compare-dest=../cmp/ src/ dst/
Results in the following error:
rsync: [generator] copy_xattrs: lsetxattr("/home/user/dst/x","user.foo")
failed: No such file or directory (2)
The issue only seems to occur if:
x is a directory (inside another, not as a direct argument)
--recursive, --xattrs and --dry-run are set
--compare-dest, --copy-dest or --link-dest are set
The error seems to come from:
source file generator.c in function recv_generator (line 1489)
https://github.com/WayneD/rsync/blob/v3.2.7/generator.c#L1489
The error seems to be either:
recv_generator (in generator.c) calling copy_xattrs when it should not
or
copy_xattrs (in xattrs.c) / sys_lsetxattr (in lib/sysxattrs.c) not checking the
dry-run flag similarly to the functions in syscall.c do
--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 5124] Parallelize the rsync run using multiple threads and/or connections
https://bugzilla.samba.org/show_bug.cgi?id=5124 --- Comment #12 from Paulo Marques --- Using multiple connections also helps when you have LACP network links, which are relatively common in data center setups to have both redundancy and increased bandwidth. If you have two 1Gbps links aggregated, you can only use 1Gbps using rsync, but you could use 2Gbps if rsync made several connections from different TCP ports. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15335] Environment variables in remote host's path do not resolve properly
https://bugzilla.samba.org/show_bug.cgi?id=15335 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |INVALID --- Comment #1 from Wayne Davison --- Read about it in the NEWS file, including how you can get the old behavior: https://download.samba.org/pub/rsync/NEWS#3.2.4 -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15335] New: Environment variables in remote host's path do not resolve properly
https://bugzilla.samba.org/show_bug.cgi?id=15335 Bug ID: 15335 Summary: Environment variables in remote host's path do not resolve properly Product: rsync Version: 3.2.0 Hardware: x86 OS: Linux Status: NEW Severity: major Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: ryan.lar...@infinitetactics.com QA Contact: rsync...@samba.org Target Milestone: --- We've had an rsync command that has worked for quite a while (1+ years), but suddenly started getting errors after we upgraded rsync from version 3.2.3 to 3.2.7. I'm not sure exactly which minor release broke our expectation of how environment variables are used in remote host paths, but there certainly seems to be some major change. See the example command below in which we were protecting the remote environment variable with single quotes to ensure it resolved on the remote host and not locally. I was able to reliable reproduce this issue with 3.2.7 and show that the command worked with 3.2.3. Example Command: rsync -rl /home/user/test_dir 'user@hostname:$PROJECTS_HOME/INFINITE' 3.2.7 -> rsync: mkdir "/p/home/rlarson/$PROJECTS_HOME/INFINITE" failed: No such file or directory 3.2.3 -> successful rsync A few notes: 1. The env var and path does exist on remote system 2. "/p/home/rlarson" is my $HOME dir on remote system 3. Both source and dest OS are Linux Anything else I can add to help troubleshoot this issue please let me know. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10170] rsync should support reflink similar to cp --reflink
https://bugzilla.samba.org/show_bug.cgi?id=10170 --- Comment #6 from Brian J. Murrell --- Does this --reflink feature have any parity/functionality with --link-dest, which is often used in "snapshot" style [i.e. daily] backup scripts on non-snapshottable filesystems such as XFS? XFS supports COW reflinks but not snapshots and as such rsync --link-dest is more appropriate for creating new backups. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 8690] Simple ACLs abort with "Unsupported attribute value (124)"
https://bugzilla.samba.org/show_bug.cgi?id=8690 --- Comment #1 from Björn Jacke --- AIXC type ACLs are incompatible with other ACLs like thised used on Linux. The only ACLs which are standardized are actually NFS4 ACLs. Rsync doesn't really support those unfortunately, yet. Linux also lacks support for NFS4 ACLs. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15163] rsync timeout non-effective and incorrect behaviour
https://bugzilla.samba.org/show_bug.cgi?id=15163 --- Comment #6 from roland --- nobody has a clue? i think proper rsync timeout handling is important. i have had whole nightly backup procedures hung for the whole night because rsync got stuck and didn't get timeout, i.e. machines did not get proper backup because of this. i would like to help making this more reliable. i think i have provided some useful input for it. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 Frank B changed: What|Removed |Added Resolution|WORKSFORME |FIXED --- Comment #10 from Frank B --- Ok, so you're saying the behavior before the upgrade was a bug (not transferring old files) and this has now been fixed in the new binary so that it's transferring the files again. This is what I was looking for - question answered. The documentation still doesn't cover that combination, but never mind. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #9 from Wayne Davison --- The combo of -I with -u briefly changed to be broken but it was fixed. The -u option means that older files on the sender are ignored. -I means that files with the same date are TRANSFERRED. When that was not occuring, it was a bug. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 Wayne Davison changed: What|Removed |Added Resolution|--- |WORKSFORME Status|REOPENED|RESOLVED -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #8 from Frank B --- Quick addition: You can say it's a "corner case", a result of wrong assumptions regarding u/I or a weird usecase but you can't say "nothing has changed" since that clearly isn't true. The question is: is uI supported (the documentation does not state this) and which behavior is correct, the old one before the update or the new one - or which behavior would you expect? I have a working solution now but that's definitely a point to be clarified and at least to be updated in the documentation. There could be other users too using this combination. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 Frank B changed: What|Removed |Added Status|RESOLVED|REOPENED Resolution|WORKSFORME |--- -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #7 from Frank B --- Yes, it has. The crontab was unchanged for months and directly after the update of rsync via apt, it started performing full replications. It's clearly a result of the new binary. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #6 from Kevin Korb --- I can confirm that something did change... # mkdir /tmp/src /tmp/dest # touch /tmp/src/a /tmp/src/b /tmp/src/c # rsync -vai /tmp/src/ /tmp/dest/ sending incremental file list .d..t.. ./ >f+ a >f+ b >f+ c sent 211 bytes received 76 bytes 574.00 bytes/sec total size is 0 speedup is 0.00 # touch /tmp/dest/b # /usr/src/rsync-3.2.0/rsync -vai -nIu /tmp/src/ /tmp/dest/ sending incremental file list sent 77 bytes received 12 bytes 178.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN) # /usr/src/rsync-3.2.7/rsync -vai -nIu /tmp/src/ /tmp/dest/ sending incremental file list >f. a >f. c sent 97 bytes received 22 bytes 238.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN) Guess that is up to Wayne if this is a fix or a regression. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #5 from Kevin Korb --- I had the same knee jerk reaction as Wayne to this question. -I means re-copy everything (or at least re-diff everything unless --whole-file). But I never attempted to mix it with -u so I held my tongue. Is it possible that in the past -I + -u meant re-copy everything but don't overwrite anything newer and now that exception is no longer honored? If that is the case I am not even sure whether this would be a regression or not. I have always considered -I to mean re-copy everything as an alternative to the even worse option of --checksum. But I can see a possible use case for it interacting with -u to redefine "everything". -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #4 from Wayne Davison --- No it didn't. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #3 from Frank B --- The behavior still changed as this was working for months and clearly changed directly after upgrading to the new binary but it's okay for me. Looks like the I should have been l in my case due to a copy and paste issue and different font appearances. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |WORKSFORME --- Comment #2 from Wayne Davison --- You need to re-read the manpage for -I. You seem to have reversed its meaning. https://download.samba.org/pub/rsync/rsync.1#opt--ignore-times -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 --- Comment #1 from Frank B --- rsync --version rsync version 3.2.3 protocol version 31 -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15254] New: rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31
https://bugzilla.samba.org/show_bug.cgi?id=15254 Bug ID: 15254 Summary: rsync performs full replication with option -I since last upgrade to version 3.2.3 protocol version 31 Product: rsync Version: 3.2.0 Hardware: x64 OS: Linux Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: coffeecupfr...@gmail.com QA Contact: rsync...@samba.org Target Milestone: --- Since rsync was upgraded to a more recent version (3.2.3 protocol version 31) on my Ubuntu Linux, rsync is performing a full replication every time although it shouldn’t because nothing changed in the source or the destination folder. The command I used was for example: rsync -aurvxHI --delete --log-file=/mnt/mirror1/logs/ironwolf_mirror1.log /media/ironwolf /mnt/mirror1/IronWolf >/dev/null 2>&1 I can replicate the issue in a simple test scenario /tmp/source to /tmp/dest via rsync -rauvxHI --delete --stats --log-file=/tmp/rsync.log /tmp/source/ /tmp/dest/ Performs a full replication every time the command is being launched rsync -rauvxH --delete --stats --log-file=/tmp/rsync.log /tmp/source/ /tmp/dest/ Does NOT perform a full synchronization unless there are real changes The problem appears to be the I but that should just avoid relying on timestamp and size only…and this was working for months in my environment as it was in a fixed crontab running several times a day. The behaviour has definitely changed in any case. From the man page: --ignore-times, -I don't skip files that match size and time My system: uname -a Linux home 5.15.0-53-generic #59-Ubuntu SMP Mon Oct 17 18:53:30 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 13082] [REQ] Hardware / SSE based MD5 operations
https://bugzilla.samba.org/show_bug.cgi?id=13082 --- Comment #6 from Andrew Bartlett --- Samba 4.11 moved to GnuTLS for our MD5 and other hash operations, and so uses any hardware optimisation available there. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12964] Maybe we can add the '--bind-cpu' option
https://bugzilla.samba.org/show_bug.cgi?id=12964 --- Comment #1 from Stefan Metzmacher --- On Linux you can use taskset (in combination with nice and ionice)... -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15163] rsync timeout non-effective and incorrect behaviour
https://bugzilla.samba.org/show_bug.cgi?id=15163
--- Comment #5 from roland ---
apparently, this is causing the problem:
if (am_receiver) {
return;
}
if i comment out the return statement, things work again.
@wayne, what is the reason that timeout checking is being skipped when
"am_receiver" = true ?
I don't understand the code well enough
--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15163] rsync timeout non-effective and incorrect behaviour
https://bugzilla.samba.org/show_bug.cgi?id=15163 --- Comment #4 from roland --- here with debug=all working correctly # /root/rsync/rsync-3.2.5/rsync -avi --timeout=5 --exclude='/proc' --exclude='/dev/' --exclude='/sys' --debug=all --msgs2stderr root@172.20.37.189:/iscsipool /zfspool/backup/172.20.37.189/backup opening connection using: ssh -l root 172.20.37.189 rsync --server --sender -vlogDtpre.iLsfxCIvu --msgs2stderr --timeout=5 . /iscsipool (12 args) (Client) Protocol versions: remote=31, negotiated=31 Client negotiated checksum: md5 receiving incremental file list [Receiver] send_msg(0, 36) [Receiver] io timeout after 6 seconds -- exiting rsync error: timeout in data send/receive (code 30) at io.c(197) [Receiver=3.2.5] [Receiver] _exit_cleanup(code=30, file=io.c, line=197): about to call exit(30) hang forever # /root/rsync/rsync-3.2.5/rsync -avi --timeout=5 --exclude='/proc' --exclude='/dev/' --exclude='/sys' --debug=all --msgs2stderr root@172.20.37.189:/ /zfspool/backup/172.20.37.189/backup 2>&1 |head -n50 opening connection using: ssh -l root 172.20.37.189 rsync --server --sender -vlogDtpre.iLsfxCIvu --msgs2stderr --timeout=5 . / (12 args) (Client) Protocol versions: remote=31, negotiated=31 Client negotiated checksum: md5 receiving incremental file list [Receiver] send_msg(0, 36) [Receiver] got msg=0, len=0 [Receiver] got msg=0, len=30051 get_local_name count=25 /zfspool/backup/172.20.37.189/backup [Receiver] change_dir(/zfspool/backup/172.20.37.189/backup) generator starting pid=9452 delta-transmission enabled recv_generator(.,0) recv_files(25) starting recv_generator(.,1) recv_generator(etc/ssl/certs/f39fc864.0,1770) recv_generator(etc/ssl/certs/f51bb24c.0,1771) recv_generator(etc/ssl/certs/fc5a8f99.0,1772) recv_generator(etc/ssl/certs/fe8a2cd8.0,1773) recv_generator(etc/ssl/certs/ff34af3f.0,1774) [receiver] send_msg(0, 0) [generator] send_msg(0, 0) [generator] got msg=0, len=0 [generator] send_msg(0, 0) [generator] send_msg(0, 0) [generator] send_msg(0, 0) [generator] send_msg(0, 0) [generator] send_msg(0, 0) [generator] send_msg(0, 0) -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15163] rsync timeout non-effective and incorrect behaviour
https://bugzilla.samba.org/show_bug.cgi?id=15163
--- Comment #3 from roland ---
ah, getting a clue
in io.c
static void check_timeout(BOOL allow_keepalive, int keepalive_flags)
{
time_t t, chk;
/* On the receiving side, the generator is now the one that decides
* when a timeout has occurred. When it is sifting through a lot of
* files looking for work, it will be sending keep-alive messages to
* the sender, and even though the receiver won't be sending/receiving
* anything (not even keep-alive messages), the successful writes to
* the sender will keep things going. If the receiver is actively
* receiving data, it will ensure that the generator knows that it is
* not idle by sending the generator keep-alive messages (since the
* generator might be blocked trying to send checksums, it needs to
* know that the receiver is active). Thus, as long as one or the
* other is successfully doing work, the generator will not timeout. */
if (!io_timeout)
return;
t = time(NULL);
if (allow_keepalive) {
/* This may put data into iobuf.msg w/o flushing. */
maybe_send_keepalive(t, keepalive_flags);
}
if (!last_io_in)
last_io_in = t;
if (am_receiver)
return;
chk = MAX(last_io_out, last_io_in);
if (t - chk >= io_timeout) {
if (am_server)
msgs2stderr = 1;
rprintf(FERROR, "[%s] io timeout after %d seconds --
exiting\n",
who_am_i(), (int)(t-chk));
exit_cleanup(RERR_TIMEOUT);
}
}
so , apparently when syncing "/" instead of "/iscsipool", keepalive messages
come into the play
i see keepalive messages getting sent, but the are NOT replied by the remote
rsync machine, as strace shows. there is only sshd process actively processing
data.
| 0 00 00 00 07 |
[pid 131555] 09:57:47.454101 select(14, [4 5 11 13], [], NULL, NULL) = 1 (in
[4])
[pid 131555] 09:57:50.455771 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[pid 131555] 09:57:50.455887 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 131555] 09:57:50.455963 read(4, "\0\0\0
\v\10\372\371(\256V\16$\17\375Z5\252Km-\357\311~A\352\202\232\365\234\33>"...,
16384) = 44
[pid 131555] 09:57:50.456064 select(14, [4 5 11 13], [10], NULL, NULL) = 1 (out
[10])
[pid 131555] 09:57:50.456225 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[pid 131555] 09:57:50.456745 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 131555] 09:57:50.456950 write(10, "\0\0\0\7", 4) = 4
| 0 00 00 00 07 |
[pid 131555] 09:57:50.457163 select(14, [4 5 11 13], [], NULL, NULL) = 1 (in
[4])
[pid 131555] 09:57:53.459934 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[pid 131555] 09:57:53.460391 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 131555] 09:57:53.460602 read(4, "\0\0\0
\303M\204S\275D\23r\225a\210\247Q\314\256\373]\\C\27\246\241\265\212\235\210\312o"...,
16384) = 44
[pid 131555] 09:57:53.460817 select(14, [4 5 11 13], [10], NULL, NULL) = 1 (out
[10])
[pid 131555] 09:57:53.461023 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[pid 131555] 09:57:53.461221 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 131555] 09:57:53.461314 write(10, "\0\0\0\7", 4) = 4
| 0 00 00 00 07 |
[pid 131555] 09:57:53.461526 select(14, [4 5 11 13], [], NULL, NULL) = 1 (in
[4])
[pid 131555] 09:57:56.463410 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[pid 131555] 09:57:56.463858 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 131555] 09:57:56.464211 read(4, "\0\0\0
\367\270\356\21\304-u\200cQ\16\350UAo\264*\231B\303\275\250\221Nzo\33\231"...,
16384) = 44
[pid 131555] 09:57:56.464336 select(14, [4 5 11 13], [10], NULL, NULL) = 1 (out
[10])
[pid 131555] 09:57:56.464684 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
[pid 131555] 09:57:56.464915 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid 131555] 09:57:56.465239 write(10, "\0\0\0\7", 4) = 4
| 0 00 00 00 07 |
[pid 131555] 09:57:56.465354 select(14, [4 5 11 13], [], NULL, NULL^Z
[1]+ Angehalten strace -f -p 131431 -tt -e write=all
root@debian:~# ps -ef|grep 131555
root 131555 131431 0 09:57 ?00:00:00 sshd: root@notty
root 131562 131555 3 09:57 ?00:00:02 rsync --server --sender
-vlogDtpre.iLsfxCIvu --timeout=5 . /
root 131565 131540 0 09:58 pts/100:00:00 grep 131555
--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read:
[Bug 15163] rsync timeout non-effective and incorrect behaviour
https://bugzilla.samba.org/show_bug.cgi?id=15163 --- Comment #2 from roland --- here is another bugreport, where timeout is not effective/working https://bugzilla.redhat.com/show_bug.cgi?id=1944132 -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15163] rsync timeout non-effective
https://bugzilla.samba.org/show_bug.cgi?id=15163
--- Comment #1 from roland ---
here is some strace from the backup host to show the difference
if i set timeout=60 , rsyncing "root@172.20.37.189:/" hangs forever:
08:25:30 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}) = 1 (in [3], left
{tv_sec=0, tv_usec=316390})
08:26:00 read(3, "\0\0\0\7", 32768) = 4
08:26:00 select(5, [3], [4], [3], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=94})
08:26:00 write(4, "\0\0\0\7", 4)= 4
| 0 00 00 00 07 |
08:26:00 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:26:30 select(5, [3], [4], [3], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=95})
08:26:30 write(4, "\0\0\0\7", 4)= 4
| 0 00 00 00 07 |
08:26:30 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:27:00 select(5, [3], [4], [3], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=94})
08:27:00 write(4, "\0\0\0\7", 4)= 4
| 0 00 00 00 07 |
08:27:00 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:27:30 select(5, [3], [4], [3], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=94})
08:27:30 write(4, "\0\0\0\7", 4)= 4
| 0 00 00 00 07 |
08:27:30 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:28:00 select(5, [3], [4], [3], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=94})
08:28:00 write(4, "\0\0\0\7", 4)= 4
| 0 00 00 00 07 |
08:28:00 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:28:30 select(5, [3], [4], [3], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=94})
08:28:30 write(4, "\0\0\0\7", 4)= 4
| 0 00 00 00 07 |
08:28:30 select(4, [3], [], [3], {tv_sec=30, tv_usec=0}
mind that tv_sec=30 is half of the timeout. if i set timeout=10 , tv_sec is 5.
rsyncing only /iscsipool also hangs, but times out . for my curiosity setting
60s timeout leads to 90s timeout, but not 60s. apparently, half of the timeout
is being added to the real timeout.
# time /root/rsync/rsync-3.2.5/rsync -avi --timeout=10 --exclude='/proc'
--exclude='/dev/' --exclude='/sys' root@172.20.37.189:/iscsipool
/zfspool/backup/172.20.37.189/backup
receiving incremental file list
[Receiver] io timeout after 10 seconds -- exiting
rsync error: timeout in data send/receive (code 30) at io.c(197)
[Receiver=3.2.5]
real0m15.356s <- !!!
user0m0.001s
sys 0m0.005s
# time /root/rsync/rsync-3.2.5/rsync -avi --timeout=60 --exclude='/proc'
--exclude='/dev/' --exclude='/sys' root@172.20.37.189:/iscsipool
/zfspool/backup/172.20.37.189/backup
receiving incremental file list
[Receiver] io timeout after 60 seconds -- exiting
rsync error: timeout in data send/receive (code 30) at io.c(197)
[Receiver=3.2.5]
real1m30.440s <-!!!
user0m0.002s
sys 0m0.003s
| 0 72 65 63 65 69 76 69 6e 67 20 69 6e 63 72 65 6d receiving increm |
| 00010 65 6e 74 61 6c 20 66 69 6c 65 20 6c 69 73 74 0a ental file list. |
08:35:49 mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = 0x7f74e5536000
08:35:49 mmap(NULL, 266240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS,
-1, 0) = 0x7f74e54f5000
08:35:49 select(6, [5], [4], [5], {tv_sec=30, tv_usec=0}) = 1 (out [4], left
{tv_sec=29, tv_usec=95})
08:35:49 write(4, "$\0\0\7\7\0\0\0- /proc\7\0\0\0- /dev/\6\0\0\0- "..., 40) =
40
| 0 24 00 00 07 07 00 00 00 2d 20 2f 70 72 6f 63 07 $...- /proc. |
| 00010 00 00 00 2d 20 2f 64 65 76 2f 06 00 00 00 2d 20 ...- /dev/- |
| 00020 2f 73 79 73 00 00 00 00 /sys |
08:35:49 select(6, [5], [], [5], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:36:19 select(6, [5], [], [5], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:36:49 select(6, [5], [], [5], {tv_sec=30, tv_usec=0}) = 0 (Timeout)
08:37:19 write(2, "[Receiver] io timeout after 60 s"..., 50[Receiver] io
timeout after 60 seconds -- exiting
) = 50
| 0 5b 52 65 63 65 69 76 65 72 5d 20 69 6f 20 74 69 [Receiver] io ti |
| 00010 6d 65 6f 75 74 20 61 66 74 65 72 20 36 30 20 73 meout after 60 s |
| 00020 65 63 6f 6e 64 73 20 2d 2d 20 65 78 69 74 69 6e econds -- exitin |
| 00030 67 0a g. |
08:37:19 rt_sigaction(SIGUSR1, {sa_handler=SIG_IGN, sa_mask=[],
sa_flags=SA_RESTORER, sa_restorer=0x7f74e4965400}, NULL, 8) = 0
08:37:19 rt_sigaction(SIGUSR2, {sa_handler=SIG_IGN, sa_mask=[],
sa_flags=SA_RESTORER, sa_restorer=0x7f74e4965400}, NULL, 8) = 0
08:37:19 wait4(27237, 0x7ffda918da9c, WNOHANG, NULL) = 0
08:37:19 getpid()
[Bug 15163] New: rsync timeout non-effective
https://bugzilla.samba.org/show_bug.cgi?id=15163 Bug ID: 15163 Summary: rsync timeout non-effective Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: devz...@web.de QA Contact: rsync...@samba.org Target Milestone: --- there seem to be circumstances, where timeout param is not effective if i do /root/rsync/rsync-3.2.5/rsync -avi --timeout=10 --exclude='/proc' --exclude='/dev/' root@172.20.37.189:/ /zfspool/backup/172.20.37.189/ it hangs forever , as soon as rsync tries to traverse some suspended/hanging /iscsipool on the remote machine (as this results in locked process as it's in non-interruptible syscall) if i do /root/rsync/rsync-3.2.5/rsync -avi --timeout=10 --exclude='/proc' --exclude='/dev/' root@172.20.37.189:/iscsipool /zfspool/backup/172.20.37.189/ i.e. if i try to rsync the "hung/stalled" directory directly, i'm getting proper timeout behaviour: receiving incremental file list [Receiver] io timeout after 10 seconds -- exiting rsync error: timeout in data send/receive (code 30) at io.c(197) [Receiver=3.2.5] so, apparently there seem to be circumstances where the timeout detection is not effective i tried to have a look with strace but don't get a clue what's the problem, as in both cases rsync gets stuck in /iscsipool if i run rsync locally on the machine containing the hung/stalled dir, it behaves like this: # rsync -av --timeout=1 --dry-run /iscsipool /tmp sending incremental file list [Receiver] io timeout after 1 seconds -- exiting rsync error: timeout in data send/receive (code 30) at io.c(197) [Receiver=3.2.3] i have seen rsync hang in my scripted backup solution on other situations, e.g. stale nfs mount etc and since i have a repro-case now, i decided to open a bugticket for it also see https://www.mail-archive.com/rsync@lists.samba.org/msg33555.html -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15122] Potential vulnerability: rsync creates files outside the target directory
https://bugzilla.samba.org/show_bug.cgi?id=15122 --- Comment #3 from Aditya Basu --- Apologies for the late response. It is definitely a bad idea to mix multi-case systems. However, note that even copying between case-honoring systems can have similar consequences, for ex. case-insensitive (icase) ZFS considers K (unicode kelvin sign) and k (alphabet) to be equivalent while icase ext4 does not. I agree with you analysis of the ordering. However, IMHO traversing symlinks at the target is not a wise choice. An *immediate* fix to this particular issue would be to prevent rsync for traversing symlinks at the target. However, a more *complete* fix should involve detecting collisions and stopping the copy. We're currently exploring different types of defenses for collisions. If you're interested, I will be happy to keep you in the loop. Finally, does it make sense to get a CVE number assigned? -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15122] Potential vulnerability: rsync creates files outside the target directory
https://bugzilla.samba.org/show_bug.cgi?id=15122 --- Comment #2 from Wayne Davison --- BTW, what happens in the test case you provided is that the generator creates TOPDIR and then TOPDIR/secret dirs before asking the sender to start a transfer of TOPDIR/secret/config. It then goes on to notice that topdir is present (since it uses stat) and that topdir/secret is an empty directory that is in the way of a symlink, so it replaces the dir with a symlink prior to the receiver doing its file-create work. If the topdirs had sorted in the opposite order, the symlink would have been replaced with a directory. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15122] Potential vulnerability: rsync creates files outside the target directory
https://bugzilla.samba.org/show_bug.cgi?id=15122 --- Comment #1 from Wayne Davison --- Yes, it's always bad to copy from a case-honoring filesystem to a case-ignoring filesystem as the filenames can overlap. This is something that the user just shouldn't do, as rsync is written to handle case-honoring filesystems. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 5642] Support use as an engine behind a GUI
https://bugzilla.samba.org/show_bug.cgi?id=5642 --- Comment #1 from c.bu...@posteo.jp --- As a maintainer of one of that rsync-using-GUIs I would find that also very nice. But I don't see an advantage with XML. To complex and not human readable. Cost a lot of resources (think about sustainability) to parse. For me JSON seems to me as a good minimal consent. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15154] New: manpage: Describe default behavior in context of --old-args/--protect-args
https://bugzilla.samba.org/show_bug.cgi?id=15154 Bug ID: 15154 Summary: manpage: Describe default behavior in context of --old-args/--protect-args Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: c.bu...@posteo.jp QA Contact: rsync...@samba.org Target Milestone: --- I am not a well experienced rsync user. So it could be that I misunderstood something and this report is wrong. Context: This is about how --old-args and --protected-args are described in the manpage. Problem: It seems to me that the manpage doesn't mention the default value or default behavior. Details: In my understanding that two arguments have an opposite meaning. So they can't be used together. The question about the default values is if I call rsync without any of that arguments how will rsync behave? I assume that `--protect-args` is the default behavior but I'm for away from being sure. If I am not wrong I would suggest to improve the manpage and describe the "default" behavior. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 15122] New: Potential vulnerability: rsync creates files outside the target directory
https://bugzilla.samba.org/show_bug.cgi?id=15122 Bug ID: 15122 Summary: Potential vulnerability: rsync creates files outside the target directory Product: rsync Version: 3.2.0 Hardware: All OS: Linux Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: azb...@psu.edu QA Contact: rsync...@samba.org Target Milestone: --- Created attachment 17422 --> https://bugzilla.samba.org/attachment.cgi?id=17422=edit POC of vulnerability The problem arises when trying to copy from a "case-sensitive" source to a "case-insensitive" target. The copy involves directories, files, and symbolic links (to directories). A maliciously crafted source directory can result in rsync following symbolic links and writing data outside the target directory. For a concrete example, consider the following source directory structure: SRC/ topdir/ secret (symlink to /tmp) TOPDIR/ secret/ config (file) We use rsync to recursively copy from SRC/ to TARGET/. Command: "rsync -a SRC/ TARGET/" Additionally, TARGET/ is on case-insensitive filesystem. Problem: During the copy, rsync creates the TOPDIR/secret/config (file) by following the symbolic link "topdir/secret". Hence, /tmp/config is created by rsync. We found a flag called: --copy-links which makes rsync follow symlinks at source before doing the copy. However, my understanding is that rsync should not follow symbolic links at the target, esp. the symbolic links it creates. I have attached a POC script that demonstrates this behavior. I have tested it on rsync versions 3.2.3 and 3.1.3. Compiling the latest version (3.2.4) of rsync results in an error during the ./configure step. Hence, I could not test it. Running Proof of concept script: The script requires two command line arguments: - Argument 1 = any empty case-sensitive directory - Argument 2 = any empty case-insensitive directory Example of invoking script for WSL: ./rsync-poc.sh ~/src /mnt/c/Users/xyz/dst -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 13463] Please consider using the IP_FREEBIND socket option
https://bugzilla.samba.org/show_bug.cgi?id=13463 --- Comment #5 from Simon Deziel --- The `Restart=on-failure` option was added in https://github.com/WayneD/rsync/commit/d41bb98c09bf0b999c4eee4e2125c7e5d0747ec4 This should paper over the problem of late showing IPv6 addresses due to DAD taking time. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 13463] Please consider using the IP_FREEBIND socket option
https://bugzilla.samba.org/show_bug.cgi?id=13463
--- Comment #4 from Simon Deziel ---
Since rsyncd exits with error code 10 ("Error in socket I/O") there are two
possible ways to improve the systemd unit:
[Service]
...
RestartForceExitStatus=10
Or:
[Service]
...
Restart=on-failure
Both should help with late showing IPv6 address due to DAD taking time.
--
You are receiving this mail because:
You are the QA Contact for the bug.
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 13463] Please consider using the IP_FREEBIND socket option
https://bugzilla.samba.org/show_bug.cgi?id=13463 --- Comment #3 from Andreas Hasenack --- Thanks for all the opinions. I have one remaining issue, and that is with "systemctl start rsync.service" not detecting the failure right away. The systemd unit file calls rsync like this: [Service] ExecStart=/usr/bin/rsync --daemon --no-detach This is correct, specially the --no-detach option. systemd should be able to tell immediately if the service started or not, but according to the systemd.service manpage, it will signal success in any case for type=simple services. It's not much better for type=exec. If I run rsync with an invalid config, it exits non-zero right away: root@j1-rsyncd:~# rsync --daemon --no-detach ; echo $? 10 But via systemctl, it exits 0: root@j1-rsyncd:~# systemctl start rsync; echo $? 0 root@j1-rsyncd:~# systemctl status rsync × rsync.service - fast remote file copy program daemon Loaded: loaded (/lib/systemd/system/rsync.service; disabled; vendor preset: enabled) Active: failed (Result: exit-code) since Wed 2022-03-30 19:10:03 UTC; 3s ago Docs: man:rsync(1) man:rsyncd.conf(5) Process: 4305 ExecStart=/usr/bin/rsync --daemon --no-detach (code=exited, status=10) Main PID: 4305 (code=exited, status=10) CPU: 3ms Mar 30 19:10:03 j1-rsyncd rsyncd[4305]: bind() failed: Cannot assign requested address (address-family 2) Mar 30 19:10:03 j1-rsyncd systemd[1]: Started fast remote file copy program daemon. Mar 30 19:10:03 j1-rsyncd rsyncd[4305]: unable to bind any inbound sockets on port 873 Mar 30 19:10:03 j1-rsyncd systemd[1]: rsync.service: Main process exited, code=exited, status=10/n/a Mar 30 19:10:03 j1-rsyncd rsyncd[4305]: rsync error: error in socket IO (code 10) at socket.c(545) [Receiver=3.2.3] Mar 30 19:10:03 j1-rsyncd systemd[1]: rsync.service: Failed with result 'exit-code'. This seems to be the norm for this type of systemd service (type=simple, type=exec), and looks like the most reliable way to have systemctl start detect immediately if the service failed or not would be to implement systemd's notify[1] mechanism in rsync. Type=forking might be an alternative, but this timeout would have to be tuned: root@j1-rsyncd:~# time systemctl start rsync Job for rsync.service failed because a timeout was exceeded. See "systemctl status rsync.service" and "journalctl -xeu rsync.service" for details. real1m30.246s With TimeoutStartSec=5 in the unit file: root@j1-rsyncd:~# time systemctl start rsync Job for rsync.service failed because a timeout was exceeded. See "systemctl status rsync.service" and "journalctl -xeu rsync.service" for details. real0m5.287s 1. https://www.freedesktop.org/software/systemd/man/sd_notify.html -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 8682] Skip current transfer keyboard function
https://bugzilla.samba.org/show_bug.cgi?id=8682 --- Comment #6 from Joachim Wagner --- The "echo > sourcefile" workaround seems to not trigger an error, at least not straight away and at least on XFS, but instead speeds up the operation and stops writing more data to the target file. Observation (both source and target filesystems are local, 5.7 TB file at 68% at time of intervention): * Before "echo > sourcefile", rsync -P reported about 200MB/s * After "echo > sourcefile", rsync -P reports about 650MB/s and the target file's mtime and size no longer change * At 100%, rsync says "read errors mapping "redacted": No data available (61) WARNING: redacted failed verification -- update discarded (will try again)." and then syncs the 1-byte file created with `echo`. Still grateful @martin for sharing the idea. I found this searching for a way to inject an I/O error. For the source file backup, `cp --reflink=always` is your friend. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14962] Crash/restart using rsync 3.2.3 on M1 Mac
https://bugzilla.samba.org/show_bug.cgi?id=14962 --- Comment #1 from Mike Bombich --- This is probably the same kernel memory leak I documented here: https://bombich.com/kb/ccc6/macos-monterey-known-issues#smb_panics You can confirm whether it's the same issue by looking for zone_map_exhaustion in the system log. e.g.: - Start your rsync task - Run `log --stream | grep zone_map_exhaustion` in a Terminal window - Run `sudo zprint -d kext.kalloc.32768` in another Terminal window Also watch memory pressure in Activity Monitor. I saw a gradual and steady increase, although it never got to a "warning" level before memoryd started killing things. If that's what you're seeing, it's not an rsync bug, it's a bug in macOS. Mike > On Jan 27, 2022, at 11:26AM, just subscribed for rsync-qa from bugzilla via > rsync wrote: > > https://bugzilla.samba.org/show_bug.cgi?id=14962 > >Bug ID: 14962 > Summary: Crash/restart using rsync 3.2.3 on M1 Mac > Product: rsync > Version: 3.2.0 > Hardware: All >OS: All >Status: NEW > Severity: normal > Priority: P5 > Component: core > Assignee: wa...@opencoder.net > Reporter: arjunm...@gmail.com >QA Contact: rsync...@samba.org > Target Milestone: --- > > Hi there, > > Whole (macOS) system is crashing when transferring a large number of files > from > Mac to NAS via rsync. > > Running the below command on a directory with thousands of images (~5MB each) > the transfer speed eventually slows down, the system gets choppy, freezes and > then eventually the entire computer reboots. > > rsync -va --update --info=progress2 /Volumes/Source/folder > /Volumes/Destination/folder > > I found this reported here as well: > https://apple.stackexchange.com/questions/421007/rsync-3-2-3-keeps-crashing-on-mac-mini-m1# > > I experienced this on a M1 Mac mini running macOS Monterey. The stackexchange > post is running Big Sur. > > rsync 3.2.3 was installed via homebrew (brew install rsync) so it may be > possible this is a homebrew distribution/compile problem. > > The only solution for me was to downgrade back to use the included rsync in > macOS (rsync version 2.6.9 protocol version 29) > > -- > You are receiving this mail because: > You are the QA Contact for the bug. > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html > -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14962] New: Crash/restart using rsync 3.2.3 on M1 Mac
https://bugzilla.samba.org/show_bug.cgi?id=14962 Bug ID: 14962 Summary: Crash/restart using rsync 3.2.3 on M1 Mac Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: arjunm...@gmail.com QA Contact: rsync...@samba.org Target Milestone: --- Hi there, Whole (macOS) system is crashing when transferring a large number of files from Mac to NAS via rsync. Running the below command on a directory with thousands of images (~5MB each) the transfer speed eventually slows down, the system gets choppy, freezes and then eventually the entire computer reboots. rsync -va --update --info=progress2 /Volumes/Source/folder /Volumes/Destination/folder I found this reported here as well: https://apple.stackexchange.com/questions/421007/rsync-3-2-3-keeps-crashing-on-mac-mini-m1# I experienced this on a M1 Mac mini running macOS Monterey. The stackexchange post is running Big Sur. rsync 3.2.3 was installed via homebrew (brew install rsync) so it may be possible this is a homebrew distribution/compile problem. The only solution for me was to downgrade back to use the included rsync in macOS (rsync version 2.6.9 protocol version 29) -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 11879] escape rrsync restricted folder
https://bugzilla.samba.org/show_bug.cgi?id=11879 Wayne Davison changed: What|Removed |Added Resolution|--- |FIXED Status|NEW |RESOLVED --- Comment #5 from Wayne Davison --- The latest rsync has a default lock similar to what you proposed (it just locks the restricted dir, not a separate file) and the improved arg checking and rejected symlink-copying options. It also accepts a "-munge" option (in the accepted_keys file) that can be used to enable rsync's symlink munging, possibly combined with a new "-no-lock" option to disable the new single-use instance locking. Thanks for your patch, and apologies that it is so late. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14916] New: --times act like a skip switch if --compare-dest is used
https://bugzilla.samba.org/show_bug.cgi?id=14916 Bug ID: 14916 Summary: --times act like a skip switch if --compare-dest is used Product: rsync Version: 3.1.3 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: m...@gutt.it QA Contact: rsync...@samba.org Target Milestone: --- I created the following dirs and files: ### mkdir /tmp/src mkdir /tmp/dst mkdir /tmp/comp touch /tmp/src/test touch /tmp/comp/test ### Now, "test" has the same checksum, but different modification time. I use "--checksum" to ignore modification times and it works as expected: ### rsync --recursive --links --perms --times --group --owner --devices --specials --verbose --checksum --dry-run /tmp/src/ /tmp/comp sending incremental file list ./ sent 79 bytes received 22 bytes 202.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN) ### But it does not if I use "--compare-dest": ### rsync --recursive --links --perms --times --group --owner --devices --specials --verbose --checksum --dry-run --compare-dest=/tmp/comp /tmp/src/ /tmp/dst sending incremental file list ./ test sent 83 bytes received 22 bytes 210.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN) ### This works only, if I disable copying the modification times by removing "--times": ### rsync --recursive --links --perms --group --owner --devices --specials --verbose --checksum --dry-run --compare-dest=/tmp/comp /tmp/src/ /tmp/dst sending incremental file list sent 73 bytes received 12 bytes 170.00 bytes/sec total size is 0 speedup is 0.00 (DRY RUN) ### This means "--times" acts like a skip switch if "--compare-dest" is used, which I think is wrong. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10629] rsync follows symlinks that point to same directory / endless loop
https://bugzilla.samba.org/show_bug.cgi?id=10629 --- Comment #2 from Kevin Korb --- Since find is one of the few utilities that actually corrects for symlink loops you can use it as a workaround. Something like: cd /source/path ; find -L . -print | rsync ... --copy-links --files-from=- ./ /target/path/ -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10629] rsync follows symlinks that point to same directory / endless loop
https://bugzilla.samba.org/show_bug.cgi?id=10629 Timothee Besset changed: What|Removed |Added CC||tt...@ttimo.net --- Comment #1 from Timothee Besset --- This problem still exists in rsync v3.2.3. `--copy-links` cannot be used when symlink loops are present. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12769] error allocating core memory buffers (code 22) depending on source file system
https://bugzilla.samba.org/show_bug.cgi?id=12769 --- Comment #14 from Roland Haberkorn --- After the new version made it into my system I can confirm it works like a charm. Many thanks for the effort. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14798] New: Metadata traffic --- uncompressed with -z, interaction with --bwlimit and ssh compression
https://bugzilla.samba.org/show_bug.cgi?id=14798 Bug ID: 14798 Summary: Metadata traffic --- uncompressed with -z, interaction with --bwlimit and ssh compression Product: rsync Version: 3.1.3 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: z...@smallinteger.com QA Contact: rsync...@samba.org Target Milestone: --- Consider the case where rsync is tasked to synchronize a large file set in which there are few changes. Anecdotal evidence (duckduckgo search) suggests most of the network traffic will be spent exchanging file metadata, rather than file content, as intended. The same anecdotal evidence suggests this "file list" is not exchanged in compressed form between rsync's endpoints, even when using the -z switch. This seems accurate: setting up a suitable experiment shows ssh compression reduces overall bandwidth usage by roughly 2x in these cases. This seems an opportunity for improvement. The benefits would be compounded when using --bwlimit. In this case, disabling ssh compression results in traffic that respects the requested shape. However, this traffic is measured at the rsync endpoints. Consequently, rsync will not use the available bandwidth effectively, precisely because in this use case there are very few file changes in the file set (which is the point of using rsync). Note that since ssh compression is unpredictable, adequately adjusting --bwlimit for maximum efficiency is impossible. Thus, bandwidth usage will be optimal without -z (but with redundant traffic without ssh or rsync compression), or suboptimal with or without -z and --bwlimit (due to ssh compressing file metadata without rsync realizing). In these cases, the time required for rsync to complete the task remains unchanged regardless of the form of compression. Would it be possible to rsync's -z switch to set up the equivalent of two compressed streams, one for file data, another for file metadata, which are then multiplexed over the wire? In that way, ssh compression would be entirely unnecessary, and --bwlimit would still result in maximum efficiency even when most traffic is file metadata. Having rsync compress the file list is likely to result in better compression than ssh could achieve because the shape of the file metadata will be known to rsync. I could not find previous bug reports on this specific issue in the bug database --- I searched for bugs related to -z and --bwlimit, and I also searched through the release notes in case this (or an equivalent) enhancement has been applied recently. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14683] failed to set permissions on symlinks; need `--omit-link-permissions` option
https://bugzilla.samba.org/show_bug.cgi?id=14683 --- Comment #2 from Ciprian Dorin Craciun --- (In reply to Ciprian Dorin Craciun from comment #1) Trying to `strace` what `rsync` does in my OpenAFS use-case I've found that the only syscals invoked by `rysync` (and pertaining to the file in question) are: * `readlink`; * `newfstatat`; * `openat`; * `symlink`; Furthermore, none of them actually fail, instead it seems that the "Operation not supported (95)" is actually generated internally (although I couldn't easily identify its origin). -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14683] failed to set permissions on symlinks; need `--omit-link-permissions` option
https://bugzilla.samba.org/show_bug.cgi?id=14683 --- Comment #1 from Ciprian Dorin Craciun --- I've encountered a similar situation, but with OpenAFS, which for some reason reports the protection for symlinks as `rwxr-xr-x`. Thus using `rsync` with `--perms` and targeting an OpenAFS folder fails with a similar error stating that it can't `chmod` the permissions for that symlink. I am similarly using 3.2.3 on OpenSUSE Tumbleweed (the client) and the same version on OpenSUSE Leap 15.2 (the server). A few monts ago (say January) I've not encountered this error (I'm using the same scripts as before). In essence to reproduce this issue one just has to: mkdir /tmp/source ln -s /dev/null /tmp/source/some-symlink stat /tmp/source/some-symlink # Access: (0777/lrwxrwxrwx) Uid: (1000/ ciprian) Gid: (1000/ ciprian) mkdir /afs/.../target ln -s /dev/null /afs/.../target/some-other-symlink stat /afs/.../target/some-other-symlink # Access: (0755/lrwxr-xr-x) Uid: (33025/ UNKNOWN) Gid: (1000/ ciprian) # up to here we've checked that the source file-system behaves normaly while OpenAFS forces the protection I've stated. rsync -r -p -i --links /tmp/source/some-symlink /afs/.../target/ rsync: [generator] failed to set permissions on "/afs/.../target/some-symlink": Operation not supported (95) cL+ some-symlink -> /dev/null rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3] (The same happens when the target is via SSH.) -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14683] New: failed to set permissions on symlinks; need `--omit-link-permissions` option
https://bugzilla.samba.org/show_bug.cgi?id=14683 Bug ID: 14683 Summary: failed to set permissions on symlinks; need `--omit-link-permissions` option Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: m...@shadsterling.com QA Contact: rsync...@samba.org Target Milestone: --- After updating to 3.2.3 (along with updating other packages) I'm getting errors of the form `rsync: [generator] failed to set permissions on "/blah/blah/blah": Operation not supported (95)` on symlinks. I assume the underlying OpenSuse system doesn't support setting permissions on symlinks, but I do want to sync permissions on everything but symlinks without getting these errors. I think fixing this would require an option like `--omit-link-permissions` similar to the existing option `--omit-link-times` -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #38 from Claudius Ellsel --- This basically is some personal preference. I know that I can do this on btrfs (which is used on the system I want to back up from), also pretty easy with tools like snapper. Maybe it would be feasible to do that comparison everytime before creating a new manual snapshot, which then acts a bit like a "commit". I basically want to have one stage at which I know that I have checked the changes made. Ideally that is at the backup stage, so the backup only has checked changes. I'll think about that method, maybe even converting my backup drive to btrfs as well. Back on topic, this feature still comes handy in many situations, imho. There might be workarounds, but those might not be that feasible or wanted. The reality of this not being merged soon might force users to turn to workarounds (which might also have other advantages) though. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #37 from elatl...@gmail.com --- The btrfs equivalent is a bit more rough but (link for rename); #./btrfs-snapshots-diff.py -sb -p /media/btrfs/v_1/s_1 -c /media/btrfs/v_1/s_2 | grep -E path=. | grep -v utimes | tail -n +2 link;path=fileB;path_link=fileC unlink;path=fileB -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #36 from elatl...@gmail.com --- (In reply to Claudius Ellsel from comment #35) > This is going off-topic On such an old bug with modern workarouds I think it's worth talking about. > backup drive is NTFS currently, which would complicate things probably. Using a COW FS would require you you use said FS (not NTFS ... you can put files systems inside each other with losetup but it's not ideal) If you really don't want to use another filesystem then the options are git or git-annex or that wrapper script I cooked up, or there is likely some backup software with the functionality baked in. > > If you want to review changes before backup > [and] detect moved files you can with a COW FS like this (R for renamed); # zfs diff pool/volume@snap1 pool/volume@snap2 M /pool/volume/ R /pool/volume/fileB -> /pool/volume/fileC -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #35 from Claudius Ellsel --- (In reply to elatllat from comment #34) >Yes you can easily access files on a COW-FS backup; it's a file system, that's >>what it's for. This is going off-topic, but my backup drive is NTFS currently, which would complicate things probably. >If you want to review changes before backup you can just diff or rsync >--dry-run >snapshot/a snapshot/b locally before sending it. Unfortunately due to rsync's current inability to detect moved files that would result in exactly the same problem I currently have: Not being able to know whether a file was deleted and another different one was created or whether it is just the same file that was moved (or renamed). -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #34 from elatl...@gmail.com --- (In reply to Claudius Ellsel from comment #33) Yes you can easily access files on a COW-FS backup; it's a file system, that's what it's for. If you want to review changes before backup you can just diff or rsync --dry-run snapshot/a snapshot/b locally before sending it. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #33 from Claudius Ellsel --- Hm, those backups won't work on file level, though afaik. Thus I cannot easily access files on a backup drive for example. Also I want to use this as some kind of confirm stage a bit like committing with git where I review all changes to files and confirm them when transferring them to the backup. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #32 from elatl...@gmail.com --- (In reply to Claudius Ellsel from comment #31) Yes any COW FS with "send/receive" will have inherent rename handaling, and will be faster than rsync because the diffs are inherent. With zfs one can even have encrypted backups without the backup server ever seeing the key or un-encrypted data. COW is not good for databases, VM imgs, etc but that's not where rename detection is useful either. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #31 from Claudius Ellsel --- To me and others it still seems relevant. I have to admit though that I haven't looked much into other solutions for backups like btrfs send/receive commands. I suppose that were the ones you meant? Note that simply using btrfs and creating snapshots can be barely seen as a backup. I am rather talking about backing up data to external media (ideally off site, but that is a different story). -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 8367] Add a feature to --rename-existing files
https://bugzilla.samba.org/show_bug.cgi?id=8367 Claudius Ellsel changed: What|Removed |Added CC||claudius.ell...@live.de --- Comment #7 from Claudius Ellsel --- I didn't have a very close look. Just for understanding, what is the difference to `-b` option? That the files remain in the folder and are not moved to the backup directory? -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #30 from elatl...@gmail.com --- This feature request is so old it has lost relavence because btrfs/zfs/etc are more optimal backup solutions than rsync. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10263] Extend Behaviour of the --fuzzy Parameter to Consider Directories
https://bugzilla.samba.org/show_bug.cgi?id=10263 --- Comment #3 from Claudius Ellsel --- I have to admit that I only skimmed through the description. There is definitely some more to this than in the other bug. Have a look at https://bugzilla.samba.org/show_bug.cgi?id=2294#c14 though. The patches linked there seem to already implement much of what you suggest (but not things like fine grained control over depth for example). Make sure to click them, they contain a detailed description of what they do. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10263] Extend Behaviour of the --fuzzy Parameter to Consider Directories
https://bugzilla.samba.org/show_bug.cgi?id=10263 --- Comment #2 from Haravikk --- It's certainly similar but I wouldn't say a direct duplicate; 2294 is requesting detection of move/rename *somehow* which is a tricky proposition (especially with rsync defaulting towards incremental send rather than processing everything in advance). I posted this issue with a specific intention of how to extend the existing --fuzzy parameter (and hopefully clear-ish on what I mean, as I definitely should have re-read it a couple more times before submitting). If the new behaviour proposed here were implemented it *could* be considered enough of a fix to satisfy 2294 as well though, or at least to satisfy it in cases of incremental sending, as this solution wouldn't require the whole file tree to be established for comparisons to be made for detecting move/renames. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 --- Comment #29 from Claudius Ellsel --- As another motivation for this, I use rsync for backups and would like to be able to see whether files have just been renamed or were deleted and some others newly created (which currently cannot be distinguished). That way I can make sure nothing was modified by mistake when backing up my data by going through the output when using dry run before. Basically I want to replace FreeFileSync that has this ability. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 2294] Detect renamed files and handle by renaming instead of delete/re-send
https://bugzilla.samba.org/show_bug.cgi?id=2294 Claudius Ellsel changed: What|Removed |Added CC||claudius.ell...@live.de --- Comment #28 from Claudius Ellsel --- What is the current state of this? Am I correct that this is still not available in official released versions? Is this still the right place for tracking it or should it be moved to the GitHub tracker? -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10263] Extend Behaviour of the --fuzzy Parameter to Consider Directories
https://bugzilla.samba.org/show_bug.cgi?id=10263 Claudius Ellsel changed: What|Removed |Added CC||claudius.ell...@live.de --- Comment #1 from Claudius Ellsel --- This is probably related, maybe even a duplicate of https://bugzilla.samba.org/show_bug.cgi?id=2294. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14529] Please add option to save metadata to single file to speed up backups
https://bugzilla.samba.org/show_bug.cgi?id=14529 --- Comment #1 from Andras Korn --- It's completely fine if using this "database" in writable modules implies or requires `max connections = 1` to avoid concurrency/locking issues. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14529] New: Please add option to save metadata to single file to speed up backups
https://bugzilla.samba.org/show_bug.cgi?id=14529 Bug ID: 14529 Summary: Please add option to save metadata to single file to speed up backups Product: rsync Version: 3.2.0 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: korn-bugzilla.samba@elan.rulez.org QA Contact: rsync...@samba.org Target Milestone: --- There are compelling reasons to use rsync as a backup tool; then snapshot the destination fs to preserve the current backup; and save the next backup to the same destination, again using rsync. In this scenario, the data in the backup filesystem is only ever changed by rsync. If there are many files, a backup run will take a very long time and most I/O will be spent in reading the metadata of files to see if the source is different from the destination: % time seconds usecs/call callserrors syscall -- --- --- - - 45.850.627125 31 20222 lstat 30.610.418682 20 20222 lgetxattr 13.540.185181 79 2338 getdents64 3.230.044241 22 1982 1982 getxattr 2.340.032001 16 1982 stat 1.780.024293 20 1169 openat 1.250.017112 14 1169 close 1.050.014389 12 1169 fstat 0.270.003737 19 187 brk 0.040.000503 4511 write 0.020.000306 2711 read 0.010.000159 1411 select -- --- --- - - 100.001.367729 50473 1982 total If rsync could be told to save all metadata to some "database" in addition to the filesystem, the load on the backup server on subsequent backups of the same source data to the same destination could be much lower. The "database" could be read into RAM, perhaps in chunks if it's very large, and checking metadata for changes would be almost free. Of course, if data is changed in the actual filesystem by a tool other than rsync (which would keep the "database" updated), the "database" gets out of sync, but that can't be helped. This could also be an enhancement of "fake super" -- instead of saving metadata in an xattr for each file separately, all metadata could be saved in a single file, in a location outside the root of the rsync module (or, to support chroot, inside it, but hidden from rsync transfers). -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14463] rsync 3.2.2 server protocol error
https://bugzilla.samba.org/show_bug.cgi?id=14463 --- Comment #4 from Wayne Davison --- The pre-release patches aren't guaranteed to be backward compatible, and in this case the bits that were used in a couple different patches actually conflicted with each other. So, when --atimes was promoted it became impossible for anything using the older --fileflags patch to work with a newer rsync (or the older --crtimes option, which also had to change bits). Your best bet is to either copy an extra rsync binary onto the non-upgraded target system (it doesn't have to be installed) and use the --rsync-path option to tell rsync where to run the newer version OR to copy an older rsync version onto the upgraded system so that you can use that rsync-old-fileflags binary to copy to a system with the older patch. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14463] rsync 3.2.2 server protocol error
https://bugzilla.samba.org/show_bug.cgi?id=14463 --- Comment #3 from bumkick...@yahoo.com --- It's not a trivial exercise to upgrade the rsync version on the target system, so it would be useful if there was some kind of "back patch" available | suspect there might be others with similar issues ... -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14463] rsync 3.2.2 server protocol error
https://bugzilla.samba.org/show_bug.cgi?id=14463 --- Comment #2 from bumkick...@yahoo.com --- (In reply to Wayne Davison from comment #1) What should we do instead to keep the same functionality? -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14463] rsync 3.2.2 server protocol error
https://bugzilla.samba.org/show_bug.cgi?id=14463 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |INVALID --- Comment #1 from Wayne Davison --- Yup, the older --file-flags patch isn't compatible with the newer one. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 14463] New: rsync 3.2.2 server protocol error
https://bugzilla.samba.org/show_bug.cgi?id=14463 Bug ID: 14463 Summary: rsync 3.2.2 server protocol error Product: rsync Version: 3.2.0 Hardware: All OS: FreeBSD Status: NEW Severity: normal Priority: P5 Component: core Assignee: wa...@opencoder.net Reporter: bumkick...@yahoo.com QA Contact: rsync...@samba.org Target Milestone: --- *** client:freebsd 8.4-STABLE rsync 3.1.1 root 224 > /usr/local/bin/rsync --address=10.246.110.2 -zz -v --fileflags --force-change --numeric-ids --force --super -8 toor@droplet02::grizzle/ /bkup/droplet02 rsync error: error in rsync protocol data stream (code 12) at io.c(226} [Receiver=3.1.1] *** server:freebsd 11.4-STABLErsync 3.2.2 rsync error: protocol incompatibility (code 2) at compat.c(723) [sender=3.2.2] This started happening after I upgraded the server from rsync version 3.1.3 to 3.2.2 it works if "--fileflags" is removed -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 11979] rsync -X fails without need
https://bugzilla.samba.org/show_bug.cgi?id=11979 --- Comment #3 from Wayne Davison --- Nope, that's the whole point. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 5728] Change --min-size and --max-size to filter the file lists, like excludes
https://bugzilla.samba.org/show_bug.cgi?id=5728 Wayne Davison changed: What|Removed |Added Status|REOPENED|RESOLVED Resolution|--- |WONTFIX --- Comment #4 from Wayne Davison --- I am not included to change how these behave. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 5820] make rsync update symlinks/devices/specials atomically
https://bugzilla.samba.org/show_bug.cgi?id=5820 Wayne Davison changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |FIXED --- Comment #2 from Wayne Davison --- The code got an atomic_create() function a while back so that the generator's updates get renamed into place in an atomic operation. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 6741] 'deleting' messages show up in improper places
https://bugzilla.samba.org/show_bug.cgi?id=6741 Wayne Davison changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |WONTFIX --- Comment #4 from Wayne Davison --- I'm not inclined to change this. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 6928] Warn the user to use --modify-window=1 --no-owner --no-group etc. on FAT partitions
https://bugzilla.samba.org/show_bug.cgi?id=6928 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |WONTFIX --- Comment #3 from Wayne Davison --- There's no good way for rsync to know that it is copying onto a FAT filesystem. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 6821] Assertion failed so rsync crashes (path includes special char)
https://bugzilla.samba.org/show_bug.cgi?id=6821 Wayne Davison changed: What|Removed |Added Status|ASSIGNED|RESOLVED Resolution|--- |FIXED --- Comment #2 from Wayne Davison --- I'm assuming that this is fixed. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 3465] Option to delete unlisted files with --files-from
https://bugzilla.samba.org/show_bug.cgi?id=3465 --- Comment #8 from Wayne Davison --- Jeff: the command you mention already works fine, since --files-from is not involved. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10405] Feature request: Add support for pre/post cmds for the rsync client
https://bugzilla.samba.org/show_bug.cgi?id=10405 Wayne Davison changed: What|Removed |Added Resolution|--- |WONTFIX Status|NEW |RESOLVED --- Comment #4 from Wayne Davison --- These are called shell scripts. :-) If you're not using a daemon, you're expected to code up your copy commands yourself. The daemon gets some special support since there is not a way to tell the server side to run some extra commands. Note that you can choose to use ssh to run a daemon rsync that has certain pre/post commands included in the config if that is something that you would find useful. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 11609] Incorrect (or at least dangerous) behaviour of --append-verify
https://bugzilla.samba.org/show_bug.cgi?id=11609 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |INVALID --- Comment #1 from Wayne Davison --- The use of --append-verify is not generally recommended, as log files are better served by --append (when you know what you're transferring) and other files are better served by --inplace. In your case, you should probably have just used --inplace. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 13660] State clearly in manpage that --append-verify is an edge-case
https://bugzilla.samba.org/show_bug.cgi?id=13660 --- Comment #2 from Jonas Eberle --- Thanks. Could you point to a commit where this has been changed please? -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 8990] It might be nice to make --append-verify also transfer non-appending files
https://bugzilla.samba.org/show_bug.cgi?id=8990 Wayne Davison changed: What|Removed |Added Resolution|--- |WONTFIX Status|ASSIGNED|RESOLVED --- Comment #4 from Wayne Davison --- The --append* options should be targetted at just log files that are known to be good except for some missing trailing data. The man page has been updated to warn more about its use. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 8856] --hard-links does not handle hard-linked symlinks correctly on FreeBSD
https://bugzilla.samba.org/show_bug.cgi?id=8856 Wayne Davison changed: What|Removed |Added Resolution|--- |FIXED Status|ASSIGNED|RESOLVED --- Comment #11 from Wayne Davison --- I've made the change to using linkat() conditional on the OS having that function. The next release will have this fixed. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12153] Possible malfunctions of rsync when destination is on fat32 filesystem
https://bugzilla.samba.org/show_bug.cgi?id=12153 Wayne Davison changed: What|Removed |Added Resolution|--- |WORKSFORME Status|NEW |RESOLVED --- Comment #5 from Wayne Davison --- If rsync aborts with an error then there might be more files yet to transfer than rsync didn't get around to. The extra transferring might be the off-by-one timestamps since some FAT filesystems cannot store an odd timestamp. You should also check to make sure that you don't have files that differ by upper/lower case that are confusing rsync. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10675] rsyncing >2GB file onto fat32 partition should fail earlier
https://bugzilla.samba.org/show_bug.cgi?id=10675 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |WONTFIX --- Comment #2 from Wayne Davison --- The problem is that rsync doesn't know that the destination filesystem is not going to allow a large file until the file grows to be too large. You could probably get an earlier error if you use --preallocate, though. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12199] multiple link-dest dirs not working
https://bugzilla.samba.org/show_bug.cgi?id=12199 Wayne Davison changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #3 from Wayne Davison --- There have been some fixes in this area a little while back, so this seems to be working for me in the latest source. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 12915] --modify-window should default to 1 for fat filesystems
https://bugzilla.samba.org/show_bug.cgi?id=12915 Wayne Davison changed: What|Removed |Added Resolution|--- |WONTFIX Status|NEW |RESOLVED --- Comment #9 from Wayne Davison --- Rsync doesn't know the filesystem type, so it's up to you to specify the option when you need it. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
[Bug 10092] Hang when remote dest's disk is full
https://bugzilla.samba.org/show_bug.cgi?id=10092 Wayne Davison changed: What|Removed |Added Status|REOPENED|RESOLVED Resolution|--- |FIXED --- Comment #6 from Wayne Davison --- The upcoming release has some improvements to get disk-full errors to the user more quickly. -- You are receiving this mail because: You are the QA Contact for the bug. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
