[Bug 1845652] Re: rsync sends everything, not just changes
Thanks for following up, I'm closing the issue as requested. ** Changed in: rsync (Ubuntu) Status: Incomplete => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
On a prior OS (16.04 or 18.04, cannot remember which) the following rsync -v -a -r -W -p -o -g -t --delete --stats ~/Pictures /media/$USER /USB-128GB-1/ >bkp_log would only send differences on subsequent runs with a fat32 USB key. Your test above seems to indicate that subsequent runs on 19.04 send everything for vfat. Therefore, my initial test on 19.04 must have been a fat32 USB key and not a LUKS/ext4 USB key. Please close this case if you are satisfied with these explanations. Thanks. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
I have run 2 test scenarios. A trivial folder containing 3 text files as source. A real world case of ~/ as source. Both cases work as expected on an LUKS encrypted ext4 USB key. I will do some more testing on fat32 and report back. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
When using vfat and running rsync twice, this is the diff in the stats output: --- log1.vfat 2019-10-08 11:43:04.381667860 -0300 +++ log2.vfat 2019-10-08 11:43:17.609728451 -0300 @@ -1,5 +1,5 @@ Number of files: 78 (reg: 77, dir: 1) -Number of created files: 77 (reg: 77) +Number of created files: 0 Number of deleted files: 0 Number of regular files transferred: 77 Total file size: 3,990,531 bytes @@ -12,6 +12,6 @@ Total bytes sent: 3,996,271 Total bytes received: 8,830 -sent 3,996,271 bytes received 8,830 bytes 534,013.47 bytes/sec +sent 3,996,271 bytes received 8,830 bytes 2,670,067.33 bytes/sec total size is 3,990,531 speedup is 1.00 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1196) [sender=3.1.2] As expected, the files are transferred again, because rsync couldn't set the permissions or ownership. These concepts don't exist in vfat. When using ext4, the same rsync command, the diff between the two runs shows a different story: --- log1.ext4 2019-10-08 11:49:01.687411326 -0300 +++ log2.ext4 2019-10-08 11:49:11.795461543 -0300 @@ -1,16 +1,16 @@ Number of files: 78 (reg: 77, dir: 1) -Number of created files: 77 (reg: 77) -Number of deleted files: 1 (dir: 1) -Number of regular files transferred: 77 +Number of created files: 0 +Number of deleted files: 0 +Number of regular files transferred: 0 Total file size: 3,990,531 bytes -Total transferred file size: 3,990,531 bytes -Literal data: 3,990,531 bytes +Total transferred file size: 0 bytes +Literal data: 0 bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.001 seconds File list transfer time: 0.000 seconds -Total bytes sent: 3,996,271 -Total bytes received: 1,504 +Total bytes sent: 1,598 +Total bytes received: 19 -sent 3,996,271 bytes received 1,504 bytes 7,995,550.00 bytes/sec -total size is 3,990,531 speedup is 1.00 +sent 1,598 bytes received 19 bytes 3,234.00 bytes/sec +total size is 3,990,531 speedup is 2,467.86 The rsync command I used was: rsync -v -a -r -W -p -o -g -t --delete --stats /testdata/ /media/andreas/dataext4/ /testdata was populated with a copy of all files in /usr/bin that start with "b*". I don't think the --exclusion parameters you used play a factor here. Can you please verify the target filesystem in your test case, and if rsync reported errors? And there are other scenarios, like outlined in comment #2. I guess, to summarize, we need a testcase. I showed you two cases with pendrives (vfat, and ext4), where what you describe does NOT happen. ** Changed in: rsync (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
I frequently rebuild USB keys. If memory serves me correctly, the USB device was LUKS encrypted ext4. ** Changed in: rsync (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
What's the filesystem on the target? If the target is vfat, for example, setting ownership and permissions won't work, and -W tells rsync to transfer the whole file. ** Changed in: rsync (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
The following rsync -v -a -r -W -p -o -g -t --delete --stats --exclude=".*" --exclude="/snap*" ~ /media/$USER/USB-128GB-3/ >bkp_log copies all contents of home folder each time it is run. The expectation is that only changed files will be copied and replaced on the target. This is reproducible. ** Changed in: rsync (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1845652] Re: rsync sends everything, not just changes
Hello peterzay, IF you are facing something like: https://unix.stackexchange.com/questions/85993/why-rsync-attempts-to- copy-file-that-is-already-up-to-date Then I would recommend reading the 1st answer to that question, which explains why and when rsync would try to re-sync everything again. If not, Could you please provide a better description of the problem, and, perhaps, a way on how to reproduce, step by step, the issue you are facing ? ** Changed in: rsync (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1845652 Title: rsync sends everything, not just changes To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/rsync/+bug/1845652/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
