[Bug 12569] Missing directory errors not ignored

2018-03-14 Thread just subscribed for rsync-qa from bugzilla via rsync
https://bugzilla.samba.org/show_bug.cgi?id=12569 --- Comment #1 from Marc Krämer --- I'd like to point out that this change is a changed behavior that breaks some scripts depending on this behavior. Can you consider to change it to the original behavior, or add a new

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Remi Gauvin via rsync
On 2018-03-14 10:29 AM, Kevin Korb via rsync wrote: > It would still be downloading the whole file only to write out a new > one. Rsync writes out a whole new file because it assumes it is doing > so locally. The source of that new file can be a combination of parts > of the existing file and

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Kevin Korb via rsync
It would still be downloading the whole file only to write out a new one. Rsync writes out a whole new file because it assumes it is doing so locally. The source of that new file can be a combination of parts of the existing file and parts of the remote file but it still writes the whole thing.

Re: Comparison of huge files ineffective?

2018-03-14 Thread Kevin Korb via rsync
Do not use --checksum. It has an extremely limited use case. Normally it is much slower than simply re-copying everything. --checksum means checksum every file on both ends (even files that only exist on one end) before doing anything else even if doing so causes a timeout failure. --checksum

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Remi Gauvin via rsync
On 2018-03-14 09:41 AM, Lentes, Bernd via rsync wrote: > > also when the target is a cifs share, it's still considered as local ? > Is there something i can do to get the diff algorithm used ? > Copying via ssh to the cifs server is unfortunately not possible. > If you really wanted to, you

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Remi Gauvin via rsync
On 2018-03-14 10:07 AM, Kevin Korb via rsync wrote: > --no-whole-file would only make it even worse. It would have to read > the remote file over the network in order to do the diff then it would > write the whole file over the network anyway (--inplace would help a > little). Local copies force

Re: Comparison of huge files ineffective?

2018-03-14 Thread Heiko Schlittermann via rsync
Kevin Korb via rsync (Mi 14 Mär 2018 14:52:55 CET): > Your observation would be right if you are using --checksum which you > shouldn't be. Otherwise, unless you are using --whole-file rsync will > use its differential algorithm to compare the files. If you are using >

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Ben RUBSON via rsync
Yes you're right, rsync would update only a few parts of the file, but network usage would be even worst. The only solution would finally be to have rsync on the target system. Ben On 14 Mar, Kevin Korb via rsync wrote: --no-whole-file would only make it even worse. It would have to read

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Kevin Korb via rsync
--no-whole-file would only make it even worse. It would have to read the remote file over the network in order to do the diff then it would write the whole file over the network anyway (--inplace would help a little). Local copies force --whole-file for a good reason. On 03/14/2018 10:05 AM,

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Ben RUBSON via rsync
On 14 Mar 2018, Lentes, Bernd via rsync wrote: - On Mar 14, 2018, at 2:19 PM, Ben RUBSON ben.rub...@gmail.com wrote: On 14 Mar 2018, Lentes, Bernd via rsync wrote: I would now expect a rsync from the snap would transfer just some megay bytes to the file from the day before. But it

Re: Comparison of huge files ineffective?

2018-03-14 Thread Kevin Korb via rsync
Your observation would be right if you are using --checksum which you shouldn't be. Otherwise, unless you are using --whole-file rsync will use its differential algorithm to compare the files. If you are using --progress you will see it step through the file at a faster speed than the network is

Comparison of huge files ineffective?

2018-03-14 Thread Heiko Schlittermann via rsync
Hi, how does rsync work if it compares two very huge files on two distinct hosts (rsync uses a networked connection, via SSH)? Some observation seems to indicate, that rsync first reads (and checksums?) the remote (destination) side, then, if finished, it reads (and checksums?) the local

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Lentes, Bernd via rsync
no backup - no mercy - On Mar 14, 2018, at 2:19 PM, Ben RUBSON ben.rub...@gmail.com wrote: > On 14 Mar 2018, Lentes, Bernd via rsync wrote: > >> I would now expect a rsync from the snap would transfer just some megay >> bytes to the file from the day before. >> But it doesn't: >> >>

Re: rsync of a reflink from OCFS2

2018-03-14 Thread Ben RUBSON via rsync
On 14 Mar 2018, Lentes, Bernd via rsync wrote: I would now expect a rsync from the snap would transfer just some megay bytes to the file from the day before. But it doesn't: ha-idg-1:/cluster/guests/servers_alive # time rsync -av --stats sa.raw.snap

rsync of a reflink from OCFS2

2018-03-14 Thread Lentes, Bernd via rsync
Hi, i have some virtual machines running on logical volumes formatted with OCFS2. I'd like to snapshot the running guests to backup them easily afterwards. The files of the guests are big (100 - 300GB), but the content changes only slowly. So i thought that rsync would be a great benefit