Ok, did the tests without --inplace, it saves the BW, but completly nullify the benefits of ZFS snapshots.
This is disturbing. Le 2015/02/03 21:54, Guillaume Friloux a écrit : > Hello, thanks for answering. > > I have to use --inplace to limit writes on the ZFS dataset, otherwise > each snapshot > will use the total file size instead of only the diff. > In my env, i dont do local copies, i do send over SSH on a BSD host. > > I will redo all the tests without --inplace to see if it does better > for > rsync > (but wont be a real solution for my ZFS vol). > > I am using /dev/urandom only because i wanted a simple test case, but > the problem > occurs with real files, like outlook PST files, PPT files and so on. > > I intentionally not used gzip because gzip itself also produce some > problems > here with the files, and i do use --rsyncable, or tell rsyncrypto to > use > gzip (encounter the problem with both methods). > > Youre saying rsyncrypto uses gzip, but i did give --gzip=nullgzip that > is > a bash script calling cat, and so, not compression should be done, or > is > rsyncrypto adding compression over what gzip did ? > > > Le 2015/02/03 20:54, Shachar Shemesh a écrit : >> rsyncrypto compresses as part of the encryption. You obviously did not >> notice this, as you were using /dev/random as your source, and hence >> producing uncompressible files. This is also the reason (at least part >> of it) that the encrypted files were not the same size. >> >> On 03/02/15 10:34, Guillaume Friloux wrote: >> >>> The only thing i can see is that between file1.iso.enc and >>> file2.iso.enc, >>> the filesize dropped a little, and between file2.iso.enc and >>> file3.iso.enc it is higher, >>> but i have no idea if this can be related... >> You are using rsync with --inplace. In that mode, rsync cannot reuse >> blocks that were already overwritten by the destination file. When the >> new file is bigger than the old one, you are overwriting the data you >> would reuse while transferring, severely limiting rsync's ability to >> optimize your transfer. If you remove --inplace, you will see that >> rsync has no problem optimizing your encrypted files, no matter the >> size changes. >> >> You have not asked your gzip question, but I am guessing it is either >> the same issue there, or you forgot to pass it the --rsyncable flag. >> >> Shachar ------------------------------------------------------------------------------ Dive into the World of Parallel Programming. The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Rsyncrypto-devel mailing list Rsyncrypto-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/rsyncrypto-devel
