Re: Need hint for my question regarding the working of rsync.

2013-11-13 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 OK, in the case of using v3 with --link-dest and not --checksum most of the initial activity on the sender would be doing calls to stat() to index what is there. The receiving side would be doing 2x the stat() calls (you have 2 - --link-dest dirs for

Re: Need hint for my question regarding the working of rsync.

2013-11-13 Thread Karl O. Pinc
On 11/13/2013 12:03:21 PM, Kevin Korb wrote: OK, in the case of using v3 with --link-dest and not --checksum most of the initial activity on the sender would be doing calls to stat() to index what is there. The receiving side would be doing 2x the stat() calls (you have 2 --link-dest dirs

Re: Need hint for my question regarding the working of rsync.

2013-11-13 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 To an extent it is serially. The sender tells the receiver what it needs to stat(). However, thanks to incremental indexing it will parallelize but the receiver will not go beyond what the receiver has sent. Read the --recursive section of the man

Re: Need hint for my question regarding the working of rsync.

2013-11-13 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Is there a hard links limit? I have been in the 70-80 million range on ext4 without a problem (other than performance which is why I switched to ZFS for that use case). On 11/13/13 13:59, Karl O. Pinc wrote: On 11/13/2013 12:03:21 PM, Kevin Korb

Re: Need hint for my question regarding the working of rsync.

2013-11-13 Thread Karl O. Pinc
necessarily 11/13/2013 01:04:29 PM, Kevin Korb wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Is there a hard links limit? I have been in the 70-80 million range on ext4 without a problem (other than performance which is why I switched to ZFS for that use case). It's a per-file

Re: Need hint for my question regarding the working of rsync.

2013-11-13 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I wasn't saying I had millions of links of the same file. It was hundreds of link-dest backups some of which contained a few but not all that many links. I wasn't doing a link-dest backup of something link heavy. On 11/13/13 15:01, Karl O. Pinc

Re: Need hint for my question regarding the working of rsync.

2013-11-12 Thread Wayne Davison
On Mon, Nov 11, 2013 at 9:04 PM, Patrick Pollen patrickpoll...@gmail.comwrote: For example suppose I have a 2 GB file, so after generating checksum on receiver side, does the receiver sends all the generated checksum to the sender at once? Yes, the receiver sends all the checksums that it

Re: Need hint for my question regarding the working of rsync.

2013-11-12 Thread Karl O. Pinc
On 11/12/2013 03:50:20 PM, Wayne Davison wrote Yes, the receiver sends all the checksums that it generates at once For really big files it would be interesting to amend this rule to one where the sending side waits only long enough for a certain number of checksums to arrive before it

Re: Need hint for my question regarding the working of rsync.

2013-11-12 Thread Karl O. Pinc
On 11/12/2013 04:13:01 PM, Karl O. Pinc wrote: On 11/12/2013 03:50:20 PM, Wayne Davison wrote Yes, the receiver sends all the checksums that it generates at once For really big files it would be interesting to amend this rule to one where the sending side waits only long enough for a

Re: Need hint for my question regarding the working of rsync.

2013-11-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 First, are you talking about --checksum checksums or the hashing of files that are different on both ends so that only the differences need to be transferred? You seem to be talking about the latter while describing the performance of the former. If

Re: Need hint for my question regarding the working of rsync.

2013-11-12 Thread Patrick Pollen
These are both a weak and a strong checksum for each chunk of the file from start to finish. So lets take an example. If a file were 7 bytes and the logical block size for rsync by default being 700, it would send : 1) 4 Bytes x 7/700 = 400 Bytes 2) 16 Bytes x 7/700 = 1600 Bytes

Need hint for my question regarding the working of rsync.

2013-11-11 Thread Patrick Pollen
Hello, Since English is my second language, forgive me for any typing errors. I have been studying rsync for my academic project. I learned quite a lot but I need little help. My question is, does rsync sends all checksum of a file at once or in batches. For example suppose I have a 2 GB file, so