Re: rsync speedup - how ?

2009-08-14 Thread Wayne Davison
On Thu, Aug 06, 2009 at 08:15:39PM +0200, devz...@web.de wrote: i read that rsync would be not very efficient with ultra-large files (i`m syncing files with up to 80gb size) Things to try: - Be sure you're using rsync 3.x, as it has a better hash algorithm for the large numbers of checksum

Re: rsync speedup - how ?

2009-08-08 Thread devzero
I really don't think it's a good idea to sync large data files in use, which is modified frequently, e.g. SQL database, VMware image file. As rsync do NOT have the algorithm to keep those frequently modified data file sync with the source file. And this will course data file corrupted.

Re: rsync speedup - how ?

2009-08-08 Thread Carlos Carvalho
devz...@web.de (devz...@web.de) wrote on 6 August 2009 20:15: i`m using rsync to sync large virtual machine files from one esx server to another. the speed is reasonable, but i guess it`s not the optimum - at least i donŽt know where the bottleneck is. That's vague and subjective, so

Re: rsync speedup - how ?

2009-08-07 Thread devzero
files, how can i calculate the risk of 2 files being NOT perfectly in sync after rsync run ? i assume there IS a risk, just as like there is a risk that 2 files may have the same md5 checksum by chance regards roland List: rsync Subject:Re: rsync speedup - how ? From: Jon

Re: rsync speedup - how ?

2009-08-07 Thread devzero
devz...@web.de wrote: so, instead of 500M i would transfer 100GB over the network. that`s no option. I don't see how you came up with such numbers. If files change completely then I don't see why you would transfer more (or less) over the network. The difference that I'm thinking of is

Re: rsync speedup - how ?

2009-08-07 Thread Daniel.Li
On Fri, 2009-08-07 at 18:44 +0200, devz...@web.de wrote: devz...@web.de wrote: so, instead of 500M i would transfer 100GB over the network. that`s no option. I don't see how you came up with such numbers. If files change completely then I don't see why you would transfer more (or

rsync speedup - how ?

2009-08-06 Thread devzero
Hello, i`m using rsync to sync large virtual machine files from one esx server to another. rsync is running inside the so called esx console which is basically a specially crafted linux vm with some restrictions. the speed is reasonable, but i guess it`s not the optimum - at least i donŽt

Re: rsync speedup - how ?

2009-08-06 Thread Jon Forrest
One way I've been trying to speedup rsync may not apply in every situation. In my situation when files change, they usually change completely. This is especially true for large files. So, the rsync algorithm does me no good. So, I've been using the W flag (e.g. rsync -avzW) to turn this off. I