Rsync takes long time to finish

2012-04-12 Thread vijay patel
Hi Friends, I am using rsync to copy data from Production File Server to Disaster Recovery file server. I have 100Mbps link setup between these two servers. Folder structure is very deep. It is having path like /reports/folder1/date/folder2/file.tx, where we have 1600 directories like

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Several suggestions... Add a lockfile to your cron job so it doesn't run two instances at the same time and you don't have to predict the run time. Make sure you are running rsync version 3+ on both systems. It has significant performance benefits

RE: Rsync takes long time to finish

2012-04-12 Thread Stier, Matthew
: rsync-boun...@lists.samba.org [mailto:rsync-boun...@lists.samba.org] On Behalf Of Kevin Korb Sent: Thursday, April 12, 2012 2:44 PM To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Several suggestions... Add a lockfile to your

RE: Rsync takes long time to finish

2012-04-12 Thread Stier, Matthew
- From: rsync-boun...@lists.samba.org [mailto:rsync-boun...@lists.samba.org] On Behalf Of Kevin Korb Sent: Thursday, April 12, 2012 2:44 PM To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Several suggestions... Add a lockfile

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
structure. It is running fine. Is there any option i can use with rsync to make this folder check fast? Regards, Vijay From: matthew.st...@us.fujitsu.com To: k...@sanitarium.net; rsync@lists.samba.org Subject: RE: Rsync takes long time to finish Date: Thu, 12 Apr 2012 19:29:03 +

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 And make sure both systems are running rsync v3. It indexes in parallel to the copying. On 04/12/12 16:59, Karl O. Pinc wrote: On 04/12/2012 03:28:18 PM, vijay patel wrote: Thanks friends. We are using Redhat Linux 5.8 on Production and

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
), will it make any difference? Date: Thu, 12 Apr 2012 17:05:06 -0400 From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 And make sure both systems are running rsync v3. It indexes in parallel

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
with its usage at the moment), will it make any difference? Date: Thu, 12 Apr 2012 17:05:06 -0400 From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish And make sure both systems are running rsync v3. It indexes in parallel to the copying

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
yes both servers are having rsync 3.0.6. Date: Thu, 12 Apr 2012 17:19:47 -0400 From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, you have the feature in your filesystem. Good

Re: Rsync takes long time to finish

2012-04-12 Thread Dan Stromberg
I've heard lots of good suggestions already - another thing that I've not seen mentioned is, upgrading your kernel may help. Somewhere shortly before kernel 3.0, pathname lookups got noticeably faster. You could also try an alternative filesystem like xfs. It's supposed to be pretty good at

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There was also a serious performance regression in 2.6.39. On 04/12/12 17:29, Dan Stromberg wrote: I've heard lots of good suggestions already - another thing that I've not seen mentioned is, upgrading your kernel may help. Somewhere shortly

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
We are running Kernel 2.6.18-308.1.1.el5 which is latest in RHEL 5.8 on both the server. I think i might have to explore option of using ext4. Date: Thu, 12 Apr 2012 17:31:35 -0400 From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish There was also a serious performance regression in 2.6.39. On 04/12/12 17:29, Dan Stromberg wrote: I've heard lots of good suggestions already - another thing that I've not seen mentioned

Re: Rsync takes long time to finish

2012-04-12 Thread Karl O. Pinc
On 04/12/2012 04:36:44 PM, vijay patel wrote: We are running Kernel 2.6.18-308.1.1.el5 which is latest in RHEL 5.8 on both the server. I think i might have to explore option of using ext4. Before you do anything you want to figure out why it is slow so you can solve the real problem.

Re: Rsync takes long time to finish

2012-04-12 Thread Brian K. White
folder structure. It is running fine. Is there any option i can use with rsync to make this folder check fast? Regards, Vijay From: matthew.st...@us.fujitsu.com To: k...@sanitarium.net; rsync@lists.samba.org Subject: RE: Rsync takes long time to finish Date: Thu, 12 Apr 2012 19:29:03

Re: Rsync takes long time to finish

2012-04-12 Thread Matthias Schniedermeyer
On 12.04.2012 23:59, vijay patel wrote: Hi Friends, I am using rsync to copy data from Production File Server to Disaster Recovery file server. I have 100Mbps link setup between these two servers. Folder structure is very deep. It is having path like