Re: Problem with Rsync or Users will be Users?

2007-02-22 Thread Stuart Halliday
Sorry but that didn't a lot of sense to me. How come it's a OS error? If the file is on the server surely then the OS allowed it? mktemp? What is that? You might be able to work around the problem by copying more deeply in the hierarchy (and avoid the errors higher up by excluding the deep

Re: Problem with Rsync or Users will be Users?

2007-02-22 Thread Stuart Halliday
Various web sites says this about NTFS files. Individual file names in NTFS are limited to 255 characters; full paths are limited to 32,767 characters This filename is a mere 191 long Path is a lot less than 32,767... :-) So how come it is an OS error when Rsync tries to copy it?

RE: Problem with Rsync or Users will be Users?

2007-02-22 Thread Stuart Halliday
Path length limitiation comes from the underlying OS, cygwin in this case. It has a max path length of 260 chars, if I remember correctly. Wow. That's quite a limitation. I didn't realise. Thanks Tevfik for that enlightenment. :-) This email is the property of ECS Technology Ltd. This

RE: Problem with Rsync or Users will be Users?

2007-02-22 Thread Tevfik Karagülle
Path length limitiation comes from the underlying OS, cygwin in this case. It has a max path length of 260 chars, if I remember correctly. Tev -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Stuart Halliday Sent: Thursday, February 22, 2007 10:34

Using rsync to sync two live servers

2007-02-22 Thread Chris Nighswonger
Hello, I have two production servers which house large drive arrays storing digitized music for our webradio. They are used in a 'failover' configuration so that if one goes down the other remains available. New music is added all of the time to this library and it is a pain to have to add it to

Re: Using rsync to sync two live servers

2007-02-22 Thread Paul Slootman
On Thu 22 Feb 2007, Chris Nighswonger wrote: I have two production servers which house large drive arrays storing digitized music for our webradio. They are used in a 'failover' configuration so that if one goes down the other remains available. New music is added all of the time to this

Re: Using rsync to sync two live servers

2007-02-22 Thread Chris Nighswonger
On 2/22/07, Paul Slootman [EMAIL PROTECTED] wrote: On Thu 22 Feb 2007, Chris Nighswonger wrote: I have two production servers which house large drive arrays storing digitized music for our webradio. They are used in a 'failover' configuration so that if one goes down the other remains

RE: Using rsync to sync two live servers

2007-02-22 Thread Tevfik Karagülle
You can use shadow copy functionality on windows 2003 systems to address open file issue. Check http://www.itefix.no/phpws/index.php?module=faqFAQ_op=viewFAQ_id=81 for a possible solution. Tev -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Chris

Re: feature request, hardlink progress......

2007-02-22 Thread George Georgalis
On Mon, Jan 15, 2007 at 01:39:53AM -0500, George Georgalis wrote: On Sat, Jan 13, 2007 at 08:43:55PM -0800, Wayne Davison wrote: On Sat, Jan 13, 2007 at 10:25:42PM -0500, George Georgalis wrote: hours left. Would it be straightforward to include progress when creating hardlinks? Please upgrade

Verbose but only problem files

2007-02-22 Thread Dave Markham
I have tried a couple of things to get what i want but am unsure how to get there. I would like in the log i use for rsync to only output files it has trouble getting. e.g if permission is denied or file changed during read something like that. If i use the v option for verbose it lists every

Re: feature request, hardlink progress......

2007-02-22 Thread Paul Slootman
On Thu 22 Feb 2007, George Georgalis wrote: Please upgrade -- hard-link handling is much improved in newer versions. Thanks, turns out there are closer to 500,000 files and 89 snapshots (@ ~90% of the files). The process was to install [EMAIL PROTECTED]:/root $ rsync --numeric-ids -avHP

Number of files to be transferred?

2007-02-22 Thread Eric Busto
I'm writing a wrapper around a large number of rsyncs that need to happen, and it would be great if I could parse the output of a running rsync and find out how far along it is. To do this, I would need to know how many files the rsync is going to transfer. If I run 'rsync -avz --progress', it

RE: Number of files to be transferred?

2007-02-22 Thread Julian Pace Ross
If you perform a dry run (-n) with --stats, before the actual run (will only add an overhead of typically a few seconds. a few minutes at worst depending on size) you can get an idea of the number of files to be transferred and total transferred file size, although the latter is (as adequately

RE: Number of files to be transferred?

2007-02-22 Thread Eric Busto
Unfortunately, the data I am transferring is many hundreds of gigabytes, mounted on NFS shares that reside on NetApp filers, and is being transferred to locations on the other side of the globe. A dry run can itself take an hour or more, quite easily. Since rsync is already doing a pass through

RE: Number of files to be transferred?

2007-02-22 Thread Julian Pace Ross
Oh. different story then. I'm sure that value that is printed with --stats is in the filelist structure. maybe it could be output on stdout after building file list. with a small tweak. Maybe Wayne/Matt would know where this variable is? _ From: Eric Busto [mailto:[EMAIL

RE: Number of files to be transferred?

2007-02-22 Thread Eric Busto
Looking at it now. :) Looks like I just need to print out file_list.count. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Julian Pace Ross Sent: Thursday, February 22, 2007 12:59 PM To: rsync@lists.samba.org

Re: Number of files to be transferred?

2007-02-22 Thread Matt McCutchen
On 2/22/07, Eric Busto [EMAIL PROTECTED] wrote: Looks like I just need to print out file_list.count. No, that gives the total number of files rsync is considering, not how many it is going to transfer. There is no way to determine how many files rsync is going to transfer short of a dry run.

RE: Number of files to be transferred?

2007-02-22 Thread Eric Busto
Yeah, I came to that conclusion unfortunately after adding in the printf. :/ Thanks! -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt McCutchen Sent: Thursday, February 22, 2007 4:01 PM To: Eric Busto Cc: Julian Pace Ross; rsync@lists.samba.org

Re: feature request, hardlink progress......

2007-02-22 Thread Wayne Davison
On Thu, Feb 22, 2007 at 10:12:17AM -0500, George Georgalis wrote: I added more memory, same problem. any recommendations to get around this? will the old version of rsync use less memory to make all the hardlinks? Newer versions should use less memory than old. Perhaps you have a ulimit set

Re: feature request, hardlink progress......

2007-02-22 Thread Matt McCutchen
On 2/22/07, Paul Slootman [EMAIL PROTECTED] wrote: I don't know what rsync will do with a link-dest that is the same as the target :-) Such a link-dest will have no effect: rsync looks in the link-dest only for source files that don't have corresponding files in the destination, and when the

Re: Verbose but only problem files

2007-02-22 Thread Matt McCutchen
On 2/22/07, Dave Markham [EMAIL PROTECTED] wrote: I would like in the log i use for rsync to only output files it has trouble getting. e.g if permission is denied or file changed during read something like that. You could leave out the -v option, because rsync outputs an error message for

Re: feature request, hardlink progress......

2007-02-22 Thread George Georgalis
On Thu, Feb 22, 2007 at 05:09:06PM +0100, Paul Slootman wrote: On Thu 22 Feb 2007, George Georgalis wrote: Please upgrade -- hard-link handling is much improved in newer versions. Thanks, turns out there are closer to 500,000 files and 89 snapshots (@ ~90% of the files). The process was to