forced copying of non readable file
Hi Tridge, Sorry for mailing directly cause i have'nt receive any responses from my previous questions from your mailing list. Im' just new with rsync, and trying to evaluate it before fully using it for implementation for mirrroring our mail server. Some of my question from my previous mail are now solved. the last question that i left here before fully implement my mirror is regarding the copying of non readable file in the server. You have an option ignore nonreadable that used to bypass unreadable file an dnt show any error message to that. But what if that file is important and should be copied for mirroring . In short can rsync be able to forced nonreadable file or folder to be copied in a remote mirror server or a backup server? I've been encountering this such problem in mirrong using rsync their are some important files and folder that cannot be transferred and issueing an error Permmission Denied. Some files that cannot be read are shadow,passwd, .bash_profile and alot more which shoulb be also included for backup. Regards, Michael P. Carel Systems Administrator TEAM PACIFIC CORPORATION FTI Taguig Metro, Manila. PHIL. Tel. Nos. (02) 838-50-05 local 363 (02) 838-84-38 direct
Re: rsync exclude/include [FIDUCIA virengepr?ft - ohne Gew?hr, da? allebekannten Viren und deren Varianten erkannt wurden.]
Tomasz Ciolek [EMAIL PROTECTED] on 13.11.2001 06:29:35 An: Thomas Schweikle [EMAIL PROTECTED] Kopie: Thema:Re: rsync exclude/include [FIDUCIA virengepr?ft - ohne Gew?hr, da? alle bekannten Viren und deren Varianten erkannt wurden.] so perhaps dropping the leading slash after the - or + will help? No. Tryied that and others. Matching just doesn't work. Same results as before. Bad. Any debug mode for matching telling exactly what rsync is dooing?
Re: rsync exclude/include [FIDUCIA virengeprüft - ohne Gewähr, daß alle bekannten Viren und deren Varianten erkannt wurden.]
On Tue, Nov 13, 2001 at 01:44:45AM +0100, Thomas Schweikle wrote: Hi again! Tomasz Ciolek [EMAIL PROTECTED] on 13.11.2001 01:21:27 An: Thomas Schweikle [EMAIL PROTECTED] Kopie: Thema:Re: rsync exclude/include [FIDUCIA virengeprüft - ohne Gewähr, daß alle bekannten Viren und deren Varianten erkannt wurden.] Try dumoping the whole file list into a text file, prefix each one with minus for exclude and a + for include. Then run rsync like this: rsync -avz --exclude-from=exclude.list ftp3.sourceforge.net::/netbsd/iso/ ./iso/ have a look at the rsync man page for more information on the exact syntax of the unified include/exclude stuff Regards. Done that. Result: rsync include/exclude matching is definitively wrong. If ... matching is done using shell filename matching ... this file should work: - /* + /iso/1.5.*/i386* but doesn't. Looks like * is matched, but does not lead to a true matched against /iso/1.5.1/i386cd.iso. Here's what you neeed: + /iso + /iso/1.5.* + /iso/1.5.*/i386* - /iso/1.5.*/* - /iso/* - /* In order to understand why you need to realize that the rsync algorithm recurses down into each directory and applies each pattern in order and stops when it finds a match, so if you don't explicitly include the parent directories they get excluded by the exclude wildcard and rsync never descends down into the directory. Note that /* only excludes things at the top level, that's why I had to explicitly exclude things at the next two levels too. A simplification is this: + /iso + /iso/1.5.* + /iso/1.5.*/i386* - * because if you don't have the slash preceding the exclude * it applies at all levels. Any way to debug this, making sure how matches are really done? The most recent snapshots of the development version of rsync will give you more helpful information if you use -vv, but it's not yet in any released version. The patch is at http://rsync.samba.org/cgi-bin/cvsweb/rsync/exclude.c.diff?r1=1.32r2=1.33 - Dave Dykstra
Re: rsync exclude/include
On Tue, Nov 13, 2001 at 10:00:59AM -0600, Dave Dykstra wrote: ... A simplification is this: + /iso + /iso/1.5.* + /iso/1.5.*/i386* - * because if you don't have the slash preceding the exclude * it applies at all levels. Wait, if i386* matches directories and not just files you'll need + /iso + /iso/1.5.* + /iso/1.5.*/i386** - * so that it will include all files below the i386* directories. Any double asterisk in a pattern tells the matching routine to go beyond slashes. - Dave
Re: rsync exclude/include
On Tue, 13 Nov 2001, Thomas Schweikle wrote: I am calling rsync using rsync -avz --include-from=include --exclude-from=exclude ftp3.sourceforge.net::/netbsd/iso iso/ Looks like you didn't copy that command exactly, because rsync would fail with a syntax error due to the '/' before the netbsd module name. Also, you're creating an iso dir inside your local iso dir, which is probably not what you want. With the include/exclude file Dave gave you, you'd need to run this command (changing iso/ into .): rsync -avz --include-from=foo ftp3.sourceforge.net::netbsd/iso . However, I'd suggest one a little simpler: add a trailing slash to the root directory you're requesting and you can leave off the references to it (and put the data wherever you like, even if the directory isn't named iso). You would run this command: rsync -avz --include-from=foo ftp3.sourceforge.net::netbsd/iso/ myiso And put this into foo: + /1.5.*/ + /1.5.*/i386* - * You'll note I also used a trailing slash for the directory include since I don't want any files that match to be included (there are none here, but it's a good general principle). ..wayne..
using rsync to backup windows workstations
I have 5 windows workstations and 1 computer running freebsd 4.4 I want to install rsync on the freebsd computer so I can use it to backup some files/documents on my windows workstations. I want rsync to run automatically every 6pm mon-sat. I wanna ask if there is someone in this list who can give me guidelines on how i can do this. And another question I have is do I need to install samba? i know this is very idiot question... I have been reading the manuals for 3 days already and nothing seems to be working. I need help very badly. :( Needless to say, I am very new to Unix/Linux. __ Do You Yahoo!? Find the one for you at Yahoo! Personals http://personals.yahoo.com
Re: Ptoblem for update 2.4.1 - 2.4.6
On Tue, Nov 13, 2001 at 12:41:10PM +0300, Eugene Frolenkov wrote: After installing nevest version of sync 2.4.6 from 2.4.1, sync --daemon wrote to logfile: 2001/11/13 12:27:46 [12261] rsyncd version 2.4.6 starting 2001/11/13 12:27:46 [12261] bind failed on port 873 2001/11/13 12:27:46 [12261] transfer interrupted (code 10) at socket.c(233) sync 2.4.1 worked normally... Chances are another process is still holding port 873 open, or you're not running as root. My question: Where i can find WATHSNEW or CHANGES documentations any versions of rsync? Unfortunately there's no long-running file like that for rsync. I see for the next release the new maintainer has added a file called NEWS so maybe it will be coming. Required rsync-2.4.6 newest version of bind? Sys: RedHat Knl: 2.2.14 bind: 8.2.3 It has nothing do with the program called bind, it is the system call bind(2). - Dave Dykstra
Re: Ptoblem for update 2.4.1 - 2.4.6
The bind mentioned in the second line refers not to Berkely Internet Name Domain, but to socket binding. I suspect that your old daemon is still running. ps ax |grep rsync will show it if that's it. Alternately, if you were running the daemon from inetd, you'll just need to make rsync available at the location pointed to by /etc/inetd.conf... inetd will be holding port 873, so you won't be able to start --daemon without reconfiguring inetd. One other possibility. port 873 1025, so if you're not root, you can't bind to that port. List: did i miss anything? Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Eugene Frolenkov [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/13/2001 02:41 AM Please respond to abyhan To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:Ptoblem for update 2.4.1 - 2.4.6 Classification: After installing nevest version of sync 2.4.6 from 2.4.1, sync --daemon wrote to logfile: 2001/11/13 12:27:46 [12261] rsyncd version 2.4.6 starting 2001/11/13 12:27:46 [12261] bind failed on port 873 2001/11/13 12:27:46 [12261] transfer interrupted (code 10) at socket.c(233) sync 2.4.1 worked normally... My question: Where i can find WATHSNEW or CHANGES documentations any versions of rsync? Required rsync-2.4.6 newest version of bind? Sys: RedHat Knl: 2.2.14 bind: 8.2.3 -- W.B.R., Evgeny Frolenkov
times difference causes write
Using rsync-2.4.6: Is a times difference supposed to cause a write? Also -t vs -I makes no difference. Below shows the problem, I think: [dmahurin@pc16 /tmp]$ mkdir x y [dmahurin@pc16 /tmp]$ cp /bin/ls x [dmahurin@pc16 /tmp]$ ls -l x/ls -rwxr-xr-x1 dmahurin users 43024 Nov 13 12:46 x/ls [dmahurin@pc16 /tmp]$ rsync -vrtW x/ y building file list ... done ./ ls ./ wrote 43112 bytes read 36 bytes 86296.00 bytes/sec total size is 43024 speedup is 1.00 [dmahurin@pc16 /tmp]$ touch x/ls [dmahurin@pc16 /tmp]$ rsync -vrtW x/ y building file list ... done ls ./ wrote 43116 bytes read 36 bytes 86304.00 bytes/sec total size is 43024 speedup is 1.00 [dmahurin@pc16 /tmp]$ touch x/ls [dmahurin@pc16 /tmp]$ rsync -vrIW x/ y building file list ... done ls wrote 43116 bytes read 36 bytes 86304.00 bytes/sec total size is 43024 speedup is 1.00
Re: using rsync to backup windows workstations
For some help in getting started, see: http://optics.ph.unimelb.edu.au/help/rsync/rsync_user.html Unfortunately an irritating person at Redhat/cygwin insisted I remove my windoze binaries because I wasn't able to provide full source for cygwin and rsync. So here are some comments I send people that query me: a) rsync binaries for win32/cygwin are now available from the binaries section of the rsync web pages, maintained by someone at redhat I think b) the version attached is so old that you really shouldn't attempt to use it. Get the latest CVS version and add Wayne Davison's patches (see mailing list archives), in particular for use with Windoze systems, then compile it yourself with cygwin c) it's easy to compile under cygwin, and cygwin is easy to install, so I strongly recommend doing that d) also install openssh which compiles cleanly under cygwin and is much more reliable than the ssh that is attached here Sorry I haven't had time to prepare a new release. One day On Tue, 13 Nov 2001, amiel ong wrote: I have 5 windows workstations and 1 computer running freebsd 4.4 I want to install rsync on the freebsd computer so I can use it to backup some files/documents on my windows workstations. I want rsync to run automatically every 6pm mon-sat. I wanna ask if there is someone in this list who can give me guidelines on how i can do this. And another question I have is do I need to install samba? i know this is very idiot question... I have been reading the manuals for 3 days already and nothing seems to be working. I need help very badly. :( Needless to say, I am very new to Unix/Linux. __ Do You Yahoo!? Find the one for you at Yahoo! Personals http://personals.yahoo.com -- Robert Scholten Tel: +61 3 8344 5457 Mob: 0412 834 196 School of Physics Fax: +61 3 9347 4783 University of Melbourne email: [EMAIL PROTECTED] Victoria 3010 AUSTRALIA http://www.ph.unimelb.edu.au/~scholten
Re: times difference causes write
In the example you give, yes, a time difference causes a write. You are using the -W (--whole-file) option, which directs rsync to simply send the file, in its entirety, if there is a discrepency in mtime (ctime too?), or extent. It is used for situations where file access is slow enough that trying to do an incremental update would take more time/resources than simply sending the file... primary example being nfs-mounted filesystems. If you have fast dasd/slow network, you should probably just drop the -W. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Don Mahurin [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/13/2001 01:52 PM To: [EMAIL PROTECTED] cc: (bcc: Tim Conway/LMT/SC/PHILIPS) Subject:times difference causes write Classification: Using rsync-2.4.6: Is a times difference supposed to cause a write? Also -t vs -I makes no difference. Below shows the problem, I think: [dmahurin@pc16 /tmp]$ mkdir x y [dmahurin@pc16 /tmp]$ cp /bin/ls x [dmahurin@pc16 /tmp]$ ls -l x/ls -rwxr-xr-x1 dmahurin users 43024 Nov 13 12:46 x/ls [dmahurin@pc16 /tmp]$ rsync -vrtW x/ y building file list ... done ./ ls ./ wrote 43112 bytes read 36 bytes 86296.00 bytes/sec total size is 43024 speedup is 1.00 [dmahurin@pc16 /tmp]$ touch x/ls [dmahurin@pc16 /tmp]$ rsync -vrtW x/ y building file list ... done ls ./ wrote 43116 bytes read 36 bytes 86304.00 bytes/sec total size is 43024 speedup is 1.00 [dmahurin@pc16 /tmp]$ touch x/ls [dmahurin@pc16 /tmp]$ rsync -vrIW x/ y building file list ... done ls wrote 43116 bytes read 36 bytes 86304.00 bytes/sec total size is 43024 speedup is 1.00
Re: times difference causes write
My first problem is that I am writing to compact flash, so I want the minimal number of writes. My second problem is that the flash is of limited size, so I need some sort of patch rsync that does not keep the old file before writing the new one. My patch now just unlinks the file ahead, and implies -W. So my wish was that a time discrepancy would lead to a checksum, where the files would match. This is not the case, however, as you say. So for now, I must use -c. It's slow, but I know that I get the minimum number of writes. -don [EMAIL PROTECTED] wrote: In the example you give, yes, a time difference causes a write. You are using the -W (--whole-file) option, which directs rsync to simply send the file, in its entirety, if there is a discrepency in mtime (ctime too?), or extent. It is used for situations where file access is slow enough that trying to do an incremental update would take more time/resources than simply sending the file... primary example being nfs-mounted filesystems. If you have fast dasd/slow network, you should probably just drop the -W.
Re: times difference causes write
-W and -C are actually kind of opposite effects... -W means never checksum... if it's different in length or time, send it. -c means ALWAYS checksum, even if time and length are identical. use neither. the default behaviour is to compare date/length, and if they are different, do the rolling checksum. This is done in chunks, then compares the sums, and sends the different blocks, with instructions on where to insert them. If the difference was simply timestamp, and in fact, the data is unchanged, the rolling checksum will sort of say never mind, and leave the file alone. Unfortunately for your situation, it builds tne new file next to the existing file, no matter what you do, then deletes the existing file and renames the new file to the old name. Every file done by rsync involves a full write. You're going to put a lot of cycles on your flash. I'm thinking rsync might not be the best solution for what you're doing. Flash is not suited to frequently-updated data. Tim Conway [EMAIL PROTECTED] 303.682.4917 Philips Semiconductor - Longmont TC 1880 Industrial Circle, Suite D Longmont, CO 80501 Available via SameTime Connect within Philips, n9hmg on AIM perl -e 'print pack(, 19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), .\n ' There are some who call me Tim? Don Mahurin [EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 11/13/2001 02:36 PM To: Tim Conway/LMT/SC/PHILIPS@AMEC cc: [EMAIL PROTECTED] Subject:Re: times difference causes write Classification: My first problem is that I am writing to compact flash, so I want the minimal number of writes. My second problem is that the flash is of limited size, so I need some sort of patch rsync that does not keep the old file before writing the new one. My patch now just unlinks the file ahead, and implies -W. So my wish was that a time discrepancy would lead to a checksum, where the files would match. This is not the case, however, as you say. So for now, I must use -c. It's slow, but I know that I get the minimum number of writes. -don [EMAIL PROTECTED] wrote: In the example you give, yes, a time difference causes a write. You are using the -W (--whole-file) option, which directs rsync to simply send the file, in its entirety, if there is a discrepency in mtime (ctime too?), or extent. It is used for situations where file access is slow enough that trying to do an incremental update would take more time/resources than simply sending the file... primary example being nfs-mounted filesystems. If you have fast dasd/slow network, you should probably just drop the -W.
RE: times difference causes write
Don Mahurin [[EMAIL PROTECTED]] writes: My second problem is that the flash is of limited size, so I need some sort of patch rsync that does not keep the old file before writing the new one. My patch now just unlinks the file ahead, and implies -W. Sounds reasonable as long as you force the -W. So my wish was that a time discrepancy would lead to a checksum, where the files would match. This is not the case, however, as you say. At least not with -W. In most cases, the time discrepancy would then cause rsync to try to synchronize the file, and during its protocol processing it would determine that it didn't need to send anything, thus the only end result would be adjusting the remote timestamp to match the source. But this requires access to the original source file, so your prior patch (and forcing -W) defeats this as a side effect. So for now, I must use -c. It's slow, but I know that I get the minimum number of writes. It definitely sounds like the best match for you. Although -c tends to be used more for cases where files may differ although they appear the same (timestamp/size) than vice versa, it will serve that purpose as well at the expense of some additional I/O and computation. Presumably you could modify your patch so that -c (or some new option) only invoked the checksum if the timestamp differed, since I don't think there's any suitable equivalent currently in rsync. -- David /---\ \ David Bolen\ E-mail: [EMAIL PROTECTED] / | FitLinxx, Inc.\ Phone: (203) 708-5192| / 860 Canal Street, Stamford, CT 06902 \ Fax: (203) 316-5150 \ \---/
Re: using rsync to backup windows workstations
As officially I'm the mantainer of cygwin's rsync package I feel I must add something.. 0=) a) rsync binaries for win32/cygwin are now available from the binaries section of the rsync web pages, maintained by someone at redhat I think Maintained by me, I don't work for redhat, and sadly I have a bit too little time to maintain it as I should... b) the version attached is so old that you really shouldn't attempt to use it. Get the latest CVS version and add Wayne Davison's patches (see mailing list archives), in particular for use with Windoze systems, then compile it yourself with cygwin ...but not so little not to compile it with Wayne's patches (actual patch is the latest anti-hang available at the date of the release of cygwin's rsync-2.4.6-2 package) ;-) If there's some patch that I missed (I read this mailing list since when I began to mantain that package, but reading only headers of most messages maybe I missed something...) please point it to me and I'll release a new verison shortly =) Please note that I'm not an expert programmer of rsync, just a person that likes and uses it (and wants maybe to become such an expert, btw): in cygwin one package can't exist in the distribution if there's not a mantainer... so I hope that the effort I put in it is not wasted, even if it's maybe not enough... [I've got a bit of guild for that eheh] c) it's easy to compile under cygwin, and cygwin is easy to install, so I strongly recommend doing that That's true, it compiles out-of-the box. The main problem is that in winsocks if the socket is closed a RST is sent, not flushing the cache. This creates many errors at the end of trasnfer, especially in the daemon mode (which is not addressed by the patch I used, I bet). d) also install openssh which compiles cleanly under cygwin and is much more reliable than the ssh that is attached here Compiles cleanly and is available as a binary. If anyone wants to ask some more question.. feel free, even if I don't know nothing special about it, as I said I had not had the time to completely study and understand rsync source... C ya, Lapo -- Lapo Luchini [EMAIL PROTECTED] (PGP X.509 keys available) http://www.lapo.it (ICQ UIN: 529796)
transfer interrupted (code20)
Hi, Im trying now to implement my mail server mirror but as i look at the logs of the mail server i used to mirror i see this type of log transfer interrupted (code20) at rsync.c(229). and it the remote who's mirrorring the mail server does not transferring file and got stocked in recieving file list. I've tried transffering other path and it works fine but in the users home directory im recieving this error. I have alot of users in my mail server. What seems to be the problem? Regards, Michael P. Carel Systems Administrator TEAM PACIFIC CORPORATION FTI Taguig Metro, Manila. PHIL. Tel. Nos. (02) 838-50-05 local 363 (02) 838-84-38 direct
