Rsync hangs with no visible reason
Hi, I have a daily rsync job (in cron) that hangs in ps process list (linux) until i kill it. The rsync suppose to backup one directory which includes about 2500 sub-dirs in it, e.g rsync of /home/user/stuff where stuff has about 2500 sub-dirs inside, but not many files in each sub-directory. I checked that the job runs when both client and server doesn't run any other CPU consuming processes. Running the command manually from shell, always succeeds and ending in about 25-30 minutes. The rsync, usually, hangs just after building file list... Done, and i couldn't find any error in the server's rsyncd log. adding -vv didn't give any clue. In `ps aux | grep rsync` it shows that the process is S Interruptible sleep (waiting for an event to complete) What can be the cause of that behaviour? What can i do to debug better, or solve it? I'm using rsync 2.6.3 (RHES 4 rpm) Regards, Milutin Voinivich NasBackup begin:vcard fn:Milutin Voinivich n:Voinivich;Milutin email;internet:[EMAIL PROTECTED] note;quoted-printable:Try NasBackup.=0D=0A= http://www.nasbackup.com/ version:2.1 end:vcard -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Is --link-dest supported on remote ssh side?
I'm trying to make backups with rsync with the --list-dest option. My first backup is made like this: rsync -azR --delete -e ssh --exclude-from=./exclude.lst /tmp/test [EMAIL PROTECTED]:/tmp/backup/backup-nelly-2007-04-17-12-34-26/ Without touching any of the files which were backed up, I launch a new backup with this command: rsync -azR --delete -e ssh --exclude-from=./exclude.lst --link-dest=/tmp/backup/nelly-2007-04-17-12-34-26 /tmp/test [EMAIL PROTECTED]:/tmp/backup/backup-nelly-2007-04-17-12-34-33/ I had expected all files in the second backup to be hard links to the first backup files, but this does not seem to be the case: ls -1 -i backup-nelly*/tmp/test/testfile 10286511 backup-nelly-2007-04-17-12-34-26/tmp/test/testfile 10286738 backup-nelly-2007-04-17-12-34-33/tmp/test/testfile Both client and server side are using rsync 2.6.9 protocol version 29. What am I doing wrong, or is this simply not supported? -- Frederik -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync SSL 'for real'
Dear All, I've been using rsync with stunnel with success for a while now, but I'm not necessarily satisfied with it, i.e. having an extra layer externally and all, no authentication against SSL-subjects .. I had a look at the patch included with the source, and it doesn't quite address everything related to SSL, i.e. forcing SSL, verify options etc. are all missing - besides, it's not included in the main source for some reason. I'd be willing to put up the effort to add full (or at least better) SSL support direclty to rsync if the maintainers would agree with it, and would seriously consider including it in the main source in the long run. My approach would: - stay backwards-compatible with the existing patch (starttls) - support SSL-only ports (a port number would have to be allocated) - support checking the certificate chain (revocation lists would have to be maintained externally, thou) - support enforcing the use of SSL (globally as well as for certain hosts) - support authentication against x509-subjects instead of usernames - configuration via existing methods (commandline, config-file) - support for hardware-aided crypto/randomness via OpenSSL engines - enough inline documentation to allow for maintenance by the main team Anything else? Please let me know wether there would be interest by the community as well as support by the maintainers by the end of the month. Thanks, Andreas -- God is a comedian playing to an audience too afraid to laugh. -- H.L.Mencken -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Is --link-dest supported on remote ssh side?
On 4/17/07, Frederik [EMAIL PROTECTED] wrote: rsync -azR --delete -e ssh --exclude-from=./exclude.lst --link-dest=/tmp/backup/nelly-2007-04-17-12-34-26 /tmp/test [EMAIL PROTECTED]:/tmp/backup/backup-nelly-2007-04-17-12-34-33/ OK, obvious typo in the above command :-) It works correct actually! -- Frederik -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Help! What is this error??
Hi all, I am running the following command from winXP: rsync -av -e bin\ssh -i trans_key\access.key -l julian -p 2227 192.168.200.254::julian .\julian\ and getting the following output: rsync: Failed to dup/close: Socket operation on non-socket (108) rsync error: error in IPC code (code 14) at pipe.c(71) [receiver=2.6.8cvs] rsync: connection unexpectedly closed (0 bytes received so far) [receiver] rsync error: error in rsync protocol data stream (code 12) at io.c(475) [receiver=2.6.8cvs] Everything had been working for months, but suddenly I am getting this. Removing the -e ssh etc.. part seems to make the problem go away, but I can only connect to this server through ssh, and it is on a vpn. Anyone got a clue!!?? I am really stuck on this one!! Thanks Julian -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
HTTP encapsulated rsync??
Hi folks, I am hoping someone here can offer some suggestions. Here is my situation: I am using rsync over the internet for several hundred clients to keep them in sync with a master repository of files. The rsync daemon is listening on port 80, because most of the clients are behind firewalls that only allow outbound port 80 (and other common ports). This works fine for most of them but not all. The trouble is that some of them are also behind devices that do not allow non-http traffic on port 80 and thus block the rsync. Therefore, I am looking for a way to use http encapsulation as a means to bypass the blocking devices. I have seen a few scattered queries around the net asking similar questions but I have not seen any good solutions offered. One person suggested that http-tunnel be used but this is not a very elegant solution and I would like to avoid it if at all possible. If the tunnel were to hang or sever then I would have a difficult time correcting it since I do not have direct access to many of the clients. Does anyone here have any ideas? Surely I am not the first person to ask this question. Thanks!! Robert -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: HTTP encapsulated rsync??
On 4/17/07, Robert Denton [EMAIL PROTECTED] wrote: Hi folks, I am hoping someone here can offer some suggestions. Here is my situation: I am using rsync over the internet for several hundred clients to keep them in sync with a master repository of files. The rsync daemon is listening on port 80, because most of the clients are behind firewalls that only allow outbound port 80 (and other common ports). This works fine for most of them but not all. The trouble is that some of them are also behind devices that do not allow non-http traffic on port 80 and thus block the rsync. Therefore, I am looking for a way to use http encapsulation as a means to bypass the blocking devices. I have seen a few scattered queries around the net asking similar questions but I have not seen any good solutions offered. One person suggested that http-tunnel be used but this is not a very elegant solution and I would like to avoid it if at all possible. If the tunnel were to hang or sever then I would have a difficult time correcting it since I do not have direct access to many of the clients. Does anyone here have any ideas? Surely I am not the first person to ask this question. Thanks!! Robert You can use an HTTP proxy. Look at the RSYNC_PROXY environment variable in the man page. -- Aaron W Morris (decep) -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: HTTP encapsulated rsync??
Do you mean for example.. Instead of running rsync on port 80, have rsync listen for requests on 873 as usual, but also have squid running on the same server listening for port 80 connections, and then just configure squid to send all port 80 traffic to 873? Robert On Apr 17, 2007, at 1:27 PM, Aaron W Morris wrote: On 4/17/07, Robert Denton [EMAIL PROTECTED] wrote: Hi folks, I am hoping someone here can offer some suggestions. Here is my situation: I am using rsync over the internet for several hundred clients to keep them in sync with a master repository of files. The rsync daemon is listening on port 80, because most of the clients are behind firewalls that only allow outbound port 80 (and other common ports). This works fine for most of them but not all. The trouble is that some of them are also behind devices that do not allow non-http traffic on port 80 and thus block the rsync. Therefore, I am looking for a way to use http encapsulation as a means to bypass the blocking devices. I have seen a few scattered queries around the net asking similar questions but I have not seen any good solutions offered. One person suggested that http-tunnel be used but this is not a very elegant solution and I would like to avoid it if at all possible. If the tunnel were to hang or sever then I would have a difficult time correcting it since I do not have direct access to many of the clients. Does anyone here have any ideas? Surely I am not the first person to ask this question. Thanks!! Robert You can use an HTTP proxy. Look at the RSYNC_PROXY environment variable in the man page. -- Aaron W Morris (decep) -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: HTTP encapsulated rsync??
On 4/17/07, Robert Denton [EMAIL PROTECTED] wrote: Do you mean for example.. Instead of running rsync on port 80, have rsync listen for requests on 873 as usual, but also have squid running on the same server listening for port 80 connections, and then just configure squid to send all port 80 traffic to 873? Robert I was thinking more along the lines of creating a proxy for all of your rsync requests. You would not necessarily have to do any special configuration to squid (except allow CONNECT with port 873, something I know has to be done with mod_proxy), it should route the request itself. Apache with mod_proxy would work as well. -- Aaron W Morris (decep) -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: HTTP encapsulated rsync??
This is an interesting idea. Here is what I am trying: I have set up squid to listen on port 81, since rsync on the same machine is already listening for requests on 80. I have set the RSYNC_PROXY env var to the hostname:81 and rand a quick test. The result is that I am getting this error: bad response from proxy -- HTTP/1.0 503 Service Unavailable rsync: failed to connect to 192.168.233.56: Success (0) I am thinking this is a problem with the squid config not passing the rsync request to the rsync daemon. Another though occurs to me: many of the clients are behind enforced proxies of their own, set via an env var: http_proxy. Is this going to work with the RSYNC_PROXY? Since the rsync proxy is pointing to an http port on a server, will the request automatically be channeled through the enforced http_proxy? Robert On Apr 17, 2007, at 4:00 PM, Aaron W Morris wrote: On 4/17/07, Robert Denton [EMAIL PROTECTED] wrote: Do you mean for example.. Instead of running rsync on port 80, have rsync listen for requests on 873 as usual, but also have squid running on the same server listening for port 80 connections, and then just configure squid to send all port 80 traffic to 873? Robert I was thinking more along the lines of creating a proxy for all of your rsync requests. You would not necessarily have to do any special configuration to squid (except allow CONNECT with port 873, something I know has to be done with mod_proxy), it should route the request itself. Apache with mod_proxy would work as well. -- Aaron W Morris (decep) -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Einladung - Invito
(Versione italiana più sotto) Sehr geehrte Damen und Herren Wir haben bemerkt, dass viele Firmenemailadressen keine HTML-Emails empfangen dürfen. Aus diesem Grund senden wir Ihnen unsere Einladung auch in Textformat zu, damit Sie diese rechtzeitig erhalten, denn die Anmeldefrist läuft bald ab. Hiermit laden wir Sie herzlich ein unsere exklusive Veranstaltung zu besuchen: Der bulgarische Finanzmarkt - Gelegenheiten und Perspektiven in Zürich am 9. Mai 2007, 9:00-14:00 Uhr, Hotel Baur au Lac Die Information ist sowohl für institutionelle als auch für private Anleger wichtig, da sie dadurch einen Insider-Einblick in den boomenden Finanzmarkt Bulgariens erhalten. Die Veranstaltung eröffnet neue Perspektiven und bietet bis jetzt nicht wahrgenommenen Chancen für den Profi-Investor. Für alle Details öffnen Sie bitte diesen Link in Ihrem Browser: http://www.swisscham-bulgaria.org/index.php?id=215tx_rlmpeventdb_pi1[showUid]=3no_cache=1 Sagen Sie uns Ihre Meinung über diese Veranstaltung in unserer anonymen Umfrage hier: http://www.swisscham-bulgaria.org/index.php?id=290 Wenn Sie keine weiteren Einladungen von der WKSB erhalten möchten, folgen Sie bitte diesem Link: http://www.swisscham-bulgaria.org/cgi-bin/acmelist/l.cgi?subid=18uss=1[EMAIL PROTECTED] Mit freundlichen Grüssen, Biliana Monova, Sekretariat Versione italiana Gentili signore, egregi signori, Abbiamo notato, che tante ditte non ricevano le e-mail in HTML. Per questo motivo inviamo il nostro invito in formato testo, per farlo diventare ricevibile e per arrivarvi in tempo utile, visto che i tempi per la registrazione si stanno esaurendo. Con il presente vinvitiamo a partecipare al nostro evento esclusivo: Il mercato monetario bulgaro - occasioni e prospettive 10 maggio 2007, 9:00-14:00 nello Splendid Royal Hotel di Lugano Linformazione è estremamente utile sia per gli investitori istituzionali, sia per quelli di carattere privato. In questo modo potete avere uno sguardo dallinterno nel prospero mercato monetario della Bulgaria. Levento apre nuove prospettive e propone agli investitori professionali opportunità fin ora sconosciute o sottovalutate. Per ottenere i dettagli riguardo allevento, vi preghiamo di aprire questo link nel Vostro Browser: http://www.swisscham-bulgaria.org/index.php?id=215L=5tx_rlmpeventdb_pi1[showUid]=4no_cache=1 Comunicateci il Vostro prezioso parere riguardo allevento attraverso la qui presente inchiesta anonima: http://www.swisscham-bulgaria.org/index.php?id=290L=5 Se volete ottenere un maggior numero dinviti dalla Camera di Economia SvizzeraBulgaria, seguite questo link: http://www.swisscham-bulgaria.org/cgi-bin/acmelist/l.cgi?subid=18uss=1[EMAIL PROTECTED] Cordiali saluti, Biliana Monova, segretario -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Problem with escaped control characters
I have created a program for myself for backing up data to tape from my disk to disk to tape system, and after the initial day, I have been using the output of a dry run of rsync to determine what files need to be sent to tape. I think due to an upgrade in rsync the current version now escapes control sequences with something like \#010 instead of ^H, however this is now breaking my backup program because the perl file::stat module cannot interpret the escaped sequence as a file. I have tried doing a replace on the string to change it back to the old with no avail. Is there way to make rsync do it the old way? Right now my only choice seems to be skipping the files or finding another program to determine the list of files to be transferred. Any ideas? thanks, Dan -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Problem with escaped control characters
On Tue, Apr 17, 2007 at 04:30:53PM -0500, Daniel Davidson wrote: I think due to an upgrade in rsync the current version now escapes control sequences with something like \#010 instead of ^H, however this is now breaking my backup program because the perl file::stat module cannot interpret the escaped sequence as a file. You can easily tweak the names in the perl script: $name =~ s/\\#(\d\d\d)/ chr(oct($1)) /eg; ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync SSL 'for real'
On Tue, Apr 17, 2007 at 12:52:57PM +0200, Andreas Kotes wrote: I'd be willing to put up the effort to add full (or at least better) SSL support direclty to rsync if the maintainers would agree with it, and would seriously consider including it in the main source in the long run. I'd be interested in getting SSL support into the main rsync codebase (in such a way that it could be conditionally omitted, if needed). I haven't had time to really check into it myself (I need to figure out why the SSL patch didn't work for me, but never had the impetus to delve into it). Your list of SSL goals seem quite reasonable to me. ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: Problems with --delete
On Mon, Apr 16, 2007 at 01:44:29PM -0700, [EMAIL PROTECTED] wrote: The strange thing is that _some_ files are being deleted.. just a few, and obviously not nearly enough. Are you checking for errors? Does rsync get an I/O error and stop deleting? (See the --ignore-errors option for one way to deal with that.) ..wayne.. -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: HTTP encapsulated rsync??
On 4/17/07, Robert Denton [EMAIL PROTECTED] wrote: This is an interesting idea. Here is what I am trying: I have set up squid to listen on port 81, since rsync on the same machine is already listening for requests on 80. I have set the RSYNC_PROXY env var to the hostname:81 and rand a quick test. The result is that I am getting this error: bad response from proxy -- HTTP/1.0 503 Service Unavailable rsync: failed to connect to 192.168.233.56: Success (0) I am thinking this is a problem with the squid config not passing the rsync request to the rsync daemon. Another though occurs to me: many of the clients are behind enforced proxies of their own, set via an env var: http_proxy. Is this going to work with the RSYNC_PROXY? Since the rsync proxy is pointing to an http port on a server, will the request automatically be channeled through the enforced http_proxy? Robert If some of the clients are already having to go through proxies, it does complicate matters. If that is the case, you may have to investigate setting up said proxies to proxy again using proxy chaining/cascading, but that's a bit out of scope for this mailing list. It's difficult for me to say what the problem is with the 503 error, but it sounds like you're on the right track. -- Aaron W Morris (decep) -- To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
CVS update: rsync/testsuite
Date: Wed Apr 18 03:30:53 2007 Author: wayned Update of /data/cvs/rsync/testsuite In directory dp.samba.org:/tmp/cvs-serv10086 Modified Files: default-acls.test Log Message: Got rid of protocol-29 check. Revisions: default-acls.test 1.3 = 1.4 http://www.samba.org/cgi-bin/cvsweb/rsync/testsuite/default-acls.test?r1=1.3r2=1.4 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
CVS update: rsync/patches
Date: Wed Apr 18 03:34:33 2007 Author: wayned Update of /data/cvs/rsync/patches In directory dp.samba.org:/tmp/cvs-serv12130 Modified Files: acls.diff Log Message: Fixed failing hunk. Revisions: acls.diff 1.195 = 1.196 http://www.samba.org/cgi-bin/cvsweb/rsync/patches/acls.diff?r1=1.195r2=1.196 ___ rsync-cvs mailing list rsync-cvs@lists.samba.org https://lists.samba.org/mailman/listinfo/rsync-cvs
