Re: rsync / cygwin hanging problem
jw schultz writes: So anyway, I know the rsync sometimes hangs under cygwin problem has been acknowledged, but does anybody know anything about when it might be identified and resolved? As far as i can tell none of the people who know the rsync code use Windows with the possible exception of Craig. He has a patch that may reduce the frequency of hangs but his recommendation (if i recall correctly) is to not use rsync over ssh on windows. Rsync apparently works OK as a daemon on whendoze and when accessing an rsync daemon. The problem on Cygwin is the parent fails to kill the child at the end of a receive. Without finding the root-cause (I don't know about cygwin internals), I rearranged the code around the msleep loop to create a patch did solve the problem for me: http://lists.samba.org/archive/rsync/msg06500.html But several other people have reported that it didn't help. Following that, Anthony Heading came up with a new patch that used an additional pipe, instead of a signal, to tell the child to exit: http://lists.samba.org/archive/rsync/msg06501.html with a diff -u patch here: http://lists.samba.org/archive/rsync/msg06527.html James Tillman reported that this fixed his hang problem on Cygwin. But JW noted this produced an error in regression testing: http://lists.samba.org/archive/rsync/msg06615.html and then he provided an improved patch: http://lists.samba.org/archive/rsync/msg06624.html and Lapo Luchini came up with a test package with the patch: http://lists.samba.org/archive/rsync/msg06625.html but noted it failed make check. Shortly after that point the thread died. The basic approach in the patch looks sound. Unfortunately I haven't had the time to follow through on these patches. Plus my test case that used to create a hang isn't easy to replicate for me. John Hunt writes: If any developer is working on this problem, I would be happy to test any new rsync executable for them, to establish if it works in my environment. Why don't you try JW's last patch above? Something close to this should be a reliable fix. Craig -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync / cygwin hanging problem
Have you tried rsync 2.5.5 ? I have reverted to that because 2.5.6 was hanging every time (see my posting on 2003-11-17). I used 2.5.5 for several months without problems, before upgrading cygwin recently. Unlike you I am NOT using ssh. I am using rsync (at work that is) as part of a script to automate part of the process of producing a distribution package for an MS$ VB application that we sell. I got really tired of spending half an hour moving files around by hand after running the VB Package Deployment Wizard, and making standard edits to some of the packaging files, so I automated it using cygwin, bash and perl. If any developer is working on this problem, I would be happy to test any new rsync executable for them, to establish if it works in my environment. ...John -- John Hunt Stokesley, N Yorks, UK e-mail: [EMAIL PROTECTED] URL: http://www.yoredale.uklinux.net/ -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync / cygwin hanging problem
On Wed, Nov 19, 2003 at 04:28:38PM -0500, Jim Salter wrote: Has any progress ever been made towards fixing the problem causing rsync processes to hang in the cygwin environment? I've tried just using Unison in the windows environment, since it has a native port, but, um. Unison is probably great for what it IS intended for, but it absolutely sucks nuts as a backup solution (has a tendency to choke and cancel the ENTIRE PROCESS without synching anything at all if it encounters a file it doesn't like, and there are LOTS of file types it doesn't like). I'm guessing you are hitting the file locking issue but that is just a guess. So anyway, I know the rsync sometimes hangs under cygwin problem has been acknowledged, but does anybody know anything about when it might be identified and resolved? As far as i can tell none of the people who know the rsync code use Windows with the possible exception of Craig. He has a patch that may reduce the frequency of hangs but his recommendation (if i recall correctly) is to not use rsync over ssh on windows. Rsync apparently works OK as a daemon on whendoze and when accessing an rsync daemon. My gut impression is that there is a problem with pipe delivery in the cygwin environment that is probably a manifestation of a whenblows IPC problem or differing semantics. If that is correct this problem may be intractable. When will this problem be resolved? That may prove to be a question of perspective. Some might say it already is. It would be nice if rsync were reliable on that other, legacy OS (Windos) so if someone can come up with a patch that actually fixes the problem without making the code even uglier and without breaking any other platforms i'd be glad to commit it. But until then any fixes will remain as external patches and rsync on Windows will be just like many other things on Windows: it requires hand-holding and often fails. -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync / cygwin hanging problem
over ssh on windows. Rsync apparently works OK as a daemon on whendoze and when accessing an rsync daemon. Actually I'm using Rsync as a client under cygwin / win2K to access rsync in daemon mode on a FreeBSD 5.1 server. The process hangs for no apparent reason, and even when not utterly hung sits around consuming almost zero CPU resources, only to suddenly and inexplicably lurch back into action and start actually doing things again and eat 25-70% of system resources again minutes later. It may be worth noting that this behavior only shows up when updating an existing mirror - if using rsync as a fancy networked version of cp, it behaves as expected. It is also worth noting that I just replaced the stock 2.5.6 under Cygwin with the recommended replacement from the BackupPC project, with no apparent difference whatsoever. And finally, SSH under Cygwin seems to work fine; as aforementioned I've experimented with using Unison and it operates quickly, smoothly and reliably using Cygwin/SSH as a transport... unfortunately it also does an incredibly bad job of being a one-size-fits-all automated mirroring tool, since by design it bombs entirely out of synch jobs and does absolutely nothing with ANY of the job if it encounters anything it doesn't like on any single file. When will this problem be resolved? That may prove to be a question of perspective. Some might say it already is. It would be nice if rsync were reliable on that other, legacy OS (Windos) so ... If you're trying to convince me that Windows sucks, you're preaching to the clergy. But I still have to support an awful lot of people who use it for disturbingly important things, so... ::helpless shrug:: WAY off-topic: Let's just hope that the EU doesn't back down from forcing them to deinstall Media Player from European versions of Windows. If they stick to their guns, MS will either have to open themselves up to retrograde American lawsuits (if they can strip out WMP, why couldn't they strip out IE?) or back out of Europe entirely and give the OS world an ENORMOUS un-microsoft-fettered market to grow in. Either way, big win for the rest of us. Jim Salter JRS Systems -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync / cygwin hanging problem
On Wed, Nov 19, 2003 at 06:20:16PM -0500, Jim Salter wrote: over ssh on windows. Rsync apparently works OK as a daemon on whendoze and when accessing an rsync daemon. Actually I'm using Rsync as a client under cygwin / win2K to access rsync in daemon mode on a FreeBSD 5.1 server. The process hangs for no apparent reason, and even when not utterly hung sits around consuming almost zero CPU resources, only to suddenly and inexplicably lurch back into action and start actually doing things again and eat 25-70% of system resources again minutes later. Sounds like data getting stuck in the pipes perhaps combined with poor process scheduling. I don't think we'll ever really know unless someone fully instruments this using traces on all processes time correlated with tcpdump. That is a lot of work. It may be worth noting that this behavior only shows up when updating an existing mirror - if using rsync as a fancy networked version of cp, it behaves as expected. It is also worth noting that I just replaced the stock 2.5.6 under Cygwin with the recommended replacement from the BackupPC project, with no apparent difference whatsoever. And finally, SSH under Cygwin seems to work fine; as aforementioned I've experimented with using Unison and it operates quickly, smoothly and reliably using Cygwin/SSH as a transport... unfortunately it also does an incredibly bad job of being a one-size-fits-all automated mirroring tool, since by design it bombs entirely out of synch jobs and does absolutely nothing with ANY of the job if it encounters anything it doesn't like on any single file. When will this problem be resolved? That may prove to be a question of perspective. Some might say it already is. It would be nice if rsync were reliable on that other, legacy OS (Windos) so ... If you're trying to convince me that Windows sucks, you're preaching to the I'd no more do that than try to convince someone that water is wet. clergy. But I still have to support an awful lot of people who use it for disturbingly important things, so... ::helpless shrug:: ::sympathy:: Now if i can just convince some people that there is an alternative maybe i can improve my cashflow. [snip] -- J.W. SchultzPegasystems Technologies email address: [EMAIL PROTECTED] Remember Cernan and Schmitt -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
Re: rsync / cygwin hanging problem
recommendation (if i recall correctly) is to not use rsync over ssh on windows. Rsync apparently works OK as a daemon on whendoze and when accessing an rsync daemon. It does not make me happy to read this. Anyway, I like to describe my experience of the last two days, maybe this helps someone, or someone can help me: I wanted to synch an Win2000 machine and a WinXP machine. For some (offtopic) reason I can use Win2k only as local system, since I can´t even ping it although there is no firewall and hidden server mode is not activated (as far as I know. Hints on that are welcome too). rsync -...e ssh [EMAIL PROTECTED] /cygdrive/... works well with small folders, but with larger ones, it hangs and copies not more than one (!) file. Additionally I have a Linux machine in the network. My final solution was to synch first to that system and from that to the Win2k system. Summarized it behaves like this: local (data direction) remote Win2k -- WinXP hangs at folders with ca. 300 files after copying 1 files at max. linux -- WinXP hangs WinXP -- linux copies about 2GB, then freezes the whole WinXP (not only cygwin, ctrl+c not working) Win2k -- linux works fine, but with incredibly high processor load on Win2k (like 900MHz for maybe 3MBit/s) Uli M. P.s.: All versions of last weeks cygwin webpage and Suse8.2. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
