Re: rsync local dir copy hang - solved for me
Would it be possible to get the testcase? I have an idea for a fix, and would like to try it out on the testcases that I can see. Thanks. On Sat, 25 Oct 2003 12:17:46 +0200, Corinna Vinschen wrote: On Fri, Oct 24, 2003 at 07:41:54AM -0700, Bakken, Luke wrote: It never got past cygwin_select(). [...] The bottom point: *don't do that* ;-) i.e. leave the msleep() call there. If I leave msleep there, it hangs every time. So I'm taking it out. Now, But Hannu is right. Why not replace msleep(2) by usleep(2)? AFAICS, it should fix the hang as well. Would you mind to try it? I'm now going to see if I can isolate the heart of the problem with a simple program that forks and sleeps in the same fashion as with rsync. No worries, I have a working testcase which shows the hang. But thanks for the offer. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: rsync local dir copy hang - solved for me
On Fri, Oct 24, 2003 at 07:41:54AM -0700, Bakken, Luke wrote: It never got past cygwin_select(). [...] The bottom point: *don't do that* ;-) i.e. leave the msleep() call there. If I leave msleep there, it hangs every time. So I'm taking it out. Now, But Hannu is right. Why not replace msleep(2) by usleep(2)? AFAICS, it should fix the hang as well. Would you mind to try it? I'm now going to see if I can isolate the heart of the problem with a simple program that forks and sleeps in the same fashion as with rsync. No worries, I have a working testcase which shows the hang. But thanks for the offer. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Developermailto:[EMAIL PROTECTED] Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: rsync local dir copy hang - solved for me
From: Bakken, Luke
Sent: Friday, October 24, 2003 4:19 AM
Hi all,
I was attempting to use rsync (v 2.5.6) to copy one local directory to
another on my machine (Windows XP) today when I experienced the dreaded
hang. It copied the files, and hung waiting for a child process to exit.
When I killed that process using taskmgr, the parent exited OK.
The following fixed my particular hang situation:
--- orig_main.c 2003-10-23 19:16:22.5 -0700
+++ main.c 2003-10-23 19:16:35.828125000 -0700
@@ -47,7 +47,7 @@
int cnt;
while ((waited_pid = waitpid(pid, status, WNOHANG)) == 0) {
- msleep(20);
+ /* msleep(20); */
io_flush();
}
I hope this can be of help to someone!
Luke
Now the following text isn't meant to be rude or some such, so please don't
take offence. This is just yet another of those itchy little points of
mine...
$ man waitpid
No manual entry for waitpid
$ man msleep
No manual entry for msleep
Hmm? Is waitpid() a blocking call or not? WRT WNOHANG it looks like an
unblocking one...
Thus I'm assuming it doesn't block and that msleep() is something internal
to rsync, using usleep() or some such.
If msleep() does more than so, I would guess the problem to be inside
msleep().
A general comment: Given what I've written above the result of the patch is
called busy waiting and is a *hard* blow against all other software that
is running at the same time.
If the loop exit condition appears late the machine running the above
code will experience jerkyness and other ill effects - at the extreme but
still likely level there might even be disturbances of *vital* system
functions (e.g. related to time critical tasks).
The bottom point: *don't do that* ;-)
i.e. leave the msleep() call there. This is a simple - yet effective - way
to avoid ill effects of a tight loop.
/Hannu E K Nevalainen, B.Sc. EE - 59?16.37'N, 17?12.60'E
-- printf(Timezone: %s\n, (DST)?UTC+02:UTC+01); --
Example:
I have a cheap SCSI-interface in my PC, this is connected to my scanner;
now can you guess what happens when I start scanning?
Not very hard, ehh? Yes indeed, the entire computer stops, waiting for SCSI
I/O; i.e. 10-20 seconds for a preview, 30-60 seconds for scanning the bus at
boot time, anything from a minute and up for scanning an image.
Guess what? I *do* disable this piece of hardware when I'm not going to use
it explicitly. (Monetary reasons leaves me in this situation) :-7
--END OF MESSAGE--
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/
RE: rsync local dir copy hang - solved for me
From: Mark Thornton Sent: Friday, October 24, 2003 12:13 PM Mark, please - even if your repliy might seem a bit personal - do keep replies on the list. There might be some little spark, that makes others have a fire lit (i.e. get a bright idea). The bottom point: *don't do that* ;-) i.e. leave the msleep() call there. This is a simple - yet effective - way to avoid ill effects of a tight loop. However it remains true that there is some sort of bug in rsync's thread handling (or perhaps in cygwin's thread library). Hmmm... I think I read something about a similiar thing, having the word pthreads in the subject. Might this be related? For some a cure for this, even using busy waiting, may be preferable to the bug. Mark Thornton For the time beeing, yes it might be preferable - but not as a long time solution. And one has to be aware of the possible ill effects. /Hannu E K Nevalainen, B.Sc. EE - 59?16.37'N, 17?12.60'E -- printf(Timezone: %s\n, (DST)?UTC+02:UTC+01); -- --END OF MESSAGE-- -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: rsync local dir copy hang - solved for me
Hmm? Is waitpid() a blocking call or not? WRT WNOHANG it looks like an unblocking one... Yep. Thus I'm assuming it doesn't block and that msleep() is something internal to rsync, using usleep() or some such. It uses select(). When I used strace on a non-modified version of rsync to watch where it hung, here's the last bit of output I saw: 18 9449430 [main] rsync 3260 stopped_or_terminated: considering pid 3276 18 9449448 [main] rsync 3260 checkstate: returning -1 18 9449466 [main] rsync 3260 proc_subproc: waiting thread found no children 20 9449486 [main] rsync 3260 proc_subproc: finished processing terminated/stopped child 18 9449504 [main] rsync 3260 proc_subproc: returning 1 21 9449525 [main] rsync 3260 wait4: 0 = WaitForSingleObject (...) 19 9449544 [main] rsync 3260 wait4: intpid 3276, status 0x22FE28, w-status 0, options 1, res 0 20 9449564 [main] rsync 3260 cygwin_select: 0, 0x0, 0x0, 0x0, 0x22FDC0 19 9449583 [main] rsync 3260 cygwin_select: to-tv_sec 0, to-tv_usec 2, ms 20 18 9449601 [main] rsync 3260 cygwin_select: sel.always_ready 0 It never got past cygwin_select(). of the patch is called busy waiting and is a *hard* blow against all other software that is running at the same time. But, it's OK for me because a) It fixes the problem b) It doesn't wait long at all c) It doesn't appear to tie up resources. If the loop exit condition appears late the machine running the above code will experience jerkyness and other ill effects - at the extreme but still likely level there might even be disturbances of *vital* system functions (e.g. related to time critical tasks). Nope, I haven't seen any of this. It could happen, I'm sure. The bottom point: *don't do that* ;-) i.e. leave the msleep() call there. If I leave msleep there, it hangs every time. So I'm taking it out. Now, this fixes one hang at one time, but not others. For instance, if there is nothing to do since the sources and targets are in sync, it'll hang. If the source dir doesn't exist, it still hangs. I'm now going to see if I can isolate the heart of the problem with a simple program that forks and sleeps in the same fashion as with rsync. Luke -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
rsync local dir copy hang - solved for me
Hi all,
I was attempting to use rsync (v 2.5.6) to copy one local directory to
another on my machine (Windows XP) today when I experienced the dreaded
hang. It copied the files, and hung waiting for a child process to exit.
When I killed that process using taskmgr, the parent exited OK.
The following fixed my particular hang situation:
--- orig_main.c 2003-10-23 19:16:22.5 -0700
+++ main.c 2003-10-23 19:16:35.828125000 -0700
@@ -47,7 +47,7 @@
int cnt;
while ((waited_pid = waitpid(pid, status, WNOHANG)) == 0) {
- msleep(20);
+ /* msleep(20); */
io_flush();
}
I hope this can be of help to someone!
Luke
cygcheck -s output:
Cygwin Win95/NT Configuration Diagnostics
Current System Time: Thu Oct 23 19:05:06 2003
Windows XP Professional Ver 5.1 Build 2600 Service Pack 1
Path: C:\cyg\usr\local\bin
C:\cyg\bin
C:\cyg\bin
C:\cyg\usr\X11R6\bin
c:\VS\VSS\win32
c:\ncftp
d:\texmf\miktex\bin
c:\gs\gs8.00\bin
c:\gv\gsview
c:\Perl\bin\
c:\vim\vim62
c:\usr\local\bin
c:\WINDOWS\system32
c:\WINDOWS
c:\WINDOWS\System32\Wbem
c:\scripts
c:\Program Files\Starbase\StarGate SDK\Lib
C:\cyg\home\lukeb\bin
Output from C:\cyg\bin\id.exe (nontsec)
UID: 1003(lukeb) GID: 513(None)
513(None)
Output from C:\cyg\bin\id.exe (ntsec)
UID: 1003(lukeb) GID: 513(None)
513(None)544(Administrators)
545(Users)
SysDir: C:\WINDOWS\System32
WinDir: C:\WINDOWS
CYGWIN = `tty binmode ntsec ntea'
HOME = `C:\cyg\home\lukeb'
MAKE_MODE = `unix'
USER = `lukeb'
Use `-r' to scan registry
a: fd N/AN/A
c: hd NTFS 39997Mb 36% CP CS UN PA FC
d: hd NTFS 36318Mb 25% CP CS UN PA FC Local Disk
e: cd N/AN/A
f: cd N/AN/A
C:\cyg / system
binmode
c: /csystem
binmode
d: /dsystem
binmode
C:\cyg/bin /usr/bin system
binmode
C:\cyg/lib /usr/lib system
binmode
C:\cyg\usr\X11R6\lib\X11\fonts /usr/X11R6/lib/X11/fonts system
binmode
. /mnt system
binmode,cygdrive
Found: C:\cyg\bin\awk.exe
Found: C:\cyg\bin\bash.exe
Found: C:\cyg\bin\cat.exe
Found: c:\usr\local\bin\cat.exe
Found: C:\cyg\bin\cp.exe
Found: c:\usr\local\bin\cp.exe
Found: C:\cyg\bin\cpp.exe
Found: C:\cyg\bin\find.exe
Found: c:\usr\local\bin\find.exe
Found: C:\cyg\bin\gcc.exe
Not Found: gdb
Found: C:\cyg\bin\grep.exe
Found: c:\usr\local\bin\grep.exe
Found: c:\scripts\grep.exe
Found: C:\cyg\bin\ld.exe
Found: c:\usr\local\bin\ld.exe
Found: C:\cyg\bin\ls.exe
Found: c:\usr\local\bin\ls.exe
Found: C:\cyg\bin\make.exe
Found: c:\usr\local\bin\make.exe
Found: C:\cyg\bin\mv.exe
Found: c:\usr\local\bin\mv.exe
Found: C:\cyg\bin\rm.exe
Found: c:\usr\local\bin\rm.exe
Found: C:\cyg\bin\sed.exe
Found: c:\usr\local\bin\sed.exe
Found: C:\cyg\bin\sh.exe
Found: c:\usr\local\bin\sh.exe
Found: C:\cyg\bin\tar.exe
Found: c:\usr\local\bin\tar.exe
61k 2003/08/09 C:\cyg\bin\cygbz2-1.dll
842k 2003/09/30 C:\cyg\bin\cygcrypto-0.9.7.dll
645k 2003/04/11 C:\cyg\bin\cygcrypto.dll
22k 2003/10/17 C:\cyg\bin\cygcygipc-2.dll
380k 2002/07/24 C:\cyg\bin\cygdb-3.1.dll
487k 2002/07/24 C:\cyg\bin\cygdb_cxx-3.1.dll
132k 2003/07/12 C:\cyg\bin\cygexpat-0.dll
45k 2001/04/25 C:\cyg\bin\cygform5.dll
35k 2002/01/09 C:\cyg\bin\cygform6.dll
48k 2003/08/09 C:\cyg\bin\cygform7.dll
28k 2003/07/20 C:\cyg\bin\cyggdbm-3.dll
30k 2003/08/11 C:\cyg\bin\cyggdbm-4.dll
19k 2003/03/22 C:\cyg\bin\cyggdbm.dll
15k 2003/07/20 C:\cyg\bin\cyggdbm_compat-3.dll
15k 2003/08/11 C:\cyg\bin\cyggdbm_compat-4.dll
69k 2003/08/10 C:\cyg\bin\cyggettextlib-0-12-1.dll
12k 2003/08/10 C:\cyg\bin\cyggettextpo-0.dll
134k 2003/08/10 C:\cyg\bin\cyggettextsrc-0-12-1.dll
17k 2001/06/28 C:\cyg\bin\cyghistory4.dll
29k 2003/08/10 C:\cyg\bin\cyghistory5.dll
958k 2003/08/10 C:\cyg\bin\cygiconv-2.dll
22k 2001/12/13 C:\cyg\bin\cygintl-1.dll
37k 2003/08/10 C:\cyg\bin\cygintl-2.dll
21k 2001/06/20 C:\cyg\bin\cygintl.dll
119k 2002/02/09 C:\cyg\bin\cygjpeg6b.dll
26k 2001/04/25 C:\cyg\bin\cygmenu5.dll
20k 2002/01/09 C:\cyg\bin\cygmenu6.dll
29k 2003/08/09 C:\cyg\bin\cygmenu7.dll
156k 2001/04/25 C:\cyg\bin\cygncurses++5.dll
175k 2002/01/09 C:\cyg\bin\cygncurses++6.dll
226k 2001/04/25 C:\cyg\bin\cygncurses5.dll
202k 2002/01/09 C:\cyg\bin\cygncurses6.dll
224k 2003/08/09 C:\cyg\bin\cygncurses7.dll
15k 2001/04/25 C:\cyg\bin\cygpanel5.dll
12k 2002/01/09 C:\cyg\bin\cygpanel6.dll
19k 2003/08/09 C:\cyg\bin\cygpanel7.dll
67k 2003/09/10 C:\cyg\bin\cygpcre-0.dll
63k 2003/04/11
Re: rsync local dir copy hang - solved for me
I'm just a user and follower of cygwin, but it amazes me how many people put the output of cygcheck -s output: or cygcheck -svr in the body of their email instead of as an attachment when every stinking email has http://cygwin.com/problems.html at the bottom of it and that page has Run cygcheck -s -v -r cygcheck.out and include that as an attachment in your report. No wonder you guys are just mean. Sorry for the noise, Pete -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: rsync local dir copy hang - solved for me
At 10:34 PM 10/23/2003, Peter J. Stieber you wrote: I'm just a user and follower of cygwin, but it amazes me how many people put the output of cygcheck -s output: or cygcheck -svr in the body of their email instead of as an attachment when every stinking email has http://cygwin.com/problems.html at the bottom of it and that page has Run cygcheck -s -v -r cygcheck.out and include that as an attachment in your report. No wonder you guys are just mean. Sorry for the noise, Pete You've heard of laws are made to be broken? Well, this is documentation is written to be ignored. So why are you reading the documentation? Looks like you've been on this list too long. You've become infected with good sense. As a result, it looks like you're starting to become just plain mean! ;-) -- Larry Hall http://www.rfk.com RFK Partners, Inc. (508) 893-9779 - RFK Office 838 Washington Street (508) 893-9889 - FAX Holliston, MA 01746 -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
