Re: Cygwin fork performance followup

[Eliot Moss]

On 2/4/2021 9:29 AM, Hamish McIntyre-Bhatty via Cygwin wrote:
> Hi all,
>
> I managed to improve my Cygwin VM's performance significantly using a
> variety of methods.
>
> If anyone else is experiencing troubles, seeing what I changed at
> https://www.hamishmb.com/blog/improving-cygwins-performance-fork/ might
> also help you. Not using VirtualBox was the main improvement for me (now
> using KVM/QEMU). Note that I didn't achieve much difference, but that
> was because my system is relatively powerful. What isn't shown is that
> these changes do seem to help with the fork speed drop-off that occurs
> over time.

Yes, fork can be a bottleneck in Cygwin.  I do wonder if we might get some
improvement for a lot of common cases if spawn were used more, in favor of
fork, and Cygwin supported "fast paths" for the simpler cases of spawn, to
where the libraries in the child process would not need to be loaded in the
same place.  More complex cases of spawn could fall back to a fork-based
approach underneath.  The idea is to grab some low hanging fruit - if there is
any.  What I don't know is whether such fruit can be had.  For example, does
bash use spawn, and if so, does it use calls that could be optimized in this
way?  I don't know.  Likewise other tools that do lost of forking might need
to be inspected, such as 'make'.

I seem to recall this was discussed some before ...

Best wishes - Eliot
--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Cygwin fork performance followup

[Hamish McIntyre-Bhatty via Cygwin]

Hi all,

I managed to improve my Cygwin VM's performance significantly using a
variety of methods.

If anyone else is experiencing troubles, seeing what I changed at
https://www.hamishmb.com/blog/improving-cygwins-performance-fork/ might
also help you. Not using VirtualBox was the main improvement for me (now
using KVM/QEMU). Note that I didn't achieve much difference, but that
was because my system is relatively powerful. What isn't shown is that
these changes do seem to help with the fork speed drop-off that occurs
over time.

Hamish



0x87B761FE07F548D6.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature
--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Optimising cygwin fork performance

[Hamish McIntyre-Bhatty via Cygwin-apps]

On 16/12/2020 20:37, Brian Inglis wrote:
> On 2020-12-16 10:36, Marco Atzeri via Cygwin-apps wrote:
>> On 16.12.2020 13:13, Hamish McIntyre-Bhatty via Cygwin-apps wrote:
>>> So I know it's been mentioned a lot that fork is slow on Cygwin, but
>>> compared to other people's machines, eg when building, it seems way
>>> slower for me.
>>>
>>> First I'd like to know if there's a good way to measure this that
>>> anyone
>>> has found, because I'm not sure how to measure it. If I print multiple
>>> lines with echo in a script, I can see it printing maybe 2-3 a second -
>>> it's very slow.
>>>
>>> I think this might be because I'm using a Virtual Machine with
>>> VirtualBox, and QEMU/KVM might be quicker. I'm using Avira Antivurus,
>>> with exceptions for the cygwin install folders (C:\cygwin64,
>>> C:\cygwin).
>>>
>>> It might be nice if we could so some comparisons so I can figure out
>>> what's wrong.
>
> Running strace on your forking executable should give you accurate
> numbers in the output, with some work to extract the relevant values.
>
>> Same AV here, W10 64bit (no VM), 2 year old Laptop
>>
>> model name  : Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
>> 4 cores
>>
>> https://github.com/mondalaci/fork-benchmark
>> it seems there is a aging effect
>>
>> $ ./fork-benchmark.exe 1000
>> Forked, executed and destroyed 1000 processes in 39.928576 seconds.
>>
>> $ ./fork-benchmark.exe 1000
>> Forked, executed and destroyed 1000 processes in 42.701295 seconds.
>>
>> $ ./fork-benchmark.exe 1000
>> Forked, executed and destroyed 1000 processes in 49.890909 seconds.
>>
>> $ ./fork-benchmark.exe 1000
>> Forked, executed and destroyed 1000 processes in 61.657031 seconds.
>
> It's all part of your current process tree.
> Running cygserver may help this with appropriate process settings:
>
> $ grep -C1 'kern\.srv\..*_.*' /etc/cygserver.conf
>
> # kern.srv.cleanup_threads: No. of cygserver threads used for cleanup
> tasks.
> # Default: 2, Min: 1, Max: 16, command line option -c, --cleanup-threads
> #kern.srv.cleanup_threads 2
> kern.srv.cleanup_threads 16
>
> # kern.srv.request_threads: No. of cygserver threads used to serve
> #   application requests.
> # Default: 10, Min: 1, Max: 310, command line option -r,
> --request-threads
> #kern.srv.request_threads 10
> kern.srv.request_threads 310
>
> # kern.srv.process_cache_size: No. of concurrent processes which can
> be handled
> #  by Cygserver concurrently.
> # Default: 62, Min: 1, Max: 310, command line option -p, --process-cache
> #kern.srv.process_cache_size 62
> kern.srv.process_cache_size 310
>
> $ ./fork-benchmark 1000
> Forked, executed and destroyed 1000 processes in 33.397727 seconds.
> $ ./fork-benchmark 1000
> Forked, executed and destroyed 1000 processes in 34.70389 seconds.
> $ ./fork-benchmark 1000
> Forked, executed and destroyed 1000 processes in 34.186709 seconds.
> $ ./fork-benchmark 1000
> Forked, executed and destroyed 1000 processes in 33.65649 seconds.
> $ sed -En '/^model name|^cpu MHz/p;/MHz/q' /proc/cpuinfo
> model name  : AMD A10-9700 RADEON R7, 10 COMPUTE CORES 4C+6G
> cpu MHz : 3500.000
>
> $ strace -o fork-benchmark.strace ./fork-benchmark 10
> $ egrep '^--- Process [0-9]+ (crea|\(pid: [0-9]+\) exi)ted|dwProcessId
> [0-9]+|ExitProcess n 0x' fork-benchmark.strace > fork-benchmark.log
> $ awk '/ [0-9]+! pinfo::exit: /{t+=$2};END{print t/1"ms"}'
> fork-benchmark.log
> 34.1855ms
>
> Faster CPUs, faster memory, bigger caches, SSD drive may help.

Just running in KVM seems to have helped me.

My times are more like 13 seconds for 1000 forks now. There appears to
be drop-off, but after a big package build and some idling, it then gets
back down to 13 seconds.

Is there a way for me to check that the cygserver options are working?

Either way, my package builds are about twice as quick as they were
before now, possibly just from a reinstall and using KVM.

Hamish



0x87B761FE07F548D6.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

Re: Optimising cygwin fork performance

[Brian Inglis]

On 2020-12-16 10:36, Marco Atzeri via Cygwin-apps wrote:

On 16.12.2020 13:13, Hamish McIntyre-Bhatty via Cygwin-apps wrote:

So I know it's been mentioned a lot that fork is slow on Cygwin, but
compared to other people's machines, eg when building, it seems way
slower for me.

First I'd like to know if there's a good way to measure this that anyone
has found, because I'm not sure how to measure it. If I print multiple
lines with echo in a script, I can see it printing maybe 2-3 a second -
it's very slow.

I think this might be because I'm using a Virtual Machine with
VirtualBox, and QEMU/KVM might be quicker. I'm using Avira Antivurus,
with exceptions for the cygwin install folders (C:\cygwin64, C:\cygwin).

It might be nice if we could so some comparisons so I can figure out
what's wrong.


Running strace on your forking executable should give you accurate numbers in 
the output, with some work to extract the relevant values.



Same AV here, W10 64bit (no VM), 2 year old Laptop

model name  : Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
4 cores

https://github.com/mondalaci/fork-benchmark
it seems there is a aging effect

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 39.928576 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 42.701295 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 49.890909 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 61.657031 seconds.


It's all part of your current process tree.
Running cygserver may help this with appropriate process settings:

$ grep -C1 'kern\.srv\..*_.*' /etc/cygserver.conf

# kern.srv.cleanup_threads: No. of cygserver threads used for cleanup tasks.
# Default: 2, Min: 1, Max: 16, command line option -c, --cleanup-threads
#kern.srv.cleanup_threads 2
kern.srv.cleanup_threads 16

# kern.srv.request_threads: No. of cygserver threads used to serve
#   application requests.
# Default: 10, Min: 1, Max: 310, command line option -r, --request-threads
#kern.srv.request_threads 10
kern.srv.request_threads 310

# kern.srv.process_cache_size: No. of concurrent processes which can be handled
#  by Cygserver concurrently.
# Default: 62, Min: 1, Max: 310, command line option -p, --process-cache
#kern.srv.process_cache_size 62
kern.srv.process_cache_size 310

$ ./fork-benchmark 1000
Forked, executed and destroyed 1000 processes in 33.397727 seconds.
$ ./fork-benchmark 1000
Forked, executed and destroyed 1000 processes in 34.70389 seconds.
$ ./fork-benchmark 1000
Forked, executed and destroyed 1000 processes in 34.186709 seconds.
$ ./fork-benchmark 1000
Forked, executed and destroyed 1000 processes in 33.65649 seconds.
$ sed -En '/^model name|^cpu MHz/p;/MHz/q' /proc/cpuinfo
model name  : AMD A10-9700 RADEON R7, 10 COMPUTE CORES 4C+6G
cpu MHz : 3500.000

$ strace -o fork-benchmark.strace ./fork-benchmark 10
$ egrep '^--- Process [0-9]+ (crea|\(pid: [0-9]+\) exi)ted|dwProcessId 
[0-9]+|ExitProcess n 0x' fork-benchmark.strace > fork-benchmark.log

$ awk '/ [0-9]+! pinfo::exit: /{t+=$2};END{print t/1"ms"}' 
fork-benchmark.log
34.1855ms

Faster CPUs, faster memory, bigger caches, SSD drive may help.

--
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

This email may be disturbing to some readers as it contains
too much technical detail. Reader discretion is advised.
[Data in binary units and prefixes, physical quantities in SI.]
--- Process 9704 created
   65   18385 [main] fork-benchmark 13592 pinfo::thisproc: myself dwProcessId 
9704
--- Process 6712 created
   797102 [main] fork-benchmark 13593 pinfo::thisproc: myself dwProcessId 
6712
--- Process 11804 created
   755642 [main] fork-benchmark 13593 pinfo::thisproc: myself dwProcessId 
11804
   51   34992 [main] fork-benchmark 13593! pinfo::exit: Calling ExitProcess n 
0x400, exitcode 0x0
   41   11124 [main] fork-benchmark 13593 pinfo::exit: Calling ExitProcess n 
0x0, exitcode 0x0
--- Process 11804 (pid: 13593) exited with status 0x0
--- Process 4176 created
   686571 [main] fork-benchmark 13594 pinfo::thisproc: myself dwProcessId 
4176
--- Process 1924 created
   775786 [main] fork-benchmark 13594 pinfo::thisproc: myself dwProcessId 
1924
   51   33582 [main] fork-benchmark 13594! pinfo::exit: Calling ExitProcess n 
0x400, exitcode 0x0
   40   10465 [main] fork-benchmark 13594 pinfo::exit: Calling ExitProcess n 
0x0, exitcode 0x0
--- Process 1924 (pid: 13594) exited with status 0x0
--- Process 10576 created
   646030 [main] fork-benchmark 13595 pinfo::thisproc: myself dwProcessId 
10576
--- Process 6932 created
   765798 [main] fork-benchmark 13595 pinfo::thisproc: myself dwProcessId 
6932
   42   31674 [main] fork-benchmark 13595! pinfo::exit: Calling ExitProcess n 
0x400, exitcode 0x0
   41   10602 [main] fork-benchmark 13595 pinfo::exit: Calling ExitProcess n 

Re: Optimising cygwin fork performance

[Christian Franke]

Marco Atzeri via Cygwin-apps wrote:


On 16.12.2020 13:13, Hamish McIntyre-Bhatty via Cygwin-apps wrote:

Hi,

So I know it's been mentioned a lot that fork is slow on Cygwin, but
compared to other people's machines, eg when building, it seems way
slower for me.

First I'd like to know if there's a good way to measure this that anyone
has found, because I'm not sure how to measure it. If I print multiple
lines with echo in a script, I can see it printing maybe 2-3 a second -
it's very slow.

I think this might be because I'm using a Virtual Machine with
VirtualBox, and QEMU/KVM might be quicker. I'm using Avira Antivurus,
with exceptions for the cygwin install folders (C:\cygwin64, C:\cygwin).

It might be nice if we could so some comparisons so I can figure out
what's wrong.

Hamish


Same AV here, W10 64bit (no VM), 2 year old Laptop

model name  : Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
4 cores

https://github.com/mondalaci/fork-benchmark
it seems there is a aging effect


Result of power management? Could possibly be prevented if another minor 
cpu load is run in parallel.





$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 39.928576 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 42.701295 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 49.890909 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 61.657031 seconds.





9 year old PC, W10 64bit, no VM, Intel i7-2600K CPU @ 3.4GHz, 4 cores / 
8 threads


AV: Windows Defender
Cygwin x86_64

Protection on:

$ ./fork-benchmark-64 1000
Forked, executed and destroyed 1000 processes in 16.651101 seconds.

$ ./fork-benchmark-64 1000
Forked, executed and destroyed 1000 processes in 16.674107 seconds.

Protection off:

$ ./fork-benchmark-64 1000
Forked, executed and destroyed 1000 processes in 14.281071 seconds.

$ ./fork-benchmark-64 1000
Forked, executed and destroyed 1000 processes in 14.426482 seconds.


An alternative which could be run out-of-the-box is the good old 
'date(s) per second' benchmark. Its results are comparable:


$ while true; do date; done | uniq -c
...
 53 Wed Dec 16 19:23:26 CET 2020   <== Protection on
 54 Wed Dec 16 19:23:27 CET 2020
 56 Wed Dec 16 19:23:28 CET 2020
 55 Wed Dec 16 19:23:29 CET 2020
 56 Wed Dec 16 19:23:30 CET 2020
...
 52 Wed Dec 16 19:23:37 CET 2020
 54 Wed Dec 16 19:23:38 CET 2020
 56 Wed Dec 16 19:23:39 CET 2020
 63 Wed Dec 16 19:23:40 CET 2020   <== Protection off
 63 Wed Dec 16 19:23:41 CET 2020
 62 Wed Dec 16 19:23:42 CET 2020
 64 Wed Dec 16 19:23:43 CET 2020
...
 63 Wed Dec 16 19:23:51 CET 2020
 64 Wed Dec 16 19:23:52 CET 2020
 63 Wed Dec 16 19:23:53 CET 2020
 55 Wed Dec 16 19:23:54 CET 2020   <== Protection on
 48 Wed Dec 16 19:23:55 CET 2020
 53 Wed Dec 16 19:23:56 CET 2020
 54 Wed Dec 16 19:23:57 CET 2020
 54 Wed Dec 16 19:23:58 CET 2020
...


Cygwin x86 is somewhat slower:

Protection on:

$ ./fork-benchmark-32.exe 1000
Forked, executed and destroyed 1000 processes in 19.231766 seconds.

Protection off:

$ ./fork-benchmark-32.exe 1000
Forked, executed and destroyed 1000 processes in 17.107739 seconds.

Regards,
Christian

Re: Optimising cygwin fork performance

[Marco Atzeri via Cygwin-apps]

On 16.12.2020 13:13, Hamish McIntyre-Bhatty via Cygwin-apps wrote:

Hi,

So I know it's been mentioned a lot that fork is slow on Cygwin, but
compared to other people's machines, eg when building, it seems way
slower for me.

First I'd like to know if there's a good way to measure this that anyone
has found, because I'm not sure how to measure it. If I print multiple
lines with echo in a script, I can see it printing maybe 2-3 a second -
it's very slow.

I think this might be because I'm using a Virtual Machine with
VirtualBox, and QEMU/KVM might be quicker. I'm using Avira Antivurus,
with exceptions for the cygwin install folders (C:\cygwin64, C:\cygwin).

It might be nice if we could so some comparisons so I can figure out
what's wrong.

Hamish


Same AV here, W10 64bit (no VM), 2 year old Laptop

model name  : Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz
4 cores

https://github.com/mondalaci/fork-benchmark
it seems there is a aging effect

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 39.928576 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 42.701295 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 49.890909 seconds.

$ ./fork-benchmark.exe 1000
Forked, executed and destroyed 1000 processes in 61.657031 seconds.

Optimising cygwin fork performance

[Hamish McIntyre-Bhatty via Cygwin-apps]

Hi,

So I know it's been mentioned a lot that fork is slow on Cygwin, but
compared to other people's machines, eg when building, it seems way
slower for me.

First I'd like to know if there's a good way to measure this that anyone
has found, because I'm not sure how to measure it. If I print multiple
lines with echo in a script, I can see it printing maybe 2-3 a second -
it's very slow.

I think this might be because I'm using a Virtual Machine with
VirtualBox, and QEMU/KVM might be quicker. I'm using Avira Antivurus,
with exceptions for the cygwin install folders (C:\cygwin64, C:\cygwin).

It might be nice if we could so some comparisons so I can figure out
what's wrong.

Hamish



0x87B761FE07F548D6.asc
Description: application/pgp-keys


signature.asc
Description: OpenPGP digital signature

[newlib-cygwin] Cygwin: fork: attach child not before success

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=2986a524d853ab6d06bf88b65e0e9d60cea51dcc

commit 2986a524d853ab6d06bf88b65e0e9d60cea51dcc
Author: Michael Haubenwallner 
Date:   Wed Jul 31 12:35:33 2019 +0200

Cygwin: fork: attach child not before success

Do not attach to the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like suppressing SIGCHILD delivery with multiple threads ("waitproc")
involved.

Improves "Cygwin: fork: Remember child not before success.",
commit f03ea8e1c57bd5cea83f6cd47fa02870bdfeb1c5, which leads to fork
problems if cygserver is running:

https://cygwin.com/ml/cygwin-patches/2019-q2/msg00155.html

Diff:
---
 winsup/cygwin/fork.cc | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 0119581..7080144 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -424,14 +424,6 @@ frok::parent (volatile char * volatile stack_here)
 #endif
   goto cleanup;
 }
-  if (!child.reattach ())
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child reattach failed");
-#endif
-  goto cleanup;
-}
 
   /* CHILD IS STOPPED */
   debug_printf ("child is alive (but stopped)");
@@ -516,6 +508,17 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
+  /* Do not attach to the child before it has successfully initialized.
+ Otherwise we may wait forever, or deliver an orphan SIGCHILD. */
+  if (!child.reattach ())
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child reattach failed");
+#endif
+  goto cleanup;
+}
+
   /* Finally start the child up. */
   resume_child (forker_finished);

[PATCH v2 2/2] Cygwin: fork: attach child not before success

[Michael Haubenwallner]

Do not attach to the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like suppressing SIGCHILD delivery with multiple threads ("waitproc")
involved.

Improves "Cygwin: fork: Remember child not before success.",
commit f03ea8e1c57bd5cea83f6cd47fa02870bdfeb1c5, which leads to fork
problems if cygserver is running:

https://cygwin.com/ml/cygwin-patches/2019-q2/msg00155.html
---
 winsup/cygwin/fork.cc | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 0119581df..7080144b9 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -421,14 +421,6 @@ frok::parent (volatile char * volatile stack_here)
   this_errno = EAGAIN;
 #ifdef DEBUGGING0
   error ("child remember failed");
-#endif
-  goto cleanup;
-}
-  if (!child.reattach ())
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child reattach failed");
 #endif
   goto cleanup;
 }
@@ -516,6 +508,17 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
+  /* Do not attach to the child before it has successfully initialized.
+ Otherwise we may wait forever, or deliver an orphan SIGCHILD. */
+  if (!child.reattach ())
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child reattach failed");
+#endif
+  goto cleanup;
+}
+
   /* Finally start the child up. */
   resume_child (forker_finished);
 
-- 
2.21.0

[PATCH 2/2] Cygwin: fork: Attach child not before success.

[Michael Haubenwallner]

Do not attach to the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like suppressing SIGCHILD delivery with multiple threads ("waitproc")
involved.

Improves "Cygwin: fork: Remember child not before success.",
commit f03ea8e1c57bd5cea83f6cd47fa02870bdfeb1c5, which leads to fork
problems if cygserver is running:

https://cygwin.com/ml/cygwin-patches/2019-q2/msg00155.html
---
 winsup/cygwin/fork.cc | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 59b13806c..c6ba8cf65 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -416,11 +416,11 @@ frok::parent (volatile char * volatile stack_here)
  it in afterwards.  This requires more bookkeeping than I like, though,
  so we'll just do it the easy way.  So, terminate any child process if
  we can't actually record the pid in the internal table. */
-  if (!child.remember (false))
+  if (!child.remember_without_attach (false))
 {
   this_errno = EAGAIN;
 #ifdef DEBUGGING0
-  error ("child remember failed");
+  error ("child remember_without_attach failed");
 #endif
   goto cleanup;
 }
@@ -508,6 +508,15 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
+  if (!child.reattach ())
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child reattach failed");
+#endif
+  goto cleanup;
+}
+
   /* Finally start the child up. */
   resume_child (forker_finished);
 
-- 
2.21.0

[newlib-cygwin] Revert "Cygwin: fork: Remember child not before success."

[Ken Brown]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=000f2409b118e85daaa7064b5375245b8f48ab27

commit 000f2409b118e85daaa7064b5375245b8f48ab27
Author: Ken Brown 
Date:   Tue Jun 11 08:11:01 2019 -0400

Revert "Cygwin: fork: Remember child not before success."

This reverts commit f03ea8e1c57bd5cea83f6cd47fa02870bdfeb1c5.  That
commit leads to fork problems if cygserver is running:

https://cygwin.com/ml/cygwin-patches/2019-q2/msg00155.html

Diff:
---
 winsup/cygwin/fork.cc | 31 +++
 1 file changed, 15 insertions(+), 16 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index c69081f..59b1380 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -181,8 +181,7 @@ frok::child (volatile char * volatile here)
   cygheap->fdtab.fixup_after_fork (hParent);
 
   /* Signal that we have successfully initialized, so the parent can
- - transfer data/bss for dynamically loaded dlls (if any), and
- - start up some tracker threads to remember the child, or
+ - transfer data/bss for dynamically loaded dlls (if any), or
  - terminate the current fork call even if the child is initialized. */
   sync_with_parent ("performed fork fixups and dynamic dll loading", true);
 
@@ -412,6 +411,20 @@ frok::parent (volatile char * volatile stack_here)
   child.hProcess = hchild;
   ch.postfork (child);
 
+  /* Hopefully, this will succeed.  The alternative to doing things this
+ way is to reserve space prior to calling CreateProcess and then fill
+ it in afterwards.  This requires more bookkeeping than I like, though,
+ so we'll just do it the easy way.  So, terminate any child process if
+ we can't actually record the pid in the internal table. */
+  if (!child.remember (false))
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child remember failed");
+#endif
+  goto cleanup;
+}
+
   /* CHILD IS STOPPED */
   debug_printf ("child is alive (but stopped)");
 
@@ -495,20 +508,6 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
-  /* Hopefully, this will succeed.  The alternative to doing things this
- way is to reserve space prior to calling CreateProcess and then fill
- it in afterwards.  This requires more bookkeeping than I like, though,
- so we'll just do it the easy way.  So, terminate any child process if
- we can't actually record the pid in the internal table. */
-  if (!child.remember (false))
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child remember failed");
-#endif
-  goto cleanup;
-}
-
   /* Finally start the child up. */
   resume_child (forker_finished);

Re: [PATCH 1/2] Cygwin: fork: Always pause child after fixups.

[Corinna Vinschen]

On Apr 30 09:09, Michael Haubenwallner wrote:
> Pause the child process after performing fork fixups even if there were
> no dynamically loaded dlls with extra data/bss transfers to wait for.
> This allows the parent process to cancel the current fork call even if
> the child process was successfully initialized already.
> 
> This is a preparation for when the parent does remember the child no
> earlier than after successful child initialization.

Patchset pushed.


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer


signature.asc
Description: PGP signature

[newlib-cygwin] Cygwin: fork: Remember child not before success.

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=f03ea8e1c57bd5cea83f6cd47fa02870bdfeb1c5

commit f03ea8e1c57bd5cea83f6cd47fa02870bdfeb1c5
Author: Michael Haubenwallner 
Date:   Thu May 2 12:12:44 2019 +0200

Cygwin: fork: Remember child not before success.

Do not remember the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like cleaning up the process table and suppressing SIGCHILD delivery
with multiple threads ("waitproc") involved.  Compared to that, the
potential slowdown due to an extra yield () call should be negligible.

Diff:
---
 winsup/cygwin/fork.cc | 31 ---
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 59b1380..c69081f 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -181,7 +181,8 @@ frok::child (volatile char * volatile here)
   cygheap->fdtab.fixup_after_fork (hParent);
 
   /* Signal that we have successfully initialized, so the parent can
- - transfer data/bss for dynamically loaded dlls (if any), or
+ - transfer data/bss for dynamically loaded dlls (if any), and
+ - start up some tracker threads to remember the child, or
  - terminate the current fork call even if the child is initialized. */
   sync_with_parent ("performed fork fixups and dynamic dll loading", true);
 
@@ -411,20 +412,6 @@ frok::parent (volatile char * volatile stack_here)
   child.hProcess = hchild;
   ch.postfork (child);
 
-  /* Hopefully, this will succeed.  The alternative to doing things this
- way is to reserve space prior to calling CreateProcess and then fill
- it in afterwards.  This requires more bookkeeping than I like, though,
- so we'll just do it the easy way.  So, terminate any child process if
- we can't actually record the pid in the internal table. */
-  if (!child.remember (false))
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child remember failed");
-#endif
-  goto cleanup;
-}
-
   /* CHILD IS STOPPED */
   debug_printf ("child is alive (but stopped)");
 
@@ -508,6 +495,20 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
+  /* Hopefully, this will succeed.  The alternative to doing things this
+ way is to reserve space prior to calling CreateProcess and then fill
+ it in afterwards.  This requires more bookkeeping than I like, though,
+ so we'll just do it the easy way.  So, terminate any child process if
+ we can't actually record the pid in the internal table. */
+  if (!child.remember (false))
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child remember failed");
+#endif
+  goto cleanup;
+}
+
   /* Finally start the child up. */
   resume_child (forker_finished);

[newlib-cygwin] Cygwin: fork: Always pause child after fixups.

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=a8c23e4423ebb3458db93341d8a9bc73b0a04b03

commit a8c23e4423ebb3458db93341d8a9bc73b0a04b03
Author: Michael Haubenwallner 
Date:   Tue Apr 30 09:09:13 2019 +0200

Cygwin: fork: Always pause child after fixups.

Pause the child process after performing fork fixups even if there were
no dynamically loaded dlls with extra data/bss transfers to wait for.
This allows the parent process to cancel the current fork call even if
the child process was successfully initialized already.

This is a preparation for when the parent does remember the child no
earlier than after successful child initialization.

Diff:
---
 winsup/cygwin/fork.cc | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 7e1c089..59b1380 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -180,13 +180,10 @@ frok::child (volatile char * volatile here)
 
   cygheap->fdtab.fixup_after_fork (hParent);
 
-  /* If we haven't dynamically loaded any dlls, just signal the parent.
- Otherwise, tell the parent that we've loaded all the dlls
- and wait for the parent to fill in the loaded dlls' data/bss. */
-  if (!load_dlls)
-sync_with_parent ("performed fork fixup", false);
-  else
-sync_with_parent ("loaded dlls", true);
+  /* Signal that we have successfully initialized, so the parent can
+ - transfer data/bss for dynamically loaded dlls (if any), or
+ - terminate the current fork call even if the child is initialized. */
+  sync_with_parent ("performed fork fixups and dynamic dll loading", true);
 
   init_console_handler (myself->ctty > 0);
   ForceCloseHandle1 (fork_info->forker_finished, forker_finished);
@@ -477,7 +474,8 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
-  /* Start thread, and then wait for it to reload dlls.  */
+  /* Start the child up, and then wait for it to
+ perform fork fixups and dynamic dll loading (if any). */
   resume_child (forker_finished);
   if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
 {
@@ -508,10 +506,11 @@ frok::parent (volatile char * volatile stack_here)
  goto cleanup;
}
}
-  /* Start the child up again. */
-  resume_child (forker_finished);
 }
 
+  /* Finally start the child up. */
+  resume_child (forker_finished);
+
   ForceCloseHandle (forker_finished);
   forker_finished = NULL;

Re: [PATCH 2/2] Cygwin: fork: Remember child not before success.

[Michael Haubenwallner]

On 4/30/19 6:07 PM, Corinna Vinschen wrote:
> On Apr 30 09:09, Michael Haubenwallner wrote:
>> Do not remember the child before it was successfully initialized, or we
>> would need more sophisticated cleanup on child initialization failure,
>> like cleaning up the process table and suppressing SIGCHILD delivery
>> with multiple threads ("waitproc") involved.
>> ---
>>  winsup/cygwin/fork.cc | 33 ++---
>>  1 file changed, 18 insertions(+), 15 deletions(-)
>> [...]
>> +  yield (); /* For child.remember (), to perform async thread startup. */
> 
> Is that really necessary?  What's that fixing and what effect does this
> have on the performance of the already very slow fork()?

Indeed, that's needless.  Testing around the original patch in this thread
did lead me to the idea that the "waitproc" cygthread is started using
QueueUserAPC and async_startup, being executed by WFMO in ch.sync, which
I've reduced to yield later on.  But apparently I've been wrong here and the
waitproc thread is started synchronously.  Performance wise, I found it
negligible, but admitted I didn't perform the full >24h build.

Patch updated.

Thanks!
/haubi/
>From 69c21e8e5d19cc6ec205a4e44d6b84542b1fef98 Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner 
Date: Mon, 29 Apr 2019 16:03:51 +0200
Subject: [PATCH] Cygwin: fork: Remember child not before success.

Do not remember the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like cleaning up the process table and suppressing SIGCHILD delivery
with multiple threads ("waitproc") involved.  Compared to that, the
potential slowdown due to an extra yield () call should be negligible.
---
 winsup/cygwin/fork.cc | 31 ---
 1 file changed, 16 insertions(+), 15 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 59b13806c..c69081fc0 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -181,7 +181,8 @@ frok::child (volatile char * volatile here)
   cygheap->fdtab.fixup_after_fork (hParent);
 
   /* Signal that we have successfully initialized, so the parent can
- - transfer data/bss for dynamically loaded dlls (if any), or
+ - transfer data/bss for dynamically loaded dlls (if any), and
+ - start up some tracker threads to remember the child, or
  - terminate the current fork call even if the child is initialized. */
   sync_with_parent ("performed fork fixups and dynamic dll loading", true);
 
@@ -411,20 +412,6 @@ frok::parent (volatile char * volatile stack_here)
   child.hProcess = hchild;
   ch.postfork (child);
 
-  /* Hopefully, this will succeed.  The alternative to doing things this
- way is to reserve space prior to calling CreateProcess and then fill
- it in afterwards.  This requires more bookkeeping than I like, though,
- so we'll just do it the easy way.  So, terminate any child process if
- we can't actually record the pid in the internal table. */
-  if (!child.remember (false))
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child remember failed");
-#endif
-  goto cleanup;
-}
-
   /* CHILD IS STOPPED */
   debug_printf ("child is alive (but stopped)");
 
@@ -508,6 +495,20 @@ frok::parent (volatile char * volatile stack_here)
 	}
 }
 
+  /* Hopefully, this will succeed.  The alternative to doing things this
+ way is to reserve space prior to calling CreateProcess and then fill
+ it in afterwards.  This requires more bookkeeping than I like, though,
+ so we'll just do it the easy way.  So, terminate any child process if
+ we can't actually record the pid in the internal table. */
+  if (!child.remember (false))
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child remember failed");
+#endif
+  goto cleanup;
+}
+
   /* Finally start the child up. */
   resume_child (forker_finished);
 
-- 
2.19.2

Re: [PATCH 2/2] Cygwin: fork: Remember child not before success.

[Corinna Vinschen]

Hi Michael,

On Apr 30 09:09, Michael Haubenwallner wrote:
> Do not remember the child before it was successfully initialized, or we
> would need more sophisticated cleanup on child initialization failure,
> like cleaning up the process table and suppressing SIGCHILD delivery
> with multiple threads ("waitproc") involved.
> ---
>  winsup/cygwin/fork.cc | 33 ++---
>  1 file changed, 18 insertions(+), 15 deletions(-)
> [...]
> +  yield (); /* For child.remember (), to perform async thread startup. */

Is that really necessary?  What's that fixing and what effect does this
have on the performance of the already very slow fork()?


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer


signature.asc
Description: PGP signature

[PATCH 1/2] Cygwin: fork: Always pause child after fixups.

[Michael Haubenwallner]

Pause the child process after performing fork fixups even if there were
no dynamically loaded dlls with extra data/bss transfers to wait for.
This allows the parent process to cancel the current fork call even if
the child process was successfully initialized already.

This is a preparation for when the parent does remember the child no
earlier than after successful child initialization.
---
 winsup/cygwin/fork.cc | 19 +--
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 7e1c08990..59b13806c 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -180,13 +180,10 @@ frok::child (volatile char * volatile here)
 
   cygheap->fdtab.fixup_after_fork (hParent);
 
-  /* If we haven't dynamically loaded any dlls, just signal the parent.
- Otherwise, tell the parent that we've loaded all the dlls
- and wait for the parent to fill in the loaded dlls' data/bss. */
-  if (!load_dlls)
-sync_with_parent ("performed fork fixup", false);
-  else
-sync_with_parent ("loaded dlls", true);
+  /* Signal that we have successfully initialized, so the parent can
+ - transfer data/bss for dynamically loaded dlls (if any), or
+ - terminate the current fork call even if the child is initialized. */
+  sync_with_parent ("performed fork fixups and dynamic dll loading", true);
 
   init_console_handler (myself->ctty > 0);
   ForceCloseHandle1 (fork_info->forker_finished, forker_finished);
@@ -477,7 +474,8 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
-  /* Start thread, and then wait for it to reload dlls.  */
+  /* Start the child up, and then wait for it to
+ perform fork fixups and dynamic dll loading (if any). */
   resume_child (forker_finished);
   if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
 {
@@ -508,10 +506,11 @@ frok::parent (volatile char * volatile stack_here)
  goto cleanup;
}
}
-  /* Start the child up again. */
-  resume_child (forker_finished);
 }
 
+  /* Finally start the child up. */
+  resume_child (forker_finished);
+
   ForceCloseHandle (forker_finished);
   forker_finished = NULL;
 
-- 
2.19.2

[PATCH 2/2] Cygwin: fork: Remember child not before success.

[Michael Haubenwallner]

Do not remember the child before it was successfully initialized, or we
would need more sophisticated cleanup on child initialization failure,
like cleaning up the process table and suppressing SIGCHILD delivery
with multiple threads ("waitproc") involved.
---
 winsup/cygwin/fork.cc | 33 ++---
 1 file changed, 18 insertions(+), 15 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 59b13806c..05d5bb915 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -181,7 +181,8 @@ frok::child (volatile char * volatile here)
   cygheap->fdtab.fixup_after_fork (hParent);
 
   /* Signal that we have successfully initialized, so the parent can
- - transfer data/bss for dynamically loaded dlls (if any), or
+ - transfer data/bss for dynamically loaded dlls (if any), and
+ - start up some tracker threads to remember the child, or
  - terminate the current fork call even if the child is initialized. */
   sync_with_parent ("performed fork fixups and dynamic dll loading", true);
 
@@ -411,20 +412,6 @@ frok::parent (volatile char * volatile stack_here)
   child.hProcess = hchild;
   ch.postfork (child);
 
-  /* Hopefully, this will succeed.  The alternative to doing things this
- way is to reserve space prior to calling CreateProcess and then fill
- it in afterwards.  This requires more bookkeeping than I like, though,
- so we'll just do it the easy way.  So, terminate any child process if
- we can't actually record the pid in the internal table. */
-  if (!child.remember (false))
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child remember failed");
-#endif
-  goto cleanup;
-}
-
   /* CHILD IS STOPPED */
   debug_printf ("child is alive (but stopped)");
 
@@ -508,12 +495,28 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
+  /* Hopefully, this will succeed.  The alternative to doing things this
+ way is to reserve space prior to calling CreateProcess and then fill
+ it in afterwards.  This requires more bookkeeping than I like, though,
+ so we'll just do it the easy way.  So, terminate any child process if
+ we can't actually record the pid in the internal table. */
+  if (!child.remember (false))
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child remember failed");
+#endif
+  goto cleanup;
+}
+
   /* Finally start the child up. */
   resume_child (forker_finished);
 
   ForceCloseHandle (forker_finished);
   forker_finished = NULL;
 
+  yield (); /* For child.remember (), to perform async thread startup. */
+
   return child_pid;
 
 /* Common cleanup code for failure cases */
-- 
2.19.2

Re: [PATCH] Cygwin: fork: remember child as late as possible

[Corinna Vinschen]

On Apr 12 15:31, Michael Haubenwallner wrote:
> Otherwise, when the child does fail to reload dlls and terminates, we
> produce a SIGCHILD signal, even if we did not succeed in starting up the
> child process at all.  Also, we would need to reap that child somewhere.

I'm not that happy with the patch.  The patch is doing two things, one
is to remember child as late as possible, the other is to change the
child startup in case the parent loaded dlls.

Either write a more descriptive commit message or split this into two
patches.  I'd prefer two patches.  It may help debugging.


Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Maintainer


signature.asc
Description: PGP signature

[PATCH] Cygwin: fork: remember child as late as possible

[Michael Haubenwallner]

Otherwise, when the child does fail to reload dlls and terminates, we
produce a SIGCHILD signal, even if we did not succeed in starting up the
child process at all.  Also, we would need to reap that child somewhere.
---
 winsup/cygwin/fork.cc | 71 ---
 1 file changed, 40 insertions(+), 31 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 74ee9acf4..a5b45f851 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -186,14 +186,14 @@ frok::child (volatile char * volatile here)
 
   cygheap->fdtab.fixup_after_fork (hParent);
 
-  /* If we haven't dynamically loaded any dlls, just signal the parent.
- Otherwise, tell the parent that we've loaded all the dlls
- and wait for the parent to fill in the loaded dlls' data/bss. */
-  if (!load_dlls)
-sync_with_parent ("performed fork fixup", false);
-  else
+  /* If we have dynamically loaded some dlls, we need anoter stop to
+ wait for the parent to fill in the loaded dll's data/bss. */
+  if (load_dlls)
 sync_with_parent ("loaded dlls", true);
 
+  /* Signal the parent. */
+  sync_with_parent ("performed fork fixup", false);
+
   init_console_handler (myself->ctty > 0);
   ForceCloseHandle1 (fork_info->forker_finished, forker_finished);
 
@@ -420,20 +420,6 @@ frok::parent (volatile char * volatile stack_here)
   child.hProcess = hchild;
   ch.postfork (child);
 
-  /* Hopefully, this will succeed.  The alternative to doing things this
- way is to reserve space prior to calling CreateProcess and then fill
- it in afterwards.  This requires more bookkeeping than I like, though,
- so we'll just do it the easy way.  So, terminate any child process if
- we can't actually record the pid in the internal table. */
-  if (!child.remember (false))
-{
-  this_errno = EAGAIN;
-#ifdef DEBUGGING0
-  error ("child remember failed");
-#endif
-  goto cleanup;
-}
-
   /* CHILD IS STOPPED */
   debug_printf ("child is alive (but stopped)");
 
@@ -483,20 +469,20 @@ frok::parent (volatile char * volatile stack_here)
}
 }
 
-  /* Start thread, and then wait for it to reload dlls.  */
-  resume_child (forker_finished);
-  if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
-{
-  this_errno = EAGAIN;
-  error ("died waiting for dll loading");
-  goto cleanup;
-}
-
   /* If DLLs were loaded in the parent, then the child has reloaded all
  of them and is now waiting to have all of the individual data and
  bss sections filled in. */
   if (load_dlls)
 {
+  /* Start the child up, and then wait for it to reload dlls.  */
+  resume_child (forker_finished);
+  if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
+   {
+ this_errno = EAGAIN;
+ error ("died waiting for dll loading");
+ goto cleanup;
+   }
+
   /* CHILD IS STOPPED */
   /* write memory of reloaded dlls */
   for (dll *d = dlls.istart (DLL_LOAD); d; d = dlls.inext ())
@@ -514,8 +500,31 @@ frok::parent (volatile char * volatile stack_here)
  goto cleanup;
}
}
-  /* Start the child up again. */
-  resume_child (forker_finished);
+}
+
+  /* Hopefully, this will succeed.  The alternative to doing things this
+ way is to reserve space prior to calling CreateProcess and then fill
+ it in afterwards.  This requires more bookkeeping than I like, though,
+ so we'll just do it the easy way.  So, terminate any child process if
+ we can't actually record the pid in the internal table.
+ Note: child.remember () needs the subsequent WFMO in ch.sync (),
+ to perform the asynchronous start of the controlling threads. */
+  if (!child.remember (false))
+{
+  this_errno = EAGAIN;
+#ifdef DEBUGGING0
+  error ("child remember failed");
+#endif
+  goto cleanup;
+}
+
+  /* Start the child up, finally. */
+  resume_child (forker_finished);
+  if (!ch.sync (child->pid, hchild, FORK_WAIT_TIMEOUT))
+{
+  this_errno = EAGAIN;
+  error ("died waiting for dll loading");
+  goto cleanup;
 }
 
   ForceCloseHandle (forker_finished);
-- 
2.19.2

[newlib-cygwin] Cygwin: fork: reserve dynloaded dll areas earlier

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=023c107a22ebd2f540fa3f4888535aad32839fe2

commit 023c107a22ebd2f540fa3f4888535aad32839fe2
Author: Michael Haubenwallner 
Date:   Tue Mar 26 17:38:36 2019 +0100

Cygwin: fork: reserve dynloaded dll areas earlier

In dll_crt0_0, both threadinterface->Init and sigproc_init allocate
windows object handles using unpredictable memory regions, which may
collide with dynamically loaded dlls when they were relocated.

Diff:
---
 winsup/cygwin/dcrt0.cc | 6 ++
 winsup/cygwin/fork.cc  | 6 --
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/winsup/cygwin/dcrt0.cc b/winsup/cygwin/dcrt0.cc
index 11edcdf..fb726a7 100644
--- a/winsup/cygwin/dcrt0.cc
+++ b/winsup/cygwin/dcrt0.cc
@@ -632,6 +632,12 @@ child_info_fork::handle_fork ()
 
   if (fixup_mmaps_after_fork (parent))
 api_fatal ("recreate_mmaps_after_fork_failed");
+
+  /* We need to occupy the address space for dynamically loaded dlls
+ before we allocate any dynamic object, or we may end up with
+ error "address space needed by  is already occupied"
+ for no good reason (seen with some relocated dll). */
+  dlls.reserve_space ();
 }
 
 bool
diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 74ee9ac..7e1c089 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -136,12 +136,6 @@ frok::child (volatile char * volatile here)
 {
   HANDLE& hParent = ch.parent;
 
-  /* NOTE: Logically this belongs in dll_list::load_after_fork, but by
- doing it here, before the first sync_with_parent, we can exploit
- the existing retry mechanism in hopes of getting a more favorable
- address space layout next time. */
-  dlls.reserve_space ();
-
   sync_with_parent ("after longjmp", true);
   debug_printf ("child is running.  pid %d, ppid %d, stack here %p",
myself->pid, myself->ppid, __builtin_frame_address (0));

[newlib-cygwin] Cygwin: fork/exec: Allow all users PROCESS_QUERY_LIMITED_INFORMATION

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=24f9cb015ea011b43f9c4b865c98b61be731487f

commit 24f9cb015ea011b43f9c4b865c98b61be731487f
Author: Corinna Vinschen 
Date:   Tue Mar 12 11:41:35 2019 +0100

Cygwin: fork/exec: Allow all users PROCESS_QUERY_LIMITED_INFORMATION

Create process with standard rights, plus
PROCESS_QUERY_LIMITED_INFORMATION for authenticated users.  This
allows to fetch basic process information and thus /proc//stat
to succeed on foreign processes.

While at it, fix formatting in CreateProcess calls.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/fork.cc  | 17 -
 winsup/cygwin/spawn.cc | 30 ++
 2 files changed, 30 insertions(+), 17 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 7ae0404..74ee9ac 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -318,6 +318,13 @@ frok::parent (volatile char * volatile stack_here)
 
   ch.silentfail (!*with_forkables); /* fail silently without forkables */
 
+  tmp_pathbuf tp;
+  PSECURITY_ATTRIBUTES sa = (PSECURITY_ATTRIBUTES) tp.w_get ();
+  if (!sec_user_nih (sa, cygheap->user.saved_sid (),
+well_known_authenticated_users_sid,
+PROCESS_QUERY_LIMITED_INFORMATION))
+sa = _none_nih;
+
   while (1)
 {
   PCWCHAR forking_progname = NULL;
@@ -339,12 +346,12 @@ frok::parent (volatile char * volatile stack_here)
   sure child stack is allocated
   in the same memory location
   as in parent. */
-  _none_nih,
-  _none_nih,
-  TRUE,/* inherit handles from parent 
*/
+  sa,
+  sa,
+  TRUE,/* inherit handles */
   c_flags,
-  NULL,/* environment filled in later 
*/
-  0,   /* use current drive/directory 
*/
+  NULL,/* environ filled in later */
+  0,   /* use cwd */
   ,
   );
 
diff --git a/winsup/cygwin/spawn.cc b/winsup/cygwin/spawn.cc
index ebc34d1..4e549f7 100644
--- a/winsup/cygwin/spawn.cc
+++ b/winsup/cygwin/spawn.cc
@@ -566,6 +566,12 @@ child_info_spawn::worker (const char *prog_arg, const char 
*const *argv,
SetHandleInformation (my_wr_proc_pipe, HANDLE_FLAG_INHERIT, 0);
   parent_winpid = GetCurrentProcessId ();
 
+  PSECURITY_ATTRIBUTES sa = (PSECURITY_ATTRIBUTES) tp.w_get ();
+  if (!sec_user_nih (sa, cygheap->user.sid (),
+well_known_authenticated_users_sid,
+PROCESS_QUERY_LIMITED_INFORMATION))
+   sa = _none_nih;
+
 loop:
   /* When ruid != euid we create the new process under the current original
 account and impersonate in child, this way maintaining the different
@@ -586,13 +592,13 @@ child_info_spawn::worker (const char *prog_arg, const 
char *const *argv,
  && !::cygheap->user.groups.issetgroups ()
  && !::cygheap->user.setuid_to_restricted))
{
- rc = CreateProcessW (runpath,   /* image name - with full path */
-  cmd.wcs (wcmd),/* what was passed to exec */
-  _none_nih, /* process security attrs */
-  _none_nih, /* thread security attrs */
-  TRUE,  /* inherit handles from parent */
+ rc = CreateProcessW (runpath, /* image name w/ full path */
+  cmd.wcs (wcmd),  /* what was passed to exec */
+  sa,  /* process security attrs */
+  sa,  /* thread security attrs */
+  TRUE,/* inherit handles */
   c_flags,
-  envblock,  /* environment */
+  envblock,/* environment */
   NULL,
   ,
   );
@@ -640,13 +646,13 @@ child_info_spawn::worker (const char *prog_arg, const 
char *const *argv,
}
 
  rc = CreateProcessAsUserW (::cygheap->user.primary_token (),
-  runpath,   /* image name - with full path */
-  cmd.wcs (wcmd),/* what was passed to exec */
-  _none_nih, /* process security attrs */
-  _none_nih, /* thread security attrs */
-

[newlib-cygwin] Cygwin: fork: fix child process permissions, take 3

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=38dde5f4c4c2e33dfe6dfb9d1bc593d13d85a290

commit 38dde5f4c4c2e33dfe6dfb9d1bc593d13d85a290
Author: Corinna Vinschen 
Date:   Sun Mar 3 10:59:13 2019 +0100

Cygwin: fork: fix child process permissions, take 3

Per MSDN VirtualQueryEx requires PROCESS_QUERY_INFORMATION.
Testing showed that PROCESS_QUERY_LIMITED_INFORMATION is sufficient
since Windows 8.1.  The assumption that Windows 8 is the same as
Windows 8 was not correct, it requires PROCESS_QUERY_INFORMATION
as well.

Fix that by splitting the Windows 8 wincaps into one for Windows 8
and one for Windows 8.1.  Set needs_query_information for Windows 8.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/wincap.cc | 29 -
 1 file changed, 28 insertions(+), 1 deletion(-)

diff --git a/winsup/cygwin/wincap.cc b/winsup/cygwin/wincap.cc
index 78cc411..5bc9c37 100644
--- a/winsup/cygwin/wincap.cc
+++ b/winsup/cygwin/wincap.cc
@@ -73,6 +73,31 @@ wincaps wincap_8 __attribute__((section 
(".cygwin_dll_common"), shared)) = {
   {
 is_server:false,
 needs_count_in_si_lpres2:false,
+needs_query_information:true,
+has_gaa_largeaddress_bug:false,
+has_broken_alloc_console:true,
+has_console_logon_sid:true,
+has_precise_system_time:true,
+has_microsoft_accounts:true,
+has_processor_groups:true,
+has_broken_prefetchvm:false,
+has_new_pebteb_region:false,
+has_broken_whoami:false,
+has_unprivileged_createsymlink:false,
+has_unbiased_interrupt_time:true,
+has_precise_interrupt_time:false,
+has_posix_unlink_semantics:false,
+has_case_sensitive_dirs:false,
+has_posix_rename_semantics:false,
+no_msv1_0_s4u_logon_in_wow64:false,
+  },
+};
+
+wincaps wincap_8_1 __attribute__((section (".cygwin_dll_common"), shared)) = {
+  def_guard_pages:2,
+  {
+is_server:false,
+needs_count_in_si_lpres2:false,
 needs_query_information:false,
 has_gaa_largeaddress_bug:false,
 has_broken_alloc_console:true,
@@ -273,9 +298,11 @@ wincapc::init ()
  caps = _7;
  break;
case 2:
-   case 3:
  caps = _8;
  break;
+   case 3:
+ caps = _8_1;
+ break;
default:
  caps = _10_1507;
  break;

[newlib-cygwin] Cygwin: fork: add PROCESS_VM_OPERATION to child process permissions

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=7e671e7578c9ab8a1144a10c1187b7ced984

commit 7e671e7578c9ab8a1144a10c1187b7ced984
Author: Corinna Vinschen 
Date:   Sun Feb 17 22:51:21 2019 +0100

Cygwin: fork: add PROCESS_VM_OPERATION to child process permissions

...on parent process.  This is required for successful mmap propagation.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/release/3.0.1 | 13 +
 winsup/cygwin/sigproc.cc|  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/winsup/cygwin/release/3.0.1 b/winsup/cygwin/release/3.0.1
new file mode 100644
index 000..ae6e03e
--- /dev/null
+++ b/winsup/cygwin/release/3.0.1
@@ -0,0 +1,13 @@
+What's new:
+---
+
+
+What changed:
+-
+
+
+Bug Fixes
+-
+
+- Relax fork child permissions to avoid a potential fork failure.
+  Addresses: https://cygwin.com/ml/cygwin/2019-02/msg00234.html
diff --git a/winsup/cygwin/sigproc.cc b/winsup/cygwin/sigproc.cc
index 30dfaaa..3b6492b 100644
--- a/winsup/cygwin/sigproc.cc
+++ b/winsup/cygwin/sigproc.cc
@@ -815,7 +815,7 @@ child_info::child_info (unsigned in_cb, child_info_types 
chtype,
  we're forking, we also need handle duplicate access. */
   parent = NULL;
   DWORD perms = PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_VM_READ
-   | SYNCHRONIZE;
+   | PROCESS_VM_OPERATION | SYNCHRONIZE;
   if (type == _CH_FORK)
 {
   perms |= PROCESS_DUP_HANDLE;

[newlib-cygwin] Cygwin: fork: terminate child process unconditionally in error case

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=7225a82c1a4e90bcea2a17da12a427d1e783de30

commit 7225a82c1a4e90bcea2a17da12a427d1e783de30
Author: Corinna Vinschen 
Date:   Tue Feb 5 12:52:13 2019 +0100

Cygwin: fork: terminate child process unconditionally in error case

Move TerminateProcess call into cleanup code to make sure child
doesn't linger in some border cases.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/fork.cc | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 6f00364..5f77524 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -400,7 +400,6 @@ frok::parent (volatile char * volatile stack_here)
  we can't actually record the pid in the internal table. */
   if (!child.remember (false))
 {
-  TerminateProcess (hchild, 1);
   this_errno = EAGAIN;
 #ifdef DEBUGGING0
   error ("child remember failed");
@@ -508,8 +507,12 @@ cleanup:
 __malloc_unlock ();
 
   /* Remember to de-allocate the fd table. */
-  if (hchild && !child.hProcess) /* no child.procinfo */
-ForceCloseHandle1 (hchild, childhProc);
+  if (hchild)
+{
+  TerminateProcess (hchild, 1);
+  if (!child.hProcess) /* no child.procinfo */
+   ForceCloseHandle1 (hchild, childhProc);
+}
   if (forker_finished)
 ForceCloseHandle (forker_finished);
   debug_printf ("returning -1");

[newlib-cygwin] fork: remove cygpid.N sharedmem on fork failure

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=fb3e8bd88b06be6b5e748b99aa50968bb46653a1

commit fb3e8bd88b06be6b5e748b99aa50968bb46653a1
Author: Michael Haubenwallner 
Date:   Tue Jun 5 12:40:21 2018 +0200

fork: remove cygpid.N sharedmem on fork failure

When fork finally fails although both CreateProcess and creating the
"cygpid.N" shared memory section succeeded, we have to release that
shared memory section as well - before releasing the process handle.
Otherways we leave an orphan "cygpid.N" shared memory section, and any
subsequent cygwin process receiving the same PID fails to initialize.

* fork.cc (frok::parent): Call child.allow_remove in cleanup code.

Diff:
---
 winsup/cygwin/fork.cc | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index c083f7a..6f00364 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -499,13 +499,16 @@ frok::parent (volatile char * volatile stack_here)
 
 /* Common cleanup code for failure cases */
 cleanup:
+  /* release procinfo before hProcess in destructor */
+  child.allow_remove ();
+
   if (fix_impersonation)
 cygheap->user.reimpersonate ();
   if (locked)
 __malloc_unlock ();
 
   /* Remember to de-allocate the fd table. */
-  if (hchild && !child.hProcess)
+  if (hchild && !child.hProcess) /* no child.procinfo */
 ForceCloseHandle1 (hchild, childhProc);
   if (forker_finished)
 ForceCloseHandle (forker_finished);

[newlib-cygwin] Cygwin: fork: fix child process permissions, take 2

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=ef8ce3077f55d5a105b39f605b877da50ab80aa7

commit ef8ce3077f55d5a105b39f605b877da50ab80aa7
Author: Corinna Vinschen 
Date:   Wed Jan 30 12:18:03 2019 +0100

Cygwin: fork: fix child process permissions, take 2

VirtualQueryEx, called by fixup_mmaps_after_fork, requires
PROCESS_QUERY_INFORMATION permissions per MSDN.  However, testing
shows that PROCESS_QUERY_LIMITED_INFORMATION is sufficient when
running the same code on Windows 8.1 or Windows 10.  Fix the code
to give the forked child always PROCESS_QUERY_INFORMATION perms
on Windows Vista/7 and respective server releases.

Revert now unneeded patch to check_token_membership as well.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/security.h |  1 -
 winsup/cygwin/sigproc.cc | 10 --
 winsup/cygwin/uinfo.cc   | 14 --
 3 files changed, 8 insertions(+), 17 deletions(-)

diff --git a/winsup/cygwin/security.h b/winsup/cygwin/security.h
index ec68171..70912b4 100644
--- a/winsup/cygwin/security.h
+++ b/winsup/cygwin/security.h
@@ -17,7 +17,6 @@ details. */
 
 /* UID/GID */
 void uinfo_init ();
-bool check_token_membership (HANDLE, PSID);
 bool check_token_membership (PSID);
 
 #define ILLEGAL_UID ((uid_t)-1)
diff --git a/winsup/cygwin/sigproc.cc b/winsup/cygwin/sigproc.cc
index 080fe58..a830bff 100644
--- a/winsup/cygwin/sigproc.cc
+++ b/winsup/cygwin/sigproc.cc
@@ -819,12 +819,10 @@ child_info::child_info (unsigned in_cb, child_info_types 
chtype,
   if (type == _CH_FORK)
 {
   perms |= PROCESS_DUP_HANDLE;
-  /* For some reason fork on Windows 7 requires PROCESS_QUERY_INFORMATION
-rather than just PROCESS_QUERY_LIMITED_INFORMATION when started as a
-service. */
-  if (wincap.needs_query_information ()
- && (cygheap->user.saved_sid () == well_known_system_sid
- || check_token_membership (hProcToken, well_known_service_sid)))
+  /* VirtualQueryEx is documented to require PROCESS_QUERY_INFORMATION.
+That's true for Windows 7, but PROCESS_QUERY_LIMITED_INFORMATION
+appears to be sufficient on Windows 8 and later. */
+  if (wincap.needs_query_information ())
perms |= PROCESS_QUERY_INFORMATION;
 }
 
diff --git a/winsup/cygwin/uinfo.cc b/winsup/cygwin/uinfo.cc
index 00a2b5a..8dcf731 100644
--- a/winsup/cygwin/uinfo.cc
+++ b/winsup/cygwin/uinfo.cc
@@ -118,13 +118,16 @@ cygheap_user::init ()
This needs careful checking should we use check_token_membership in other
circumstances. */
 bool
-check_token_membership (HANDLE tok, PSID sid)
+check_token_membership (PSID sid)
 {
   NTSTATUS status;
   ULONG size;
   tmp_pathbuf tp;
   PTOKEN_GROUPS groups = (PTOKEN_GROUPS) tp.w_get ();
 
+  /* If impersonated, use impersonation token. */
+  HANDLE tok = cygheap->user.issetuid () ? cygheap->user.primary_token ()
+: hProcToken;
   status = NtQueryInformationToken (tok, TokenGroups, groups, 2 * NT_MAX_PATH,
);
   if (!NT_SUCCESS (status))
@@ -139,15 +142,6 @@ check_token_membership (HANDLE tok, PSID sid)
   return false;
 }
 
-bool
-check_token_membership (PSID sid)
-{
-  /* If impersonated, use impersonation token. */
-  HANDLE tok = cygheap->user.issetuid () ? cygheap->user.primary_token ()
-: hProcToken;
-  return check_token_membership (tok, sid);
-}
-
 static void
 internal_getlogin (cygheap_user )
 {

[newlib-cygwin] Cygwin: fork/exec: fix child process permissions

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=5a0f2c00aa019de73a6077ca3017b594c43184a4

commit 5a0f2c00aa019de73a6077ca3017b594c43184a4
Author: Corinna Vinschen 
Date:   Tue Jan 29 16:26:45 2019 +0100

Cygwin: fork/exec: fix child process permissions

- Exec'ed/spawned processes don't need PROCESS_DUP_HANDLE.  Remove that
  permission from the parent handle.

- PROCESS_QUERY_LIMITED_INFORMATION doesn't work for Windows 7 if the
  process is started as a service.  Add PROCESS_QUERY_INFORMATION for
  pre-Windows 8 in that case.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/security.h |  1 +
 winsup/cygwin/sigproc.cc | 20 
 winsup/cygwin/uinfo.cc   | 14 ++
 winsup/cygwin/wincap.cc  |  9 +
 winsup/cygwin/wincap.h   |  2 ++
 5 files changed, 38 insertions(+), 8 deletions(-)

diff --git a/winsup/cygwin/security.h b/winsup/cygwin/security.h
index 70912b4..ec68171 100644
--- a/winsup/cygwin/security.h
+++ b/winsup/cygwin/security.h
@@ -17,6 +17,7 @@ details. */
 
 /* UID/GID */
 void uinfo_init ();
+bool check_token_membership (HANDLE, PSID);
 bool check_token_membership (PSID);
 
 #define ILLEGAL_UID ((uid_t)-1)
diff --git a/winsup/cygwin/sigproc.cc b/winsup/cygwin/sigproc.cc
index 45e9482..42eeb30 100644
--- a/winsup/cygwin/sigproc.cc
+++ b/winsup/cygwin/sigproc.cc
@@ -811,12 +811,24 @@ child_info::child_info (unsigned in_cb, child_info_types 
chtype,
 }
   sigproc_printf ("subproc_ready %p", subproc_ready);
   /* Create an inheritable handle to pass to the child process.  This will
- allow the child to duplicate handles from the parent to itself. */
+ allow the child to copy cygheap etc. from the parent to itself.  If
+ we're forking, we also need handle duplicate access. */
   parent = NULL;
+  DWORD perms = PROCESS_QUERY_LIMITED_INFORMATION | PROCESS_VM_READ;
+  if (type == _CH_FORK)
+{
+  perms |= PROCESS_DUP_HANDLE;
+  /* For some reason fork on Windows 7 requires PROCESS_QUERY_INFORMATION
+rather than just PROCESS_QUERY_LIMITED_INFORMATION when started as a
+service. */
+  if (wincap.needs_query_information ()
+ && (cygheap->user.saved_sid () == well_known_system_sid
+ || check_token_membership (hProcToken, well_known_service_sid)))
+   perms |= PROCESS_QUERY_INFORMATION;
+}
+
   if (!DuplicateHandle (GetCurrentProcess (), GetCurrentProcess (),
-   GetCurrentProcess (), ,
-   PROCESS_DUP_HANDLE | PROCESS_VM_READ
-   | PROCESS_QUERY_LIMITED_INFORMATION, TRUE, 0))
+   GetCurrentProcess (), , perms, TRUE, 0))
 system_printf ("couldn't create handle to myself for child, %E");
 }
 
diff --git a/winsup/cygwin/uinfo.cc b/winsup/cygwin/uinfo.cc
index 8dcf731..00a2b5a 100644
--- a/winsup/cygwin/uinfo.cc
+++ b/winsup/cygwin/uinfo.cc
@@ -118,16 +118,13 @@ cygheap_user::init ()
This needs careful checking should we use check_token_membership in other
circumstances. */
 bool
-check_token_membership (PSID sid)
+check_token_membership (HANDLE tok, PSID sid)
 {
   NTSTATUS status;
   ULONG size;
   tmp_pathbuf tp;
   PTOKEN_GROUPS groups = (PTOKEN_GROUPS) tp.w_get ();
 
-  /* If impersonated, use impersonation token. */
-  HANDLE tok = cygheap->user.issetuid () ? cygheap->user.primary_token ()
-: hProcToken;
   status = NtQueryInformationToken (tok, TokenGroups, groups, 2 * NT_MAX_PATH,
);
   if (!NT_SUCCESS (status))
@@ -142,6 +139,15 @@ check_token_membership (PSID sid)
   return false;
 }
 
+bool
+check_token_membership (PSID sid)
+{
+  /* If impersonated, use impersonation token. */
+  HANDLE tok = cygheap->user.issetuid () ? cygheap->user.primary_token ()
+: hProcToken;
+  return check_token_membership (tok, sid);
+}
+
 static void
 internal_getlogin (cygheap_user )
 {
diff --git a/winsup/cygwin/wincap.cc b/winsup/cygwin/wincap.cc
index 4c1c7b4..132265c 100644
--- a/winsup/cygwin/wincap.cc
+++ b/winsup/cygwin/wincap.cc
@@ -23,6 +23,7 @@ wincaps wincap_vista __attribute__((section 
(".cygwin_dll_common"), shared)) = {
   {
 is_server:false,
 needs_count_in_si_lpres2:true,
+needs_query_information:true,
 has_gaa_largeaddress_bug:true,
 has_broken_alloc_console:false,
 has_console_logon_sid:false,
@@ -46,6 +47,7 @@ wincaps wincap_7 __attribute__((section 
(".cygwin_dll_common"), shared)) = {
   {
 is_server:false,
 needs_count_in_si_lpres2:false,
+needs_query_information:true,
 has_gaa_largeaddress_bug:true,
 has_broken_alloc_console:true,
 has_console_logon_sid:true,
@@ -69,6 +71,7 @@ wincaps wincap_8 __attribute__((section 
(".cygwin_dll_common"), shared)) = {
   {
 is_server:false

[newlib-cygwin] Cygwin: fork: restrict parent handle perms and drop handle after use

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=69cc7a068656b5c6ef07ca079a213f801e02e650

commit 69cc7a068656b5c6ef07ca079a213f801e02e650
Author: Corinna Vinschen 
Date:   Sun Jan 27 13:15:15 2019 +0100

Cygwin: fork: restrict parent handle perms and drop handle after use

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/dcrt0.cc   | 3 ++-
 winsup/cygwin/fork.cc| 2 ++
 winsup/cygwin/sigproc.cc | 5 +++--
 3 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/winsup/cygwin/dcrt0.cc b/winsup/cygwin/dcrt0.cc
index af5eaac..cf2a08e 100644
--- a/winsup/cygwin/dcrt0.cc
+++ b/winsup/cygwin/dcrt0.cc
@@ -637,7 +637,8 @@ child_info_fork::handle_fork ()
 bool
 child_info_spawn::get_parent_handle ()
 {
-  parent = OpenProcess (PROCESS_VM_READ, false, parent_winpid);
+  parent = OpenProcess (PROCESS_QUERY_LIMITED_INFORMATION, FALSE,
+   parent_winpid);
   moreinfo->myself_pinfo = NULL;
   return !!parent;
 }
diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index 6813446..d8c4ac4 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -203,6 +203,8 @@ frok::child (volatile char * volatile here)
  rd_proc_pipe that would be an invalid handle.  In the case of
  wr_proc_pipe it would be == my_wr_proc_pipe.  Both would be bad. */
   ch.rd_proc_pipe = ch.wr_proc_pipe = NULL;
+  CloseHandle (hParent);
+  hParent = NULL;
   cygwin_finished_initializing = true;
   return 0;
 }
diff --git a/winsup/cygwin/sigproc.cc b/winsup/cygwin/sigproc.cc
index 92fa5ea..45e9482 100644
--- a/winsup/cygwin/sigproc.cc
+++ b/winsup/cygwin/sigproc.cc
@@ -814,8 +814,9 @@ child_info::child_info (unsigned in_cb, child_info_types 
chtype,
  allow the child to duplicate handles from the parent to itself. */
   parent = NULL;
   if (!DuplicateHandle (GetCurrentProcess (), GetCurrentProcess (),
-   GetCurrentProcess (), , 0, true,
-   DUPLICATE_SAME_ACCESS))
+   GetCurrentProcess (), ,
+   PROCESS_DUP_HANDLE | PROCESS_VM_READ
+   | PROCESS_QUERY_LIMITED_INFORMATION, TRUE, 0))
 system_printf ("couldn't create handle to myself for child, %E");
 }

[newlib-cygwin] fork: Don't copy _main_tls->local_clib from *_impure_ptr

[Corinna Vinschen]

https://sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git;h=48755fb9bca8ae379a6f96619b8b7774ff4b4494

commit 48755fb9bca8ae379a6f96619b8b7774ff4b4494
Author: Corinna Vinschen 
Date:   Fri Mar 10 20:44:53 2017 +0100

fork: Don't copy _main_tls->local_clib from *_impure_ptr

So far we copy *_impure_ptr into _main_tls->local_clib if the child
process has been forked from a pthread.  But that's not required.
The local_clib area of the new thread is on the stack and the stack
gets copied from the parent anyway (in frok::parent).  So we only
have to make sure _main_tls is pointing to the right address and
do the simple post-fork thread init.

Signed-off-by: Corinna Vinschen 

Diff:
---
 winsup/cygwin/fork.cc | 9 ++---
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/winsup/cygwin/fork.cc b/winsup/cygwin/fork.cc
index ef5a268..73a72f5 100644
--- a/winsup/cygwin/fork.cc
+++ b/winsup/cygwin/fork.cc
@@ -143,15 +143,11 @@ frok::child (volatile char * volatile here)
myself->pid, myself->ppid, __builtin_frame_address (0));
   sigproc_printf ("hParent %p, load_dlls %d", hParent, load_dlls);
 
-  /* If we've played with the stack, stacksize != 0.  That means that
- fork() was invoked from other than the main thread.  Make sure that
- the threadinfo information is properly set up.  */
-  if (!fork_info->from_main)
+  /* Make sure threadinfo information is properly set up. */
+  if (&_my_tls != _main_tls)
 {
   _main_tls = &_my_tls;
   _main_tls->init_thread (NULL, NULL);
-  _main_tls->local_clib = *_impure_ptr;
-  _impure_ptr = &_main_tls->local_clib;
 }
 
   set_cygwin_privileges (hProcToken);
@@ -300,7 +296,6 @@ frok::parent (volatile char * volatile stack_here)
 
   ch.forker_finished = forker_finished;
 
-  ch.from_main = &_my_tls == _main_tls;
   ch.stackbase = NtCurrentTeb ()->Tib.StackBase;
   ch.stackaddr = NtCurrentTeb ()->DeallocationStack;
   if (!ch.stackaddr)

Re: Cygwin fork problem: abort: can't commit memory for stack

[Techie Help]

Hi,

As suggested by Marco, I stripped the PATH to bare minimum and then it started 
building.
But slowly  slowly, one at a time, I started adding more paths to path variable 
and finally I have restored it back to old one but its now building without any 
problem.
So, I am not sure what happened.

Also, can you please explain to me, why stripping path to bare minimum helped?
How was PATH interfering with fork process?

Thanks alot,
techie





On 11/30/2012 5:24 PM, Techie Help wrote:
I have set up a new PC and installed cygwin on it.
Its windows 7 pro.
Whenever I try to build our application on it, I get the following error:


0 [main] sh 3472 child_info_fork::abort: can't commit memory for stack 
0x28A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable 0 [main] sh 3220 
child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 
error 487
/bin/sh: fork: retry: Resource temporarily unavailable 0 [main] sh 4896 
child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 
error 487
/bin/sh: fork: retry: Resource temporarily unavailable 0 [main] sh 4884 
child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 
error 487
It prints this few times and then dies.

ERROR_INVALID_ADDRESS 487 (0x1E7)


Which application / tools are you using for the build ?
During the build are you running the application itself ?
on my system looking with less /proc/self/maps

0008C000-0009 rw-p 0003C000 : 0
0009-00279000 ===p  : 0 [stack (tid 2468)]
00279000-0027A000 rw-g 001E9000 : 0 [stack (tid 2468)]
0027A000-0029 rw-p 001EA000 : 0 [stack (tid 2468)]
0029-00294000 r--s  : 0
002A-002A1000 rw-p  : 0
002B-002E9000 ===p  : 0
002E9000-002EC000 rw-g 00039000 : 0
002EC000-002F rw-p 0003C000 : 0
0031-00384000 rw-p  : 0
00384000-0039 ===p 00074000 : 0
0039-003F7000 r--s  4038:403E 281474976758678 
/cygdrive/c/Windows/System32/locale.nls


the stack is in the same area and the size limiting factor
is the presence of a system dll

I have already done a lot of research on this problem.
I have already uninstalled and installed cygwin  more than 5 times.
Done rebaseall everytime I installed it.
Checked for possible BLODA, I had notron antivirus, which I have removed.

I am attaching the output of cygcheck and 'rebase -is'.
I have modified the cygcheck.out to remove login info and project specific 
paths 

nothing strange.
your rebase output is extremely clean and collision is unlikely.
My system is basically the same but with much more dlls and collisions.




I really am not sure whats going on and it has become a blocking problem for 
me.
Please if anyone can help me, I would be grateful. 

just as trial, have you tried to simplify the PATH to a bare minimum ?
Only :
C:\cygwin\usr\local\bin
C:\cygwin\bin



Thanks,
techie 

Regards
Marco


--
Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ 
Documentation: http://cygwin.com/docs.html  
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Documentation:      
  http://cygwin.com/docs.html
 Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fork problem: abort: can't commit memory for stack

[Ryan Johnson]

On 03/12/2012 5:23 AM, Techie Help wrote:

Hi,

As suggested by Marco, I stripped the PATH to bare minimum and then it started 
building.
But slowly  slowly, one at a time, I started adding more paths to path variable 
and finally I have restored it back to old one but its now building without any 
problem.
So, I am not sure what happened.

Also, can you please explain to me, why stripping path to bare minimum helped?
I'm not an expert, but I believe that several BLODA depend on the dlls 
they inject to be in PATH; strip the path and they fail to do their 
thing, leaving cygwin free to do its thing.


Ryan


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fork problem: abort: can't commit memory for stack

[marco atzeri]

On 11/30/2012 5:24 PM, Techie Help wrote:

I have set up a new PC and installed cygwin on it.
Its windows 7 pro.

Whenever I try to build our application on it, I get the following error:

  0 [main] sh 3472 child_info_fork::abort: can't commit memory for stack 
0x28A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
   0 [main] sh 3220 child_info_fork::abort: can't commit memory for stack 
0x28A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
   0 [main] sh 4896 child_info_fork::abort: can't commit memory for stack 
0x28A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
   0 [main] sh 4884 child_info_fork::abort: can't commit memory for stack 
0x28A000(90112), Win32 error 487

   It prints this few times and then dies.



ERROR_INVALID_ADDRESS   487 (0x1E7)

Which application / tools are you using for the build ?
During the build are you running the application itself ?

on my system looking with less /proc/self/maps

0008C000-0009 rw-p 0003C000 : 0
0009-00279000 ===p  : 0 [stack (tid 2468)]
00279000-0027A000 rw-g 001E9000 : 0 [stack (tid 2468)]
0027A000-0029 rw-p 001EA000 : 0 [stack (tid 2468)]
0029-00294000 r--s  : 0
002A-002A1000 rw-p  : 0
002B-002E9000 ===p  : 0
002E9000-002EC000 rw-g 00039000 : 0
002EC000-002F rw-p 0003C000 : 0
0031-00384000 rw-p  : 0
00384000-0039 ===p 00074000 : 0
0039-003F7000 r--s  4038:403E 281474976758678 
/cygdrive/c/Windows/System32/locale.nls


the stack is in the same area and the size limiting factor
is the presence of a system dll


I have already done a lot of research on this problem.
I have already uninstalled and installed cygwin  more than 5 times.
Done rebaseall everytime I installed it.
Checked for possible BLODA, I had notron antivirus, which I have removed.

I am attaching the output of cygcheck and 'rebase -is'.
I have modified the cygcheck.out to remove login info and project specific paths


nothing strange.
your rebase output is extremely clean and collision is unlikely.

My system is basically the same but with much more dlls and collisions.




I really am not sure whats going on and it has become a blocking problem for me.
Please if anyone can help me, I would be grateful.


just as trial, have you tried to simplify the PATH to a bare minimum ?
Only :
C:\cygwin\usr\local\bin
C:\cygwin\bin




Thanks,
techie


Regards
Marco


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fork problem: abort: can't commit memory for stack

[Christopher Faylor]

On Sat, Dec 01, 2012 at 09:32:38AM +0100, marco atzeri wrote:
On 11/30/2012 5:24 PM, Techie Help wrote:
 I have set up a new PC and installed cygwin on it.
 Its windows 7 pro.

 Whenever I try to build our application on it, I get the following error:

   0 [main] sh 3472 child_info_fork::abort: can't commit memory for stack 
 0x28A000(90112), Win32 error 487
 /bin/sh: fork: retry: Resource temporarily unavailable
0 [main] sh 3220 child_info_fork::abort: can't commit memory for 
 stack 0x28A000(90112), Win32 error 487
 /bin/sh: fork: retry: Resource temporarily unavailable
0 [main] sh 4896 child_info_fork::abort: can't commit memory for 
 stack 0x28A000(90112), Win32 error 487
 /bin/sh: fork: retry: Resource temporarily unavailable
0 [main] sh 4884 child_info_fork::abort: can't commit memory for 
 stack 0x28A000(90112), Win32 error 487

It prints this few times and then dies.


ERROR_INVALID_ADDRESS   487 (0x1E7)

That implies BLODA.

http://cygwin.com/acronyms#BLODA

cgf

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin fork problem: abort: can't commit memory for stack

[Massi]

Very same problem, already discussed in another thead or two.
I will also be grateful if this could be fixed.

Massi

2012/11/30 Techie Help techiei...@yahoo.co.uk:
 I have set up a new PC and installed cygwin on it.
 Its windows 7 pro.

 Whenever I try to build our application on it, I get the following error:

  0 [main] sh 3472 child_info_fork::abort: can't commit memory for stack 
 0x28A000(90112), Win32 error 487
 /bin/sh: fork: retry: Resource temporarily unavailable
   0 [main] sh 3220 child_info_fork::abort: can't commit memory for stack 
 0x28A000(90112), Win32 error 487
 /bin/sh: fork: retry: Resource temporarily unavailable
   0 [main] sh 4896 child_info_fork::abort: can't commit memory for stack 
 0x28A000(90112), Win32 error 487
 /bin/sh: fork: retry: Resource temporarily unavailable
   0 [main] sh 4884 child_info_fork::abort: can't commit memory for stack 
 0x28A000(90112), Win32 error 487

   It prints this few times and then dies.

 I have already done a lot of research on this problem.
 I have already uninstalled and installed cygwin  more than 5 times.
 Done rebaseall everytime I installed it.
 Checked for possible BLODA, I had notron antivirus, which I have removed.

 I am attaching the output of cygcheck and 'rebase -is'.
 I have modified the cygcheck.out to remove login info and project specific 
 paths

 I really am not sure whats going on and it has become a blocking problem for 
 me.
 Please if anyone can help me, I would be grateful.

 Thanks,
 techie
 --
 Problem reports:   http://cygwin.com/problems.html
 FAQ:   http://cygwin.com/faq/
 Documentation: http://cygwin.com/docs.html
 Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Greg Chicares]

On 2011-09-06 01:53Z, jan.kolar wrote:
 
 Why the newest version 
   rebase/rebase-3.0-2 utilities for rebasing DLLs [...]
 is listed in the middle of this page?

Actually, the one at the bottom is the newest:

2009 April rebase-3.0-2
2009 July  rebase-3.0.1-1

Anything following a '-' at the end is likely to be an auxiliary serial
number, e.g., to distinguish successive uploads of the same release.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Christopher Faylor]

On Mon, Sep 05, 2011 at 06:53:34PM -0700, jan.kolar wrote:

 Then you need to install the rebase package:
 http://cygwin.com/cgi-bin2/package-grep.cgi?grep=rebaseall

Why the newest version 
   rebase/rebase-3.0-2 utilities for rebasing DLLs to load at alternate
addresses
is listed in the middle of this page?

It's confusing.

Not really.  You searched for every package which contained the string
rebase.

If you were looking for the latest version of rebase then go to
http://cygwin.com/packages/, scroll down until you see rebase and then
click on that.  Or, if you want to use the package grep, type
rebase.exe into the search box to look for the package which contains
that program.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[jan.kolar]

 Then you need to install the rebase package:
 http://cygwin.com/cgi-bin2/package-grep.cgi?grep=rebaseall

Why the newest version 
rebase/rebase-3.0-2 utilities for rebasing DLLs to load at alternate
addresses
is listed in the middle of this page?

It's confusing.

Again, cygcheck -p all
seems to be sorted (it almost is).

-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32404960.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Christopher Faylor]

On Sat, Aug 20, 2011 at 05:03:16AM -0700, big glass wrote:

$ /bin/peflagsall/
/usr/bin/ash: Can't open /bin/peflagsall/

:(

Why are you putting slashes on the end of the command?  You wouldn't
do that on Windows or Linux.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

it seems to me you unix guys are very smart :)


Csaba Raduly-2 wrote:
 
 On Fri, Aug 19, 2011 at 7:07 PM, big glass  wrote:
 how can i make sure that you exit all
 Cygwin processes

 
 Get Process Explorer from sysinternals (now Microsoft:
 http://technet.microsoft.com/en-us/sysinternals/bb896653).
 It allows you to search for in-use files and handles. If cygwin1.dll
 is not loaded in any executable, that's a pretty good indicator that
 no Cygwin process is running.
 
 Csaba
 -- 
 GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++
 The Tao of math: The numbers you can count are not the real numbers.
 Life is complex, with real and imaginary parts.
 Ok, it boots. Which means it must be bug-free and perfect.  -- Linus
 Torvalds
 People disagree with me. I just ignore them. -- Linus Torvalds
 
 --
 Problem reports:   http://cygwin.com/problems.html
 FAQ:   http://cygwin.com/faq/
 Documentation: http://cygwin.com/docs.html
 Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
 
 
 

-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32300380.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

thx, im now getting this:

$/bin/rebaseall
/usr/bin/ash: $: not found
$


Corinna Vinschen-2 wrote:
 
 On Aug 19 07:35, big glass wrote:
 
 cheers, tried it in ash but get this:
 
 $ rebaseall   : not found
 
  $ /bin/rebaseall
 
 
 Corinna
 
 -- 
 Corinna Vinschen  Please, send mails regarding Cygwin to
 Cygwin Project Co-Leader  cygwin AT cygwin DOT com
 Red Hat
 
 --
 Problem reports:   http://cygwin.com/problems.html
 FAQ:   http://cygwin.com/faq/
 Documentation: http://cygwin.com/docs.html
 Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
 
 
 

-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32300438.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Corinna Vinschen]

On Aug 20 01:43, big glass wrote:
 
 thx, im now getting this:
 
 $/bin/rebaseall
 /usr/bin/ash: $: not found
 $

Sigh.  The $ is the universal sign for the shell prompt.  It's the
default shell prompt on Unix machines.  You are not supposed to type it
in.  Just start ash, then you'll see the $ on the left side and the
cursor at the right side of the $ sign.  Now just type /bin/rebaseall
and [Enter].  When the next $ sign appears, rebaseall has done its job
and you can exit ash with Ctrl-D.

Don't get me wrong, but I am really wondering if Cygwin is what you were
looking for.  You seem to be missing even the most basic idea of what a
POSIX system looks like and how to use the command line.  Maybe you
should start with a good beginners book on Unix or Linux?


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

I am a developer and i use windoo$e at work, and just want to get sproutcore
on our servers but it has this nice in the ruby area, so I need cygwin on
our servers to be compatible :)  I will install a Virtual linux on my pc at
home to play, any distro you recoomend to learn the basic with?? thx mate
-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32300974.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

$ /bin/peflagsall/
/usr/bin/ash: Can't open /bin/peflagsall/

:(




$ ./reply-to-typical-fork-complaint.sh
Please run rebaseall (and peflagsall+reboot if on Vista or Win7)


-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32301099.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Typical Cygwin fork problem

[big glass]

I have ruby installed and am trying this, but run this but lots of errors:

$ gem install sproutcore
Fetching: rack-1.3.2.gem (100%)
Fetching: json_pure-1.4.6.gem (100%)
Fetching: extlib-0.9.15.gem (100%)
Fetching: erubis-2.7.0.gem (100%)
Fetching: thor-0.14.6.gem (100%)
Fetching: haml-3.1.2.gem (100%)
Fetching: sass-3.1.7.gem (100%)
Fetching: chunky_png-1.2.1.gem (100%)
Fetching: fssm-0.2.7.gem (100%)
Fetching: compass-0.11.5.gem (100%)
Fetching: eventmachine-1.0.0.beta.3.gem (100%)
Building native extensions.  This could take a while...
  1 [main] ruby 7216 C:\cygwin\home\me\.rvm\rubies\ruby-1.9.2-p290\
bin\ruby.exe: *** fatal error - unable to remap \\?\C:\cygwin\home\me\.
rvm\rubies\ruby-1.9.2-p290\lib\ruby\1.9.1\i386-cygwin\etc.so to same 
address as
parent: 0x3E != 0x3F
Stack trace:
Frame Function  Args
00289098  6102796B  (00289098, , , )
00289388  6102796B  (6117EC60, 8000, , 61180977)
0028A3B8  61004F1B  (611A7FAC, 612492D4, 003E, 003F)
End of stack trace
  1 [main] ruby 9668 fork: child 7216 - died waiting for dll 
loading, errno
-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32293766.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Ryan Johnson]

On 19/08/2011 5:23 AM, big glass wrote:

I have ruby installed and am trying this, but run this but lots of errors:

$ gem install sproutcore
Fetching: rack-1.3.2.gem (100%)
Fetching: json_pure-1.4.6.gem (100%)
Fetching: extlib-0.9.15.gem (100%)
Fetching: erubis-2.7.0.gem (100%)
Fetching: thor-0.14.6.gem (100%)
Fetching: haml-3.1.2.gem (100%)
Fetching: sass-3.1.7.gem (100%)
Fetching: chunky_png-1.2.1.gem (100%)
Fetching: fssm-0.2.7.gem (100%)
Fetching: compass-0.11.5.gem (100%)
Fetching: eventmachine-1.0.0.beta.3.gem (100%)
Building native extensions.  This could take a while...
   1 [main] ruby 7216 C:\cygwin\home\me\.rvm\rubies\ruby-1.9.2-p290\
bin\ruby.exe: *** fatal error - unable to remap \\?\C:\cygwin\home\me\.
rvm\rubies\ruby-1.9.2-p290\lib\ruby\1.9.1\i386-cygwin\etc.so to same
address as
parent: 0x3E != 0x3F
Stack trace:
Frame Function  Args
00289098  6102796B  (00289098, , , )
00289388  6102796B  (6117EC60, 8000, , 61180977)
0028A3B8  61004F1B  (611A7FAC, 612492D4, 003E, 003F)
End of stack trace
   1 [main] ruby 9668 fork: child 7216 - died waiting for dll
loading, errno

$ ./reply-to-typical-fork-complaint.sh
Please run rebaseall (and peflagsall+reboot if on Vista or Win7)


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

cheers, tried it in ash but get this:

$ rebaseall   : not found
$ sr/bin/ash: rebaseall: not found
-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32295811.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Csaba Raduly]

On Fri, Aug 19, 2011 at 4:35 PM, big glass  wrote:
 cheers, tried it in ash but get this:

 $ rebaseall   : not found
 $ sr/bin/ash: rebaseall: not found

Then you need to install the rebase package:

http://cygwin.com/cgi-bin2/package-grep.cgi?grep=rebaseall

Csaba
-- 
GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++
The Tao of math: The numbers you can count are not the real numbers.
Life is complex, with real and imaginary parts.
Ok, it boots. Which means it must be bug-free and perfect.  -- Linus Torvalds
People disagree with me. I just ignore them. -- Linus Torvalds

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Corinna Vinschen]

On Aug 19 07:35, big glass wrote:
 
 cheers, tried it in ash but get this:
 
 $ rebaseall   : not found

 $ /bin/rebaseall


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

:)
it already is installed, ask wont recognise it :)
-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32296617.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Christopher Faylor]

On Fri, Aug 19, 2011 at 09:13:13AM -0700, big glass wrote:
it already is installed, ask wont recognise it :)

It's probably wearing glasses then.  Duh.

cgf

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Corinna Vinschen]

On Aug 19 09:13, big glass wrote:
 
 :)
 it already is installed, ask wont recognise it :)

  $ /bin/rebaseall--- use the full path


Corinna

-- 
Corinna Vinschen  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader  cygwin AT cygwin DOT com
Red Hat

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

so what is ash exactly??

Corinna Vinschen-2 wrote:
 
 On Aug 19 09:13, big glass wrote:
 
 :)
 it already is installed, ask wont recognise it :)
 
   $ /bin/rebaseall--- use the full path
 
 
 Corinna
 
 -- 
 Corinna Vinschen  Please, send mails regarding Cygwin to
 Cygwin Project Co-Leader  cygwin AT cygwin DOT com
 Red Hat
 
 --
 Problem reports:   http://cygwin.com/problems.html
 FAQ:   http://cygwin.com/faq/
 Documentation: http://cygwin.com/docs.html
 Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
 
 
 

-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32296836.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Jeremy Bopp]

On 8/19/2011 11:38, big glass wrote:
 
 so what is ash exactly??

ash is another name for dash, which is described here:

http://en.wikipedia.org/wiki/Debian_Almquist_shell

The short answer is that it's a minimal shell that you need to use in
your case for running the rebaseall command so that there aren't any
DLLs locked by other Cygwin processes.  BTW, make sure that you exit all
Cygwin processes except for ash before you start rebaseall.

-Jeremy

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Eric Blake]

On 08/19/2011 10:38 AM, big glass wrote:


so what is ash exactly??


Another name for dash, which is a lighter-weight shell than bash.  In 
particular, since dash pulls in fewer shared libraries, it can rebase 
things that bash cannot.


--
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[big glass]

i am noob, but prof windows programmer 3+ years :)

how can i make sure that you exit all
Cygwin processes 

what is rebasing?
-- 
View this message in context: 
http://old.nabble.com/Typical-Cygwin-fork-problem-tp32293766p32297033.html
Sent from the Cygwin list mailing list archive at Nabble.com.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: Typical Cygwin fork problem

[Csaba Raduly]

On Fri, Aug 19, 2011 at 7:07 PM, big glass  wrote:
 how can i make sure that you exit all
 Cygwin processes


Get Process Explorer from sysinternals (now Microsoft:
http://technet.microsoft.com/en-us/sysinternals/bb896653).
It allows you to search for in-use files and handles. If cygwin1.dll
is not loaded in any executable, that's a pretty good indicator that
no Cygwin process is running.

Csaba
-- 
GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++
The Tao of math: The numbers you can count are not the real numbers.
Life is complex, with real and imaginary parts.
Ok, it boots. Which means it must be bug-free and perfect.  -- Linus Torvalds
People disagree with me. I just ignore them. -- Linus Torvalds

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

GNU pth + cygwin + fork [Was: Re: fork failure?]

[Charles Wilson]

Charles Wilson wrote:

 Ach, the purist in me just wants to get pth working...

Hmm...it appears the right way to do this is NOT to add another special
case in pth: no, on cygwin THIS is the way you poke around in the
jmp_buf + extra cygwin TLC in pth_fork().  Instead, cygwin pth should
use the standard posix sigstack/sigaltstack approach.

But that'll have to wait until after cygwin-1.7.1:
http://cygwin.com/ml/cygwin/2009-07/msg00859.html
 Let me add a new data point: I'll implement sigaltstack after 1.7.1 is
 released.

And, of course, cgf's statement above doesn't mean that sigaltstack will
be available the day after 1.7.1 is released, either. I'm sure it will
be devilishly tricky to get right, and will take a lot of time and effort.

In the short-to-medium term, it looks like converting libassuan and
gnupg to use pthreads instead of pth won't be terribly difficult.  Once
once sig[alt]stack is available I can modify cygwin-pth to use the
sig[alt]stack Machine Context Implementation instead of the current
sjlj/sjljw32/none one, and then restore libassuan and gnupg to the pth
status quo ante.

I think that pretty much ends this nightmare thread -- but chalk another
vote up there for pretty please, cgf, implement sigaltstack soonish.

--
Chuck

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: GNU pth + cygwin + fork [Was: Re: fork failure?]

[Dave Korn]

Charles Wilson wrote:

 In the short-to-medium term, it looks like converting libassuan and
 gnupg to use pthreads instead of pth won't be terribly difficult.  Once
 once sig[alt]stack is available I can modify cygwin-pth to use the
 sig[alt]stack Machine Context Implementation instead of the current
 sjlj/sjljw32/none one, and then restore libassuan and gnupg to the pth
 status quo ante.

  My first thought would be to figure out what pth is attempting to do while
messing in jmp_buf, and make it work.  It's bad, unmaintainable code, that
will break again in the future if ever jmp_buf is rearranged - but it only has
to stagger along for another couple of months until you can do it right using
sigaltstack.  Until then, slapping a band-aid on pth might be a lot less
work-that-soon-has-to-be-thrown-away than hacking both libassuan and gpg to
handle a different API.  (I say this without having yet done the research to
figure out exactly what pth thinks it is doing to that jmp_buf and whether
it's necessarily possible, but it ought to be.)

  Anyway, it's your effort so it's your call but I suggest this strategy
because you didn't explicitly mention having considered it in your
deliberations above.

cheers,
  DaveK

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: GNU pth + cygwin + fork [Was: Re: fork failure?]

[Charles Wilson]

Dave Korn wrote:
   My first thought would be to figure out what pth is attempting to do while
 messing in jmp_buf, and make it work.  It's bad, unmaintainable code, that
 will break again in the future if ever jmp_buf is rearranged - but it only has
 to stagger along for another couple of months until you can do it right using
 sigaltstack.

I did consider that, but frankly this whole business of adjusting the
stack by messing around inside the undocumented jmp_buf scares the
*bleep* out of me.  WAY too easy to get wrong, and my asm skills are too
rusty -- and too non-x86 -- to be trusted.

 Until then, slapping a band-aid on pth might be a lot less
 work-that-soon-has-to-be-thrown-away than hacking both libassuan and gpg to
 handle a different API.  (I say this without having yet done the research to
 figure out exactly what pth thinks it is doing to that jmp_buf and whether
 it's necessarily possible, but it ought to be.)

I'm sure it's possible, but IMO migrating libassuan/gnupg is easier --
for me.  As it turns out, in libassuan there's only one file that needs
to be modified. gnupg is a bit trickier, but not terrible -- and the
APIs, while annoyingly different, aren't totally dissimilar.

--
Chuck

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

cygwin/fork slowness due to UNCs in PATH

[Song, Conrad W]

Just a quick comment.  I've been a long user of cygwin, and until recently 
noticed that cygwin was running extremely slow (it would take several seconds 
to fork off a command).  After debugging a while, it turns out that I had added 
UNC (\\ style windows) paths to my system PATH environment variable.  The 
removal of these (maybe slow network drives) from the PATH (or disconnection 
from the network) solved the problem for me.  Perhaps further investigation is 
warranted.  Thanks.

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cygwin/fork slowness due to UNCs in PATH

[Dave Korn]

On 01 February 2007 16:54, Song, Conrad W wrote:

 it turns out that
 I had added UNC (\\ style windows) paths to my system PATH environment
 variable.  The removal of these (maybe slow network drives) from the PATH
 (or disconnection from the network) solved the problem for me.  Perhaps
 further investigation is warranted.  

  Yep, like perhaps you should have investigated the FAQ?

http://cygwin.com/faq/faq.using.html#faq.using.slow


cheers,
  DaveK
-- 
Can't think of a witty .sigline today


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

cygwin fork()

[clayne]

Is it just me or is cygwin fork(), or a support syscall underneath,
terribly slow for some reason? While building projects using libtool
(which using heavy sh, hence fork() calls) I regularly have to fire off
'make -j16's just to get around waiting ages when using a single make job.

-cl

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cygwin fork()

[Dave Korn]

On 01 September 2006 11:02, [EMAIL PROTECTED] wrote:

 Is it just me or is cygwin fork(), or a support syscall underneath,
 terribly slow for some reason? 

  Some reason == lack of O/S support.


cheers,
  DaveK
-- 
Can't think of a witty .sigline today


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 11:12:59AM +0100, Dave Korn wrote:
  Is it just me or is cygwin fork(), or a support syscall underneath,
  terribly slow for some reason? 
 
   Some reason == lack of O/S support.

Yes I can understand that. I'm assuming there is some CreateProcess()
magic happening behind the scenes. However, what I've noticed is that it
is WAY slower than one would think it to be.

-cl

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cygwin fork()

[Dave Korn]

On 01 September 2006 11:22, [EMAIL PROTECTED] wrote:

 On Fri, Sep 01, 2006 at 11:12:59AM +0100, Dave Korn wrote:
 Is it just me or is cygwin fork(), or a support syscall underneath,
 terribly slow for some reason?
 
   Some reason == lack of O/S support.
 
 Yes I can understand that. I'm assuming there is some CreateProcess()
 magic happening behind the scenes.  However, what I've noticed is that it
 is WAY slower than one would think it to be.

  Why, how slow were *you* expecting it to be?  I was expecting it to be 7.


cheers,
  DaveK
-- 
Can't think of a witty .sigline today


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 11:12:59AM +0100, Dave Korn wrote:
 On 01 September 2006 11:02, [EMAIL PROTECTED] wrote:
 
  Is it just me or is cygwin fork(), or a support syscall underneath,
  terribly slow for some reason? 
 
   Some reason == lack of O/S support.

Basically, this is what I'm talking about:

Opteron 180, fast disks, ran make clean, make 4 times or so to keep the
disk/mem/cpu caches hot; no extraneous system usage going on:

$ ( make clean; time make -j1 ; make clean; time make -j8 ) /dev/null

real2m20.062s
user0m13.252s
sys 0m37.139s

real0m45.654s
user0m15.968s
sys 0m49.201s

That's a huge difference, and it's only like 6300 lines of code!  I've
narrowed it down to something between 20060309 and 20060313 snapshots. Every
snapshot since 20060313 has had this incredible slowness issue with fork().

The dual-coreness isn't going to make a huge difference here as I can get
the same factor of speed up just by using 20060309's dll and the exact
same make jobs. Nor is it a case of minor differences in times. I've tried
different versions of libtool, gcc, etc. Everything just goes back to
20060309 vs 20060313.

-cl

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Igor Peshansky]

On Fri, 1 Sep 2006, clayne wrote:

 Is it just me or is cygwin fork(), or a support syscall underneath,
 terribly slow for some reason? While building projects using libtool
 (which using heavy sh, hence fork() calls) I regularly have to fire off
 'make -j16's just to get around waiting ages when using a single make job.

While Cygwin is an *emulation* layer, and emulation is inherently slower
than straight execution, there are other potential reasons for the
slowness.  Check your anti-virus and firewall software settings.  If
possible, exclude the Cygwin filesystem from checking by those tools...
Even little things (like making the tool aware of the often-used Cygwin
programs and telling it to not check outbound email messages (!) sent by
those programs) can help speed up Cygwin...

A one-sentence rant: even with those changes, whenever I run a Cygwin
shell script, the stupid vsmon process takes up 100% of the CPU.  No known
solution.  Sigh...
Igor
-- 
http://cs.nyu.edu/~pechtcha/
  |\  _,,,---,,_[EMAIL PROTECTED] | [EMAIL PROTECTED]
ZZZzz /,`.-'`'-.  ;-;;,_Igor Peshansky, Ph.D. (name changed!)
 |,4-  ) )-,_. ,\ (  `'-'   old name: Igor Pechtchanski
'---''(_/--'  `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Las! je suis sot... -Mais non, tu ne l'es pas, puisque tu t'en rends compte.
But no -- you are no fool; you call yourself a fool, there's proof enough in
that! -- Rostand, Cyrano de Bergerac

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Igor Peshansky]

On Fri, 1 Sep 2006, clayne wrote:

 On Fri, Sep 01, 2006 at 11:12:59AM +0100, Dave Korn wrote:
  On 01 September 2006 11:02, [EMAIL PROTECTED] wrote:
 
   Is it just me or is cygwin fork(), or a support syscall underneath,
   terribly slow for some reason?
 
Some reason == lack of O/S support.

 Basically, this is what I'm talking about:

 Opteron 180, fast disks, ran make clean, make 4 times or so to keep the
 disk/mem/cpu caches hot; no extraneous system usage going on:

 $ ( make clean; time make -j1 ; make clean; time make -j8 ) /dev/null

 real2m20.062s
 user0m13.252s
 sys 0m37.139s

 real0m45.654s
 user0m15.968s
 sys 0m49.201s

 That's a huge difference, and it's only like 6300 lines of code!  I've
 narrowed it down to something between 20060309 and 20060313 snapshots. Every
 snapshot since 20060313 has had this incredible slowness issue with fork().

 The dual-coreness isn't going to make a huge difference here as I can get
 the same factor of speed up just by using 20060309's dll and the exact
 same make jobs. Nor is it a case of minor differences in times. I've tried
 different versions of libtool, gcc, etc. Everything just goes back to
 20060309 vs 20060313.

Ok, but now you're in a perfect position to debug this further.  There
were only 6 sets of changes between those two snapshots (see the
cygwin-cvs archives).  If you can build Cygwin as of those times and
pinpoint the change that made the difference, that in itself would
probably be helpful in diagnosing the problem.
Igor
-- 
http://cs.nyu.edu/~pechtcha/
  |\  _,,,---,,_[EMAIL PROTECTED] | [EMAIL PROTECTED]
ZZZzz /,`.-'`'-.  ;-;;,_Igor Peshansky, Ph.D. (name changed!)
 |,4-  ) )-,_. ,\ (  `'-'   old name: Igor Pechtchanski
'---''(_/--'  `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Las! je suis sot... -Mais non, tu ne l'es pas, puisque tu t'en rends compte.
But no -- you are no fool; you call yourself a fool, there's proof enough in
that! -- Rostand, Cyrano de Bergerac

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 10:33:47AM -0400, Igor Peshansky wrote:
 While Cygwin is an *emulation* layer, and emulation is inherently slower
 than straight execution, there are other potential reasons for the
 slowness.  Check your anti-virus and firewall software settings.  If
 possible, exclude the Cygwin filesystem from checking by those tools...
 Even little things (like making the tool aware of the often-used Cygwin
 programs and telling it to not check outbound email messages (!) sent by
 those programs) can help speed up Cygwin...

It's definitely none of those as I don't run any firewall or antivirus
software whatsoever on this box. Windows 2003 Server, minimal set of
services. The machine literally sits at 0% CPU unless I'm using it.

 A one-sentence rant: even with those changes, whenever I run a Cygwin
 shell script, the stupid vsmon process takes up 100% of the CPU.  No known
 solution.  Sigh...

Don't know what vsmon is.. A likely solution sounds to get rid of vsmon ;).

Also, not to break out the ole Linux vs Cygwin time comparisons type
deal, because we all know hot it's apples and oranges, in this case it's
a bit absurd a difference and I kinda have to. On a Celeron 2gz Linux
box w/512M RAM (which is WAY slower than a dual core Opteron 180
w/ 4GB of RAM):

real0m11.507s
user0m9.100s
sys 0m2.320s

11s! For the same make -j1 vs the 2m20s I got with make -j1 on my
20060718 Cygwin box. 11s is about 1/4th the time of the best I could get
my opteron to do with 'make -j8'. This is with all the exact same code.
Now we know libtool is intensive, but the libtool being used is the exact
same script (the build script is running a local generated version e.g.
/bin/sh ../libtool etc). All I did was copy the exact same build
directory to my linux host, cleaned it out, and ran the tests.

Yes apples to oranges for sure, but let's look at some figures here:

Linux 2.6.17.11, Celeron 2ghz/512MB/IDE drive,11.507s (make -j1)
Cygwin 20060718, Opteron [EMAIL PROTECTED]/4GB/U320SCSI, 2m20s   (make -j1)
Cygwin 20060718, Opteron [EMAIL PROTECTED]/4GB/U320SCSI, 45.654s (make -j8)

This is what specifically makes me wonder what is up. Just one small
segment of the build which I copied the commands from one to the other:


Linux (Celeron):
[EMAIL PROTECTED] lib]$ time if /bin/sh ../libtool --tag=CC --mode=compile gcc 
-DHAVE_CONFIG_H -I. -I. -I.. -I../lib-march=pentium4 -mtune=pentium4 
-mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo -c -o 
network.lo network.c; then mv -f .deps/network.Tpo .deps/network.Plo; else 
rm -f .deps/network.Tpo; exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib -march=pentium4 -mtune=pentium4 
-mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo -c network.c  
-fPIC -DPIC -o .libs/network.o
 gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib -march=pentium4 -mtune=pentium4 
-mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo -c network.c -o 
network.o /dev/null 21

real0m1.254s
user0m1.140s
sys 0m0.110s
[EMAIL PROTECTED] lib]$ time gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib 
-march=pentium4 -mtune=pentium4 -mcpu=pentium4 -Os -MT network.lo -MD -MP -MF 
.deps/network.Tpo -c network.c  -fPIC -DPIC -o .libs/network.o

real0m0.549s
user0m0.520s
sys 0m0.030s
[EMAIL PROTECTED] lib]$ time gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib 
-march=pentium4 -mtune=pentium4 -mcpu=pentium4 -Os -MT network.lo -MD -MP -MF 
.deps/network.Tpo -c network.c -o network.o /dev/null 21

real0m0.522s
user0m0.510s
sys 0m0.010s



Cygwin (Opteron):
$ time if /bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. 
-I. -I.. -I../lib-march=pentium4 -mtune=pentium4 -mcpu=pentium4 -Os -MT 
network.lo -MD -MP -MF .deps/network.Tpo -c -o network.lo network.c; then mv 
-f .deps/network.Tpo .deps/network.Plo; else rm -f .deps/network.Tpo; 
exit 1; fi
 gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib -march=pentium4 -mtune=pentium4 
-mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo -c network.c  
-DPIC -o .libs/network.o
 gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib -march=pentium4 -mtune=pentium4 
-mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo -c network.c -o 
network.o /dev/null 21

real0m8.032s
user0m1.166s
sys 0m2.216s

$ time gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib -march=pentium4 
-mtune=pentium4 -mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo 
-c network.c  -DPIC -o .libs/network.o

real0m0.506s
user0m0.249s
sys 0m0.061s

$ time gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../lib -march=pentium4 
-mtune=pentium4 -mcpu=pentium4 -Os -MT network.lo -MD -MP -MF .deps/network.Tpo 
-c network.c -o network.o /dev/null 21

real0m0.506s
user0m0.233s
sys 0m0.062s


Obviously it's not gcc, or the execution of binary code that is slow, Same 
libtool,
different systems, 8 times slower.

Just as a crude test, I dumped the output of that same libtool command but with

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 08:37:09AM -0700, [EMAIL PROTECTED] wrote:
 
 1157123322 + echo blah1
 1157123322 blah1
 3 seconds of doing absolutely nothing here
 1157123325 + test -z ''
 1157123325 + echo blah2
 1157123325 blah2

I just also copied the same libtool to 3 different hard drives, one
even a slow usb drive. Same results everytime. Even recompiling bash
and specifying that instead of /bin/sh. Same.

-cl

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 09:09:11AM -0700, [EMAIL PROTECTED] wrote:
 On Fri, Sep 01, 2006 at 11:54:03AM -0400, Christopher Faylor wrote:
  It's definitely none of those as I don't run any firewall or antivirus
  software whatsoever on this box. Windows 2003 Server, minimal set of
  services. The machine literally sits at 0% CPU unless I'm using it.
  
  Try using binary mounts instead of text mounts.
  
  cgf

BTW:
I started up filemon to watch what was going on from it's standpoint, and it
shows a huge number of READs on libtool, all SUCCESS, but the offset is always
1 higher than previous, with a length of 1. Like it's literally reading the file
1 byte at a time, then incrementing the offset - until it has fully been read.

-cl

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Igor Peshansky]

On Fri, 1 Sep 2006, Christopher Faylor wrote:

 On Fri, Sep 01, 2006 at 08:37:09AM -0700, [EMAIL PROTECTED] wrote:
 On Fri, Sep 01, 2006 at 10:33:47AM -0400, Igor Peshansky wrote:
  While Cygwin is an *emulation* layer, and emulation is inherently
  slower than straight execution, there are other potential reasons for
  the slowness.  Check your anti-virus and firewall software settings.
  If possible, exclude the Cygwin filesystem from checking by those
  tools... Even little things (like making the tool aware of the
  often-used Cygwin programs and telling it to not check outbound
  email messages (!) sent by those programs) can help speed up
  Cygwin...
 
 It's definitely none of those as I don't run any firewall or antivirus
 software whatsoever on this box. Windows 2003 Server, minimal set of
 services. The machine literally sits at 0% CPU unless I'm using it.

 Try using binary mounts instead of text mounts.

This thread already had two suggestions for possibly speeding up Cygwin...
Does anyone, as a service to the community, want to start a Cygwin
performance tuning web page and keep track?  It might even end up in the
User's Guide eventually...
Igor
-- 
http://cs.nyu.edu/~pechtcha/
  |\  _,,,---,,_[EMAIL PROTECTED] | [EMAIL PROTECTED]
ZZZzz /,`.-'`'-.  ;-;;,_Igor Peshansky, Ph.D. (name changed!)
 |,4-  ) )-,_. ,\ (  `'-'   old name: Igor Pechtchanski
'---''(_/--'  `-'\_) fL a.k.a JaguaR-R-R-r-r-r-.-.-.  Meow!

Las! je suis sot... -Mais non, tu ne l'es pas, puisque tu t'en rends compte.
But no -- you are no fool; you call yourself a fool, there's proof enough in
that! -- Rostand, Cyrano de Bergerac

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 09:34:15AM -0700, [EMAIL PROTECTED] wrote:
 BTW:
 I started up filemon to watch what was going on from it's standpoint, and it
 shows a huge number of READs on libtool, all SUCCESS, but the offset is always
 1 higher than previous, with a length of 1. Like it's literally reading the 
 file
 1 byte at a time, then incrementing the offset - until it has fully been read.
 
 -cl

So right smack dab in the middle of _evalfile() under bash-3.1:

#if defined (__CYGWIN__)  defined (O_TEXT)
  setmode (fd, O_TEXT);
#endif

Also in read_comsub():

#ifdef __CYGWIN__
  setmode (fd, O_TEXT); /* we don't want CR/LF, we want Unix-style */
#endif

And most importantly in open_shell_script():

  /* Open the script.  But try to move the file descriptor to a randomly
 large one, in the hopes that any descriptors used by the script will
 not match with ours. */
  fd = move_to_high_fd (fd, 0, -1);

#if defined (__CYGWIN__)  defined (O_TEXT)
  setmode (fd, O_TEXT);
#endif

The high fd part jives with the '255' seen in the readv() strace output as well.

This post from 2000 looks related:

http://ecos.sourceware.org/ml/cygwin/2000-10/msg00213.html

In regards to setting the fd to textmode as a way of stripping CRs. Only problem
is that it's making 213,110 syscalls for a 213k libtool script. That cannot be 
an
efficient way to remove CRs from input.

Found the old references to that too:

#if 0
#if defined (__CYGWIN__)
  if (c == '\n'  istring_index  1  istring[istring_index - 2] == '\r')
{
  istring_index--;
  istring[istring_index - 1] = '\n';
}
#endif
#endif

I've since removed the setmode() calls within a bash build and am testing now.

UPDATE:
SOLVED.

Filemon now shows bash reading in 8k chunks. There is now no 3 second delay on
reading the rest of the bash script (which I evidenced earlier as well).


p.s. Seriously old stuff in there, for example:

#if defined (__CYGWIN__)
  /* Under CygWin 1.1.0, the unlink will fail if the file is
 open. This hack will allow the previous action of silently
 ignoring the error, but will still leave the file there. This
 needs some kind of magic. */
  if (r == EACCES)
return (fd2);
#endif /* __CYGWIN__ */


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 10:04:51AM -0700, [EMAIL PROTECTED] wrote:
 I've since removed the setmode() calls within a bash build and am testing now.
 
 UPDATE:
 SOLVED.
 
 Filemon now shows bash reading in 8k chunks. There is now no 3 second delay on
 reading the rest of the bash script (which I evidenced earlier as well).

I also changed the input buffer size to be 32768, as opposed to 8192, and
additionally enabled mmap() (since bash source had a stone age define in there
indicating cygwin did not support mmap()).

New 'make -j8':

real0m16.806s
user0m6.458s
sys 0m13.328s

vs.

44 seconds previously.

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Christopher Faylor]

On Fri, Sep 01, 2006 at 10:04:51AM -0700, [EMAIL PROTECTED] wrote:
On Fri, Sep 01, 2006 at 09:34:15AM -0700, [EMAIL PROTECTED] wrote:
 BTW:
 I started up filemon to watch what was going on from it's standpoint, and it
 shows a huge number of READs on libtool, all SUCCESS, but the offset is 
 always
 1 higher than previous, with a length of 1. Like it's literally reading the 
 file
 1 byte at a time, then incrementing the offset - until it has fully been 
 read.
 
 -cl

So right smack dab in the middle of _evalfile() under bash-3.1:

#if defined (__CYGWIN__)  defined (O_TEXT)
  setmode (fd, O_TEXT);
#endif

Also in read_comsub():

#ifdef __CYGWIN__
  setmode (fd, O_TEXT); /* we don't want CR/LF, we want Unix-style */
#endif

And most importantly in open_shell_script():

  /* Open the script.  But try to move the file descriptor to a randomly
 large one, in the hopes that any descriptors used by the script will
 not match with ours. */
  fd = move_to_high_fd (fd, 0, -1);

#if defined (__CYGWIN__)  defined (O_TEXT)
  setmode (fd, O_TEXT);
#endif

The high fd part jives with the '255' seen in the readv() strace output as 
well.

This post from 2000 looks related:

http://ecos.sourceware.org/ml/cygwin/2000-10/msg00213.html

In regards to setting the fd to textmode as a way of stripping CRs.
Only problem is that it's making 213,110 syscalls for a 213k libtool
script.  That cannot be an efficient way to remove CRs from input.

Opening a file with O_TEXT should not, AFAIK, cause a bunch of one-byte
reads.

A simple test case (tm) seems to confirm that.

cgf

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[clayne]

On Fri, Sep 01, 2006 at 01:24:57PM -0400, Christopher Faylor wrote:
 In regards to setting the fd to textmode as a way of stripping CRs.
 Only problem is that it's making 213,110 syscalls for a 213k libtool
 script.  That cannot be an efficient way to remove CRs from input.
 
 Opening a file with O_TEXT should not, AFAIK, cause a bunch of one-byte
 reads.
 
 A simple test case (tm) seems to confirm that.
 
 cgf

You're right. I also verified this.

I found the real culprit, which I had also ifdef'd out because it looked
bogus and crufty:

/* Return 1 if a seek on FD will succeed. */
#ifndef __CYGWIN__
#  define fd_is_seekable(fd) (lseek ((fd), 0L, SEEK_CUR) = 0)
#else
#  define fd_is_seekable(fd) 0
#endif /* __CYGWIN__ */

/* Take FD, a file descriptor, and create and return a buffered stream
   corresponding to it.  If something is wrong and the file descriptor
   is invalid, return a NULL stream. */
BUFFERED_STREAM *
fd_to_buffered_stream (fd)
 int fd;
{
  char *buffer;
  size_t size;
  struct stat sb;

  if (fstat (fd, sb)  0)
{
  close (fd);
  return ((BUFFERED_STREAM *)NULL);
}

  size = (fd_is_seekable (fd)) ? min (sb.st_size, MAX_INPUT_BUFFER_SIZE) : 1;
  if (size == 0)
size = 1;
  buffer = (char *)xmalloc (size);

  return (make_buffered_stream (fd, buffer, size));
}



--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

bash problems ? (was Re: cygwin fork())

[Christopher Faylor]

I'm just changing the subject so that Eric will, with luck, notice
and comment on what's going on here.

cgf

On Fri, Sep 01, 2006 at 10:47:13AM -0700, [EMAIL PROTECTED] wrote:
On Fri, Sep 01, 2006 at 01:24:57PM -0400, Christopher Faylor wrote:
 In regards to setting the fd to textmode as a way of stripping CRs.
 Only problem is that it's making 213,110 syscalls for a 213k libtool
 script.  That cannot be an efficient way to remove CRs from input.
 
 Opening a file with O_TEXT should not, AFAIK, cause a bunch of one-byte
 reads.
 
 A simple test case (tm) seems to confirm that.

You're right. I also verified this.

I found the real culprit, which I had also ifdef'd out because it looked
bogus and crufty:

/* Return 1 if a seek on FD will succeed. */
#ifndef __CYGWIN__
#  define fd_is_seekable(fd) (lseek ((fd), 0L, SEEK_CUR) = 0)
#else
#  define fd_is_seekable(fd) 0
#endif /* __CYGWIN__ */

/* Take FD, a file descriptor, and create and return a buffered stream
   corresponding to it.  If something is wrong and the file descriptor
   is invalid, return a NULL stream. */
BUFFERED_STREAM *
fd_to_buffered_stream (fd)
 int fd;
{
  char *buffer;
  size_t size;
  struct stat sb;

  if (fstat (fd, sb)  0)
{
  close (fd);
  return ((BUFFERED_STREAM *)NULL);
}

  size = (fd_is_seekable (fd)) ? min (sb.st_size, MAX_INPUT_BUFFER_SIZE) : 1;
  if (size == 0)
size = 1;
  buffer = (char *)xmalloc (size);

  return (make_buffered_stream (fd, buffer, size));
}

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cygwin fork()

[Dave Korn]

On 01 September 2006 18:47, [EMAIL PROTECTED] wrote:


 I found the real culprit, which I had also ifdef'd out because it looked
 bogus and crufty:
 
 /* Return 1 if a seek on FD will succeed. */
 #ifndef __CYGWIN__
 #  define fd_is_seekable(fd) (lseek ((fd), 0L, SEEK_CUR) = 0)
 #else
 #  define fd_is_seekable(fd) 0
 #endif /* __CYGWIN__ */

  Yeeesh.  This is a terrible way of dealing with the fact that you can't seek
a stream accurately if you open it in text mode, because of the ambiguity
about whether you've advanced one or two chars through the underlying file
when you see an LF that could perhaps have actually been a CR/LF.  What we
really want is 

#define fd_is_seekable(fd)  fd_is_in_binary_not_text_mode(fd)

...although of course I'm paraphrasing there.

  Anyway, congratulations, well tracked down!  Definitely one to raise with
upstream...




cheers,
  DaveK
-- 
Can't think of a witty .sigline today


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Christopher Faylor]

On Fri, Sep 01, 2006 at 06:57:10PM +0100, Dave Korn wrote:
On 01 September 2006 18:47, [EMAIL PROTECTED] wrote:
I found the real culprit, which I had also ifdef'd out because it looked
bogus and crufty:

/* Return 1 if a seek on FD will succeed. */
#ifndef __CYGWIN__
#  define fd_is_seekable(fd) (lseek ((fd), 0L, SEEK_CUR) = 0)
#else
#  define fd_is_seekable(fd) 0
#endif /* __CYGWIN__ */

Yeeesh.  This is a terrible way of dealing with the fact that you can't
seek a stream accurately if you open it in text mode, because of the
ambiguity about whether you've advanced one or two chars through the
underlying file when you see an LF that could perhaps have actually
been a CR/LF.  What we really want is

AFAIK, Cygwin's lseek should handle seeking on text streams.  DJ
implemented that years ago.

cgf

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: cygwin fork()

[Gary R. Van Sickle]

 From: Christopher Faylor
 Sent: Friday, September 01, 2006 1:01 PM
 Subject: Re: cygwin fork()
 
 On Fri, Sep 01, 2006 at 06:57:10PM +0100, Dave Korn wrote:
 On 01 September 2006 18:47, clayne wrote:
 I found the real culprit, which I had also ifdef'd out because it 
 looked bogus and crufty:
 
 /* Return 1 if a seek on FD will succeed. */ #ifndef __CYGWIN__ #  
 define fd_is_seekable(fd) (lseek ((fd), 0L, SEEK_CUR) = 0) 
 #else #  
 define fd_is_seekable(fd) 0 #endif /* __CYGWIN__ */
 
 Yeeesh.  This is a terrible way of dealing with the fact 
 that you can't 
 seek a stream accurately if you open it in text mode, because of the 
 ambiguity about whether you've advanced one or two chars through the 
 underlying file when you see an LF that could perhaps have actually 
 been a CR/LF.  What we really want is
 
 AFAIK, Cygwin's lseek should handle seeking on text streams.  
 DJ implemented that years ago.
 

Last I looked, which was admittedly also years ago, it was #if 0'ed out,
with a comment to the effect of Nobody has any business seeking around in
text files.

-- 
Gary R. Van Sickle
 


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Brian Dessent]

Gary R. Van Sickle wrote:

  AFAIK, Cygwin's lseek should handle seeking on text streams.
  DJ implemented that years ago.
 
 Last I looked, which was admittedly also years ago, it was #if 0'ed out,
 with a comment to the effect of Nobody has any business seeking around in
 text files.

FWIW, mingw's lseek() (which is actually Microsoft's, since mingw
targets MSVCRT.DLL) is horribly broken when seeking on a file opened in
text mode.  But it's documented as such on MSDN, so at least there's
that.  So there is some precedence for the concept that this won't work
on windows platforms.

But Cygwin's should work fine, and if it means saving a bazillion 1-byte
syscalls then I think bash should be patched to remove this sillyness
ASAP.

Brian

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[mwoehlke]

Brian Dessent wrote:

Gary R. Van Sickle wrote:

AFAIK, Cygwin's lseek should handle seeking on text streams.
DJ implemented that years ago.

Last I looked, which was admittedly also years ago, it was #if 0'ed out,
with a comment to the effect of Nobody has any business seeking around in
text files.


FWIW, mingw's lseek() (which is actually Microsoft's, since mingw
targets MSVCRT.DLL) is horribly broken when seeking on a file opened in
text mode.  But it's documented as such on MSDN, so at least there's
that.  So there is some precedence for the concept that this won't work
on windows platforms.

But Cygwin's should work fine, and if it means saving a bazillion 1-byte
syscalls then I think bash should be patched to remove this sillyness
ASAP.


I'll *emphatically* second the 'doing something about it'; I'm also 
cursed by really slow builds. (Assuming I understand what is going on, 
anyway :-))


--
Matthew
Do not expose to hippos. Doing so may void your warranty.


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Christopher Faylor]

On Fri, Sep 01, 2006 at 05:07:49PM -0500, Gary R. Van Sickle wrote:
 From: Christopher Faylor
 Sent: Friday, September 01, 2006 1:01 PM
 Subject: Re: cygwin fork()
 
 On Fri, Sep 01, 2006 at 06:57:10PM +0100, Dave Korn wrote:
 On 01 September 2006 18:47, clayne wrote:
 I found the real culprit, which I had also ifdef'd out because it 
 looked bogus and crufty:
 
 /* Return 1 if a seek on FD will succeed. */ #ifndef __CYGWIN__ #  
 define fd_is_seekable(fd) (lseek ((fd), 0L, SEEK_CUR) = 0) 
 #else #  
 define fd_is_seekable(fd) 0 #endif /* __CYGWIN__ */
 
 Yeeesh.  This is a terrible way of dealing with the fact 
 that you can't 
 seek a stream accurately if you open it in text mode, because of the 
 ambiguity about whether you've advanced one or two chars through the 
 underlying file when you see an LF that could perhaps have actually 
 been a CR/LF.  What we really want is
 
 AFAIK, Cygwin's lseek should handle seeking on text streams.  
 DJ implemented that years ago.

Last I looked, which was admittedly also years ago, it was #if 0'ed out,
with a comment to the effect of Nobody has any business seeking around in
text files.

AFAIK, lseek is supposed to work.  There was some old pre-DJ code which
I believe was ifdef'ed out up until around 2002 when Corinna introduced
64-bit support.

However, I'm sure there are probably corner cases where it breaks down.
However, determining the position in a file using lseek and then seeking
to that position later should work ok.

cgf


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork()

[Eric Blake]

 But Cygwin's should work fine, and if it means saving a bazillion 1-byte
 syscalls then I think bash should be patched to remove this sillyness
 ASAP.

I get the hint!  Look for a new release of bash in the near future; and
once I get the cygport build going, I will also try to get the bash-3.2
alpha up as well.

-- 
Eric Blake
volunteer cygwin bash maintainer

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork problem maybe?

[Linda Walsh]

Brian Dessent wrote:

Linda W wrote:

  

Windows just doesn't support forking at all, as far as I know.
activeperl emulates forking using win32 threads. I don'tknow how cygwin
handles it, but my guess is that it's not very well :-(



This smells like total FUD.  This person that admittedly does not use
Cygwin nor obviously know anything about Cygwin says that it's a fork
problem and that's that? 

=
   They don't have a windows machine to test with, but they
are trying to help pinpoint a problem that works under linux
and ActiveState, which seems like the most useful response
to date.  I can't give them a Windows machine or a cygwin
install to test with.  I can run the programs and report on
the results, and I can get a copy of the source to you that
fails.  Is there anything else I could do to be helpful?

:-)
Linda


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork problem maybe?

[Yitzchak Scott-Thoennes]

On Tue, Jun 06, 2006 at 01:30:43PM -0700, Linda W wrote:
 A problem I reported in trying to install File::BOM (module to handle 
 files with Unicode Byte-Order-Marks) under CPAN originally here:
 http://sourceware.org/ml/cygwin/2006-02/msg00238.html last February
 is still a problem.  I bounced it off of the module maintainer, and he
 believes the problem has to do with forking, not FIFO use.
 
 His test works under AS Perl, but he doesn't think he'll be able to
 get his module to work under cygwin with these problems.

Can he or you reduce the problem to a non-File::BOM dependent test script?

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

cygwin fork problem maybe?

[Linda W]

A problem I reported in trying to install File::BOM (module to handle files 
with Unicode Byte-Order-Marks) under CPAN originally here:

http://sourceware.org/ml/cygwin/2006-02/msg00238.html last February
is still a problem.  I bounced it off of the module maintainer, and he
believes the problem has to do with forking, not FIFO use.

His test works under AS Perl, but he doesn't think he'll be able to
get his module to work under cygwin with these problems.

A summary snippet of the last email I got from him is included below.

The symptoms of the hung processes are still the same -- both instances
of perl running the test completely lock-up and are unkillable in the cygwin
environment

Is there any chance of getting someone to look at this problem?

I'm rather depressed at the number of Win32 modules that don't work under
cygwin.  I know this may be considered a feature by some, but if I wanted
a POSIXish environment that was incompatible with Win32, I'd run linux, but
I want a *nix environment that lets me access the native system's
features.  If it was only a few Win32 modules that had problems, that'd be
one thing, but I feel overwhelmed by the number of incompatibilities.

Anyway I didn't know fork was such a problem on Windows...

thanks for any help on this...
-linda



 Original Message 
Subject: Re: [perl #39211] cygperl hangs on FIFO use (attempt install of CPAN 
File::BOM hangs during test using FIFO)

Date: Tue, 06 Jun 2006 12:18:09 +0100
From: CPAN File::BOM module maintainer

...
I would say it's definitely fork() that's the problem, not fifo support.
...
For what it's worth, the tests succeed on ActivePerl.



I know that fork is a bit of a bugbear for the activeperl team. forking
on win32 is probably always going to be a problem. *sigh*
... 

Windows just doesn't support forking at all, as far as I know.
activeperl emulates forking using win32 threads. I don'tknow how cygwin
handles it, but my guess is that it's not very well :-(




Those exception test results seem to indicate that there's no problem
reading from filehandles that were opened for writing, or even reading
from non-existent files.



...

The tests that are failing are these:
open WRITER, '', $tmpfile or die Couldn't write to '$tmpfile': $!;

throws_ok { File::BOM::_get_encoding_seekable(\*WRITER) }
qr/^Couldn't read from handle/,
_get_encoding_seekable on unreadable handle fails;

throws_ok { File::BOM::_get_encoding_unseekable(\*WRITER) }
qr/^Couldn't read byte/,
_get_encoding_unseekable() on unreadable handle fails;


The first one, at least, returns normally, which indicates that
it succeeded in reading from the WRITER handle, which was
opened for writing.

One of the other failing tests tried to
read from a generated (non-existent) tempfile name, and also didn't die.

...

I don't think I'm going to be able to fix the tests to pass on cygwin,
other than testing $^O and skipping all forking and exception tests,
which seems a bit like cheating.





--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: cygwin fork problem maybe?

[Brian Dessent]

Linda W wrote:

 Windows just doesn't support forking at all, as far as I know.
 activeperl emulates forking using win32 threads. I don'tknow how cygwin
 handles it, but my guess is that it's not very well :-(

This smells like total FUD.  This person that admittedly does not use
Cygwin nor obviously know anything about Cygwin says that it's a fork
problem and that's that?  Fork may be a problem for Activestate but
Cygwin fully emulates fork just fine, as does Cygwin's perl.  The Cygwin
DLL goes to extreme lengths to make sure this work.  Just about every
Cygwin compiled app that invokes a subprogram uses fork in some form,
for example every shell command you type is a fork/exec.

my guess is that it's not very well without any specific indication as
to why or how fork is even related seems like nonsense to me.  I
certainly don't know why this module is failing, but just because
Activestate's implementation of fork is terrible doesn't mean Cygwin's
is too.

Brian

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Cygwin fork implementation

[Sudhahar]

Hi,
   In Cygwin fork code the statement

int res = setjmp (grouped.ch.jmp);
  if (res)
res = fork_child (grouped.ch.parent, grouped.first_dll, grouped.load_dlls);
  else
res = fork_parent (grouped.ch.parent, grouped.first_dll,
grouped.load_dlls, esp, grouped.ch);

avoids the fork being called repeatedly by the created process of
parent. Can anyone tell me how this actually happens. Thanks in
advance for your time and comments.

Thanks
Sudhakar

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Cygwin fork implementation

[Eric Blake]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

According to Sudhahar on 2/6/2006 5:33 AM:
 Hi,
In Cygwin fork code the statement
 
 int res = setjmp (grouped.ch.jmp);
   if (res)
 res = fork_child (grouped.ch.parent, grouped.first_dll, 
 grouped.load_dlls);
   else
 res = fork_parent (grouped.ch.parent, grouped.first_dll,
 grouped.load_dlls, esp, grouped.ch);
 
 avoids the fork being called repeatedly by the created process of
 parent. Can anyone tell me how this actually happens. Thanks in
 advance for your time and comments.

If you are asking how setjmp works, it is not cygwin specific.  Any good
systems C programming book should give you more details, or you can read
what POSIX says:
http://www.opengroup.org/onlinepubs/009695399/functions/setjmp.html.

setjmp() is one of those MAGIC functions, that when paired with the
longjmp() in fork_parent(), returns 2 distinct values at different points
during execution.

You may also be interested in
http://cygwin.com/cgi-bin/cvsweb.cgi/src/winsup/cygwin/how-cygheap-works.txt?rev=1.5content-type=text/x-cvsweb-markupcvsroot=src

- --
Life is short - so eat dessert first!

Eric Blake [EMAIL PROTECTED]
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFD51Z484KuGfSFAYARAvXNAJ0cgMN8AqYiOFeY7oVKQcHe5EvJ/ACgpFfU
l4KjZDRVF39JKEI41g/mYAc=
=yYHr
-END PGP SIGNATURE-

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Cygwin fork failure

[David Svoboda]

Larry Hall (Cygwin) wrote:


[EMAIL PROTECTED] wrote:


On Saturday 19 November 2005 01:07 am, Larry Hall (Cygwin) wrote:


David Svoboda wrote:


I have built a C++ server program called 'analyzer'. But when I try to
run it, it fails upon trying to execute a 'fork' command. Why?





snip




If running 'rebaseall' doesn't help (see the README) and 'analyzer' 
doesn't

use any DLLs that were built as part of that package (in which case you
need to tell 'rebaseall' about them), then try:

http://sources.redhat.com/ml/cygwin/2005-09/msg00945.html




Thanks for the response. The 'analyzer' program is statically linked; 
it involves no DLLs. Rebaseall did nothing. And increasing the size 
of the heaps did nothing (I tried 2x, 4x, 8x and 16x). My machine is 
totally wedged now...even safe mode doesn't work.


I can only say that this fork problem happens on several other 
Windows machines at our university, but it does not seem to happen to 
machines not at the university. Furthermore this fork problem now 
affects previous versions of my 'analyzer' software...last week's 
version worked then, but does not now. It's as if  Satan put a curse 
on all the PCs only at this university.


Any other suggestions?




Trying a snapshot it a good subsequent step.  If you're going to be 
building
things against Cygwin, you'll want to pull the cygwin-inst.  
Otherwise, cygwin1-*.dll is enough and is probably plenty for starters.



http://cygwin.com/snapshots/



Well, the latest snapshot available is 2005-Nov-17. Planting its
cygwin1.dll into cygwin/bin generates the following result:

   analyzer --KANTOO_LANGUAGE=halo --ANALYZER_PORT=4052 
-server-multiple -fork

   Analyzer 2.0; DMK 0.2 (HALO); DOMO 0.2 (KCE)
 6 [main] analyzer 2164 
fhandler_disk_file::fixup_mmap_after_fork: requested 0x1873 != 0x0 
mem alloc base 0x1873, state 0x1000, size 4096, Win32 error 487
   d:\svoboda\kantoo\bin\analyzer (2164): *** 
recreate_mmaps_after_fork_failed

   ]0;~
   [EMAIL PROTECTED] ~
   $

which is a slight improvement over the previous result:

   [EMAIL PROTECTED] ~
   $ analyzer --KANTOO_LANGUAGE=halo --ANALYZER_PORT=4052 
-server-multiple -fork

   Analyzer 2.0; DMK 0.2 (HALO); DOMO 0.2 (KCE)
 6 [main] analyzer 224 
fhandler_disk_file::fixup_mmap_after_fork: requested 0x1873 != 0x0 
mem alloc base 0x1873, state 0x1000, size 4096, Win32 error 487
   d:\svoboda\kantoo\bin\analyzer (224): *** 
recreate_mmaps_after_fork_failed
11 [main] analyzer 1136 fork_parent: child 224 died waiting for 
dll loading

   Failed to fork properly!
   ]0;~
   [EMAIL PROTECTED] ~
   $

but still crashes my analyzer.

My best guess is that this bug was introduced by a new version of
cygwin, as it did not occur until I did a cygwin-update.

It seems that the oldest cygwin I can get is 1.5.17.1, (1.5.18.1 being 
the current version) Is there any way to get older c ygwins?

~Dave


~Dave


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Cygwin fork failure

[Larry Hall (Cygwin)]

On 11/23/2005, David Svoboda wrote:

My best guess is that this bug was introduced by a new version of
cygwin, as it did not occur until I did a cygwin-update.




But didn't you claim that switching back to an older version of Cygwin
didn't help?


It seems that the oldest cygwin I can get is 1.5.17.1, (1.5.18.1 being the 
current version) Is there any way to get older c ygwins?
~Dave 



There's the Cygwin Time Machine.  Look it up in the email archives if you're
interested.  And note that use of the Cygwin Time Machine isn't supported by
this list.

--
Larry Hall  http://www.rfk.com
RFK Partners, Inc.  (508) 893-9779 - RFK Office
838 Washington Street   (508) 893-9889 - FAX
Holliston, MA 01746

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Cygwin fork failure

[Larry Hall (Cygwin)]

[EMAIL PROTECTED] wrote:

On Saturday 19 November 2005 01:07 am, Larry Hall (Cygwin) wrote:


David Svoboda wrote:


I have built a C++ server program called 'analyzer'. But when I try to
run it, it fails upon trying to execute a 'fork' command. Why?



snip




If running 'rebaseall' doesn't help (see the README) and 'analyzer' doesn't
use any DLLs that were built as part of that package (in which case you
need to tell 'rebaseall' about them), then try:

http://sources.redhat.com/ml/cygwin/2005-09/msg00945.html



Thanks for the response. The 'analyzer' program is statically linked; it 
involves no DLLs. Rebaseall did nothing. And increasing the size of the heaps 
did nothing (I tried 2x, 4x, 8x and 16x). My machine is totally wedged 
now...even safe mode doesn't work.


I can only say that this fork problem happens on several other Windows 
machines at our university, but it does not seem to happen to machines not at 
the university. Furthermore this fork problem now affects previous versions 
of my 'analyzer' software...last week's version worked then, but does not 
now. It's as if  Satan put a curse on all the PCs only at this university.


Any other suggestions?



Trying a snapshot it a good subsequent step.  If you're going to be building
things against Cygwin, you'll want to pull the cygwin-inst.  Otherwise, 
cygwin1-*.dll is enough and is probably plenty for starters.



http://cygwin.com/snapshots/


--
Larry Hall  http://www.rfk.com
RFK Partners, Inc.  (508) 893-9779 - RFK Office
838 Washington Street   (508) 893-9889 - FAX
Holliston, MA 01746

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Cygwin fork failure

[svoboda]

On Saturday 19 November 2005 01:07 am, Larry Hall (Cygwin) wrote:
 David Svoboda wrote:
  I have built a C++ server program called 'analyzer'. But when I try to
  run it, it fails upon trying to execute a 'fork' command. Why?
 
 
 $ analyzer --KANTOO_LANGUAGE=toy -server-multiple -fork
 Analyzer 2.0
  ***  5 [main] analyzer 1712
  fhandler_disk_file::fixup_mmap_after_fork: requested 0x1873 != 0x0
  mem alloc base 0x1873, state 0x1000, size 4096, Win32 error 487
  *** d:\svoboda\kantoo\bin\analyzer (1712): ***
  recreate_mmaps_after_fork_failed
  *** 9 [main] analyzer 2200 fork_parent: child 1712 died waiting for
  dll loading
  *** Failed to fork properly!
  ]0;~
  [EMAIL PROTECTED] ~
  $
 
  The *** lines indicate the errors.
 
  Version info: The analyzer was built with G++ 3.3.3, on WinXPSP2. Here
  is the output of cygcheck. So why does the fork call fail?

 If running 'rebaseall' doesn't help (see the README) and 'analyzer' doesn't
 use any DLLs that were built as part of that package (in which case you
 need to tell 'rebaseall' about them), then try:

 http://sources.redhat.com/ml/cygwin/2005-09/msg00945.html

Thanks for the response. The 'analyzer' program is statically linked; it 
involves no DLLs. Rebaseall did nothing. And increasing the size of the heaps 
did nothing (I tried 2x, 4x, 8x and 16x). My machine is totally wedged 
now...even safe mode doesn't work.

I can only say that this fork problem happens on several other Windows 
machines at our university, but it does not seem to happen to machines not at 
the university. Furthermore this fork problem now affects previous versions 
of my 'analyzer' software...last week's version worked then, but does not 
now. It's as if  Satan put a curse on all the PCs only at this university.

Any other suggestions?
~Dave



--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Cygwin fork failure

[David Svoboda]

I have built a C++ server program called 'analyzer'. But when I try to
run it, it fails upon trying to execute a 'fork' command. Why?


   $ analyzer --KANTOO_LANGUAGE=toy -server-multiple -fork
   Analyzer 2.0
***  5 [main] analyzer 1712 
fhandler_disk_file::fixup_mmap_after_fork: requested 0x1873 != 0x0 
mem alloc base 0x1873, state 0x1000, size 4096, Win32 error 487
*** d:\svoboda\kantoo\bin\analyzer (1712): *** 
recreate_mmaps_after_fork_failed
*** 9 [main] analyzer 2200 fork_parent: child 1712 died waiting for 
dll loading

*** Failed to fork properly!
]0;~
[EMAIL PROTECTED] ~
$

The *** lines indicate the errors.

Version info: The analyzer was built with G++ 3.3.3, on WinXPSP2. Here
is the output of cygcheck. So why does the fork call fail?
~Dave


Cygwin Configuration Diagnostics
Current System Time: Fri Nov 18 11:23:33 2005

Windows XP Professional Ver 5.1 Build 2600 Service Pack 2

Path:   d:\svoboda\kantoo\bin
C:\cygwin\usr\local\bin
C:\cygwin\bin
C:\cygwin\bin
C:\cygwin\usr\X11R6\bin
C:\cygwin\bin
c:\WINNT\system32
c:\WINNT
c:\WINNT\System32\Wbem
c:\Program Files\ATI Technologies\ATI Control Panel
c:\Program Files\OpenAFS\Common
c:\Program Files\OpenAFS\Client\Program
c:\Program Files\MIT\Kerberos\bin
c:\Program Files\Java\j2re1.4.2_10\bin

Output from C:\cygwin\bin\id.exe (nontsec)
UID: 15467(svoboda) GID: 10545(mkgroup-l-d)
0(root) 544(Administrators) 545(Users)
10545(mkgroup-l-d)

Output from C:\cygwin\bin\id.exe (ntsec)
UID: 15467(svoboda) GID: 10545(mkgroup-l-d)
0(root) 544(Administrators) 545(Users)
10545(mkgroup-l-d)

SysDir: C:\WINNT\system32
WinDir: C:\WINNT

USER = `svoboda'
PWD = `/cygdrive/c/Documents and Settings/svoboda/Desktop'
HOME = `/cygdrive/c'
MAKE_MODE = `unix'

HOMEPATH = `\Documents and Settings\svoboda'
KANTOO_LANGUAGE = `halo'
MANPATH = `/usr/local/man:/usr/share/man:/usr/man::/usr/ssl/man'
APPDATA = `C:\Documents and Settings\svoboda\Application Data'
HOSTNAME = `quebec'
SVOBODA = `/cygdrive/d/svoboda'
TERM = `emacs'
SHELL = `bash'
PROCESSOR_IDENTIFIER = `x86 Family 15 Model 4 Stepping 3, GenuineIntel'
WINDIR = `C:\WINNT'
TMPDIR = `/cygdrive/c/DOCUME~1/svoboda/LOCALS~1/Temp'
EMACSDATA = `C:/Program Files/emacs-21.3/etc'
OLDPWD = `/cygdrive/c/Documents and Settings'
EMACSPATH = `C:/Program Files/emacs-21.3/bin'
CCC = `g++'
USERDOMAIN = `SCS'
OS = `Windows_NT'
ALLUSERSPROFILE = `C:\Documents and Settings\All Users'
KANTOO = `/cygdrive/d/svoboda/kantoo'
EMACSLOCKDIR = `C:/Program Files/emacs-21.3/lock'
EMACS = `t'
TEMP = `/cygdrive/c/DOCUME~1/svoboda/LOCALS~1/Temp'
COMMONPROGRAMFILES = `C:\Program Files\Common Files'
SANDBOX = `/cygdrive/d/svoboda/kantoo'
EM_PARENT_PROCESS_ID = `4048'
TERMCAP = `emacs:co#80:tc=unknown:'
USERNAME = `svoboda'
PROCESSOR_LEVEL = `15'
FP_NO_HOST_CHECK = `NO'
SYSTEMDRIVE = `C:'
USERPROFILE = `C:\Documents and Settings\svoboda'
LANG = `ENU'
CLIENTNAME = `Console'
PS1 = `\[\e]0;[EMAIL PROTECTED] \[\e[33m\]\w\[\e[0m\]\n\$ '
LOGONSERVER = `\\GEMSTONE'
EMACS_DIR = `C:/Program Files/emacs-21.3'
PROCESSOR_ARCHITECTURE = `x86'
EMACSLOADPATH = `C:/Program Files/emacs-21.3/site-lisp;C:/Program 
Files/emacs-21.3/../site-lisp;C:/Program Files/emacs-21.3/lisp;C:/Program 
Files/emacs-21.3/leim'
SHLVL = `1'
USERDNSDOMAIN = `SCS.AD.CS.CMU.EDU'
PATHEXT = `.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH'
HOMEDRIVE = `C:'
COMSPEC = `C:\WINNT\system32\cmd.exe'
TMP = `/cygdrive/c/DOCUME~1/svoboda/LOCALS~1/Temp'
SYSTEMROOT = `C:\WINNT'
PRINTER = `Microsoft Office Document Image Writer'
CVS_RSH = `/bin/ssh'
PROCESSOR_REVISION = `0403'
PKG_CONFIG_PATH = `/usr/X11R6/lib/pkgconfig'
DESKTOP = `/cygdrive/d/svoboda/Desktop'
INFOPATH = `/usr/local/info:/usr/share/info:/usr/info:'
PROGRAMFILES = `C:\Program Files'
NUMBER_OF_PROCESSORS = `2'
SESSIONNAME = `Console'
EMACSDOC = `C:/Program Files/emacs-21.3/etc'
COMPUTERNAME = `QUEBEC'
SONICCENTRAL = `C:\Program Files\Common Files\Sonic Shared\Sonic Central\'
_ = `/usr/bin/cygcheck'
POSIXLY_CORRECT = `1'

HKEY_CURRENT_USER\Software\Cygnus Solutions
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\mounts v2
HKEY_CURRENT_USER\Software\Cygnus Solutions\Cygwin\Program Options
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2
  (default) = `/cygdrive'
  cygdrive flags = 0x0022
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/
  (default) = `C:\cygwin'
  flags = 0x000a
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/bin
  (default) = `C:\cygwin/bin'
  flags = 0x000a
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus Solutions\Cygwin\mounts v2\/usr/lib
  (default) = `C:\cygwin/lib'
  flags = 0x000a
HKEY_LOCAL_MACHINE\SOFTWARE\Cygnus 

Re: Cygwin fork failure

[Larry Hall (Cygwin)]

David Svoboda wrote:

I have built a C++ server program called 'analyzer'. But when I try to
run it, it fails upon trying to execute a 'fork' command. Why?


   $ analyzer --KANTOO_LANGUAGE=toy -server-multiple -fork
   Analyzer 2.0
***  5 [main] analyzer 1712 
fhandler_disk_file::fixup_mmap_after_fork: requested 0x1873 != 0x0 
mem alloc base 0x1873, state 0x1000, size 4096, Win32 error 487
*** d:\svoboda\kantoo\bin\analyzer (1712): *** 
recreate_mmaps_after_fork_failed
*** 9 [main] analyzer 2200 fork_parent: child 1712 died waiting for 
dll loading

*** Failed to fork properly!
]0;~
[EMAIL PROTECTED] ~
$

The *** lines indicate the errors.

Version info: The analyzer was built with G++ 3.3.3, on WinXPSP2. Here
is the output of cygcheck. So why does the fork call fail?


If running 'rebaseall' doesn't help (see the README) and 'analyzer' doesn't
use any DLLs that were built as part of that package (in which case you need
to tell 'rebaseall' about them), then try:

http://sources.redhat.com/ml/cygwin/2005-09/msg00945.html

--
Larry Hall  http://www.rfk.com
RFK Partners, Inc.  (508) 893-9779 - RFK Office
838 Washington Street   (508) 893-9889 - FAX
Holliston, MA 01746

--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Error linking under Cygwin: fork: can't reserve memory for stack XXX, Win32 error 487

[Martin Egholm Nielsen]

Hi,
I've ended up here after having rounded the gcc-irc-channel and the 
crosscompiler mailing-list.

The story:
I have compiled a gcc crosscompiler hosted under Cygwin using Dan 
Kegel's Crosstool scripts.

However, as my application is growing in size (number of .o files) I 
suddenly get the following fault message from collect2.exe when trying 
to link them all together:

$ powerpc-405-linux-gnu-gcj --main=foo.Main *.o
C:\cygwin\opt\crosstool\powerpc-405-linux-gnu\gcc-3.4.0-glibc-2.2.5\libexec\gcc\powerpc-405-linux-gnu\3.4.0\collect2.exe 
(1740): *** fork: can't reserve memory for stack 0x4 - 0x24, Win32 
error 487
REPEATED

There is roughly 600 .o files to link together.
Andrew Haley from RedHat mentioned that
There's some magic in Win32 to extend the size of a stack segment
I can't remember the command
Can anybody on this list help me out here?

  % gcc -v --help | grep stack
-fstack-limit-register=register  Trap if the stack goes past register
-fstack-limit-symbol=name  Trap if the stack goes past symbol name
-fstack-check   Insert stack checking code into the program
-fomit-frame-pointerWhen possible do not generate stack frames
-fdefer-pop Defer popping functions args from stack until later
-fstack-checkingEnable stack checking (same as `{$S+}')
-fno-stack-checking Disable stack checking (same as `{$S-} (default)')
-mstack-arg-probe Enable stack probing
-mpreferred-stack-boundar Attempt to keep stack aligned to this power of 2
--stack size Set size of the initial stack

Hum, these options are not available in my crosscompiler:
$ powerpc-405-linux-gnu-gcc -v --help 21 | grep stack
  -fdefer-pop Defer popping functions args from stack until
  -fomit-frame-pointerWhen possible do not generate stack frames
  -fstack-check   Insert stack checking code into the program
  -fstack-limit   This switch lacks documentation
  -fstack-limit-register=register Trap if the stack goes past register
  -fstack-limit-symbol=name Trap if the stack goes past symbol name
  --execstack require executable stack for this object
  --noexecstack   don't require executable stack for this object
  -z execstack  Mark executable as requiring executable stack
  -z noexecstackMark executable as not requiring executable stack
  -z execstack  Mark executable as requiring executable stack
  -z noexecstackMark executable as not requiring executable stack
  -z execstack  Mark executable as requiring executable stack
  -z noexecstackMark executable as not requiring executable stack
However, this may actually also be a sign that you need to run rebaseall.
I've tried running rebaseall without any errors, but also without any 
changes in the above behavior...

// Martin
--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

RE: Error linking under Cygwin: fork: can't reserve memory for stack XXX, Win32 error 487

[Dave Korn]

Original Message
From: Martin Egholm Nielsen
Sent: 03 March 2005 08:42


 However, as my application is growing in size (number of .o files) I
 suddenly get the following fault message from collect2.exe when trying
 to link them all together:
 
 $ powerpc-405-linux-gnu-gcj --main=foo.Main *.o

C:\cygwin\opt\crosstool\powerpc-405-linux-gnu\gcc-3.4.0-glibc-2.2.5\libexec\
gcc\powerpc-405-linux-gnu\3.4.0\collect2.exe
 (1740): *** fork: can't reserve memory for stack 0x4 - 0x24,
 Win32 error 487 REPEATED

 --stack size Set size of the initial stack
 
 Hum, these options are not available in my crosscompiler:
 
 $ powerpc-405-linux-gnu-gcc -v --help 21 | grep stack


  You would need to add it to the CFLAGS when _building_ the crossbinutils
linker rather than to the command line when _using_ it, so that the compiled
crossversion of collect2 runs with a bigger stack.

  If indeed that is the underlying problem.


cheers,
  DaveK
-- 
Can't think of a witty .sigline today


--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/

Re: Error linking under Cygwin: fork: can't reserve memory for stack XXX, Win32 error 487

[Martin Egholm Nielsen]

Hi,
However, as my application is growing in size (number of .o files) I
suddenly get the following fault message from collect2.exe when trying
to link them all together:
$ powerpc-405-linux-gnu-gcj --main=foo.Main *.o
C:\cygwin\opt\crosstool\powerpc-405-linux-gnu\gcc-3.4.0-glibc-2.2.5\libexec\gcc\powerpc-405-linux-gnu\3.4.0\collect2.exe
(1740): *** fork: can't reserve memory for stack 0x4 - 0x24,
Win32 error 487 REPEATED

   --stack size Set size of the initial stack
   
Hum, these options are not available in my crosscompiler:
$ powerpc-405-linux-gnu-gcc -v --help 21 | grep stack

You would need to add it to the CFLAGS when _building_ the crossbinutils
linker rather than to the command line when _using_ it, so that the compiled
crossversion of collect2 runs with a bigger stack.
  If indeed that is the underlying problem.
I suddenly ran into another problem today (after adding some more .o 
files) - bash reported that the argument line was too long. That is, the 
expansion of *.o was too large. Hence, I created a shell script that 
renamed all the .o files into incremental filenames: 1.o, 2.o, etc.
Then suddenly both problems vanished...
Weird?
Maybe the first problem was related to the fact that I almost ran into 
the *.o expansion-boundary?

BR,
 Martin
--
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple
Problem reports:   http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ:   http://cygwin.com/faq/