Re: Bash login breaks if too many environment variables are set
On Sep 16 21:55, Hommersom, Fred wrote: You can help by running the new snapshot under strace, like you did before. I have done three tests: below the maximum exactly the maximum over the maximum Further empirical tests turned out that the environment in Windows NT is not limited, at least not up to a tested size of about 800K. But there is a big problem in Windows nevertheless. When using ASCII functions to access the environment, as Cygwin does, the environment is silently cut off at 32K. There's no error message or any hint in the documentation that something like this would happen when trying to retrieve the environment. Trying to start another process with an ASCII environment 32K results in an error code returned by CreateProcess. So, for now you have to live with an environment 32K. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
Thanks for the answer. Fred Further empirical tests turned out that the environment in Windows NT is not limited, at least not up to a tested size of about 800K. But there is a big problem in Windows nevertheless. When using ASCII functions to access the environment, as Cygwin does, the environment is silently cut off at 32K. There's no error message or any hint in the documentation that something like this would happen when trying to retrieve the environment. Trying to start another process with an ASCII environment 32K results in an error code returned by CreateProcess. So, for now you have to live with an environment 32K. Corinna -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
Thanks for the replies so far. Unfortunately is the best option (set variables in bash) not feasable. The .bat file is a complex set of bat files with logic inside so that would take a lot of effort to convert But I have done some experiments with bash without --login option and the advised export -p. The number of variables itself does not seem to be the problem. The length of the values is also important. So to get an idea what is going on I have incremented the number of variables up to the point where export -p still gives the right output. Now I add 1 character at a time to a value and tested export - p. Only the built-in commands work now. All others end with Resource temporarily unavailable There comes a point where adding 1 more character ends up with: 138 [main] bash 2556 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 887 [main] bash 2556 open_stackdumpfile: Dumping stack trace to bash.exe.stackdump The stackdump contains Exception: STATUS_ACCESS_VIOLATION at eip=61014DEE eax=00246000 ebx=10012910 ecx=0003 edx=00245FFF esi= edi=6110A1C4 ebp=0022EEE8 esp=0022EE90 program=c:\cygwin\bin\bash.exe, pid 2556, thread main cs=001B ds=0023 es=0023 fs=003B gs= ss=0023 Stack trace: Frame Function Args 0022EEE8 61014DEE (, , 0022EF18, 61087B02) 0022EF18 61068028 (, , 7C90EE18, 7C919AF0) 0022EFD8 61004B16 (0022EFF0, 77D70467, 77D49A18, ) 0022FF88 6100594F (, , , ) End of stack trace Adding another character gives no errors but export -p shows now only: declare -x HOME=/cygdrive/c/cygwin/home/fhommexx declare -x OLDPWD declare -x PWD=/cygdrive/c/Data/locations/tc50_custy00 declare -x SHLVL=1 declare -x TERM=cygwin Adding still more characters reveals more variables. The last export -p that gave correct results was piped to a file. I tried to find if the limit was the 32k limit as suggested by Eric. The rough size of the file is 45,098 bytes. Stripping 'declare -x cuts it down to 37,816 bytes Stripping leading and trailing cuts it down to 36,494 bytes Modifying \\ into \ cuts it down to 33.065 bytes The file contains 662 lines and if we also cut the linefeed (no idea how the administration works) There are only 32403 bytes left. And 32 k = 32.768. Pretty near but no exact match. Now I am stuck. Did I reach a limit of windoze, a limit in cygwin or a supporting library or a bug? -Original Message- Hommersom, Fred wrote: The file bigsetup.bat contains a huge amount environment variables. For a medium number (~ 600) everything works fine For a larger number the output is: bash: /usr/bin/id: Resource temporarily unavailable ... i would think that you should declare your env variables in either your .bashrc or your .bash_profile rather than in a .bat file. It may solve your problem. I haven't had a chance to look at this further, but it is on my list. I tried looking in the Windows documentation to see if there is a limit on environment size for CreateProcess that you might be exceeding - all I can find is that an individual variable can be no more than 32k, but nothing about the overall environment size. Does anyone else know what limits windows imposes on the environment? POSIX only specifies ARG_MAX, which is the combination of command line and environment together. You might also want to experiment with 'export -p' in a bash where you are experiencing failures, to see if bash's list of environment variables is shorter than what you thought should have been inherited into bash. Being a builtin, it won't have then invocation problems like you are having with id, find, or sort. I also agree with Reid's suggestion - try sticking things into the environment AFTER bash is started, not beforehand. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Sep 16 11:08, Hommersom, Fred wrote: Thanks for the replies so far. Unfortunately is the best option (set variables in bash) not feasable. The .bat file is a complex set of bat files with logic inside so that would take a lot of effort to convert But I have done some experiments with bash without --login option and the advised export -p. The number of variables itself does not seem to be the problem. The length of the values is also important. So to get an idea what is going on I have incremented the number of variables up to the point where export -p still gives the right output. Now I add 1 character at a time to a value and tested export - p. Only the built-in commands work now. All others end with Resource temporarily unavailable There comes a point where adding 1 more character ends up with: 138 [main] bash 2556 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 887 [main] bash 2556 open_stackdumpfile: Dumping stack trace to bash.exe.stackdump The stackdump contains Exception: STATUS_ACCESS_VIOLATION at eip=61014DEE eax=00246000 ebx=10012910 ecx=0003 edx=00245FFF esi= edi=6110A1C4 ebp=0022EEE8 esp=0022EE90 program=c:\cygwin\bin\bash.exe, pid 2556, thread main cs=001B ds=0023 es=0023 fs=003B gs= ss=0023 Stack trace: Frame Function Args 0022EEE8 61014DEE (, , 0022EF18, 61087B02) 0022EF18 61068028 (, , 7C90EE18, 7C919AF0) 0022EFD8 61004B16 (0022EFF0, 77D70467, 77D49A18, ) 0022FF88 6100594F (, , , ) End of stack trace The stackdump isn't very useful unless there's debug information available, which isn't for 1.5.18. Would you mind to try the same with the latest snapshot DLL, http://cygwin.com/snapshots/cygwin1-20050916.dll.bz2 and report back if either the problem is solved or if not, send the resulting strace? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
The problem can be reproduced with cygwin1-20050916.dll The output of strace in the case of a stackdump is below. ** Program name: c:\cygwin\bin\bash.exe (pid 2692, ppid 1) App version: 1005.18, api: 0.132 DLL version: 1005.19, api: 0.138 DLL build:20050916 00:00:39SNP OS version: Windows NT-5.1 Heap size:1073741824 Date/Time:2005-09-16 12:38:36 ** 26 275 [main] bash 2692 set_myself: myself-dwProcessId 2692 24 299 [main] bash 2692 time: 1126867116 = time (0) 440 739 [main] bash 2692 environ_init: 0x10010238: X$ 37 776 [main] bash 2692 environ_init: 0x10010248: ðH$ 102 878 [main] bash 2692 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x610D6971 sp 0x22EE64 25 903 [main] bash 2692 handle_exceptions: In cygwin_except_handler sig 11 at 0x610D6971 21 924 [main] bash 2692 handle_exceptions: In cygwin_except_handler calling 0x0 20 944 [main] bash 2692 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 3021246 [main] bash 2692 try_to_debug: debugger_command '' 6821928 [main] bash 2692 open_stackdumpfile: Dumping stack trace to bash.exe.stackdump 1760359 1762287 [main] bash 2692 signal_exit: about to call do_exit (8B) 66 1762353 [main] bash 2692 do_exit: do_exit (139), exit_state 0 57 1762410 [main] bash 2692 void: 0x0 = signal (20, 0x1) 48 1762458 [main] bash 2692 void: 0x0 = signal (1, 0x1) 45 1762503 [main] bash 2692 void: 0x0 = signal (2, 0x1) 45 1762548 [main] bash 2692 void: 0x0 = signal (3, 0x1) 66 1762614 [main] bash 2692 sigproc_terminate: entering 49 1762663 [main] bash 2692 sig_send: my_sendsig 0x0, myself-sendsig 0x0, exit_state 9 47 1762710 [main] bash 2692 __set_errno: int sig_send(_pinfo*, siginfo_t, _cygtls*):548 val 11 48 1762758 [main] bash 2692 sig_send: returning 0x1 from sending signal -42 46 1762804 [main] bash 2692 proc_terminate: nprocs 0 44 1762848 [main] bash 2692 proc_terminate: leaving 84 1762932 [main] bash 2692 sigproc_terminate: already performed 49 1762981 [main] bash 2692 __to_clock_t: dwHighDateTime 0, dwLowDateTime 100144 47 1763028 [main] bash 2692 __to_clock_t: total 000A 46 1763074 [main] bash 2692 __to_clock_t: dwHighDateTime 0, dwLowDateTime 200288 47 1763121 [main] bash 2692 __to_clock_t: total 0014 933 1764054 [main] bash 2692 pinfo::maybe_set_exit_code_from_windows: pid 2692, exit value - old 0x88B, windows 0xDEADBEEF, cygwin 0x88B 145 1764199 [main] bash 2692 pinfo::exit: Calling ExitThread hProcess 0x0, n 0x8B, exitcode 0x0 -Original Message- On Sep 16 11:08, Hommersom, Fred wrote: Thanks for the replies so far. Unfortunately is the best option (set variables in bash) not feasable. The .bat file is a complex set of bat files with logic inside so that would take a lot of effort to convert But I have done some experiments with bash without --login option and the advised export -p. The number of variables itself does not seem to be the problem. The length of the values is also important. So to get an idea what is going on I have incremented the number of variables up to the point where export -p still gives the right output. Now I add 1 character at a time to a value and tested export - p. Only the built-in commands work now. All others end with Resource temporarily unavailable There comes a point where adding 1 more character ends up with: 138 [main] bash 2556 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 887 [main] bash 2556 open_stackdumpfile: Dumping stack trace to bash.exe.stackdump The stackdump contains Exception: STATUS_ACCESS_VIOLATION at eip=61014DEE eax=00246000 ebx=10012910 ecx=0003 edx=00245FFF esi= edi=6110A1C4 ebp=0022EEE8 esp=0022EE90 program=c:\cygwin\bin\bash.exe, pid 2556, thread main cs=001B ds=0023 es=0023 fs=003B gs= ss=0023 Stack trace: Frame Function Args 0022EEE8 61014DEE (, , 0022EF18, 61087B02) 0022EF18 61068028 (, , 7C90EE18, 7C919AF0) 0022EFD8 61004B16 (0022EFF0, 77D70467, 77D49A18, ) 0022FF88 6100594F (, , , ) End of stack trace The stackdump isn't very useful unless there's debug information available, which isn't for 1.5.18. Would you mind to try the same with the latest snapshot DLL, http://cygwin.com/snapshots/cygwin1-20050916.dll.bz2 and report back if either the problem is solved or if not, send the resulting strace? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Sep 16 12:51, Hommersom, Fred wrote: The problem can be reproduced with cygwin1-20050916.dll The output of strace in the case of a stackdump is below. ** Program name: c:\cygwin\bin\bash.exe (pid 2692, ppid 1) App version: 1005.18, api: 0.132 DLL version: 1005.19, api: 0.138 DLL build:20050916 00:00:39SNP OS version: Windows NT-5.1 Heap size:1073741824 Date/Time:2005-09-16 12:38:36 ** 26 275 [main] bash 2692 set_myself: myself-dwProcessId 2692 24 299 [main] bash 2692 time: 1126867116 = time (0) 440 739 [main] bash 2692 environ_init: 0x10010238: X$ 37 776 [main] bash 2692 environ_init: 0x10010248: ðH$ 102 878 [main] bash 2692 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x610D6971 sp 0x22EE64 25 903 [main] bash 2692 handle_exceptions: In cygwin_except_handler sig 11 at 0x610D6971 21 924 [main] bash 2692 handle_exceptions: In cygwin_except_handler calling 0x0 20 944 [main] bash 2692 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 3021246 [main] bash 2692 try_to_debug: debugger_command '' 6821928 [main] bash 2692 open_stackdumpfile: Dumping stack trace to bash.exe.stackdump 1760359 1762287 [main] bash 2692 signal_exit: about to call do_exit (8B) 66 1762353 [main] bash 2692 do_exit: do_exit (139), exit_state 0 57 1762410 [main] bash 2692 void: 0x0 = signal (20, 0x1) 48 1762458 [main] bash 2692 void: 0x0 = signal (1, 0x1) 45 1762503 [main] bash 2692 void: 0x0 = signal (2, 0x1) 45 1762548 [main] bash 2692 void: 0x0 = signal (3, 0x1) 66 1762614 [main] bash 2692 sigproc_terminate: entering 49 1762663 [main] bash 2692 sig_send: my_sendsig 0x0, myself-sendsig 0x0, exit_state 9 47 1762710 [main] bash 2692 __set_errno: int sig_send(_pinfo*, siginfo_t, _cygtls*):548 val 11 48 1762758 [main] bash 2692 sig_send: returning 0x1 from sending signal -42 46 1762804 [main] bash 2692 proc_terminate: nprocs 0 44 1762848 [main] bash 2692 proc_terminate: leaving 84 1762932 [main] bash 2692 sigproc_terminate: already performed 49 1762981 [main] bash 2692 __to_clock_t: dwHighDateTime 0, dwLowDateTime 100144 47 1763028 [main] bash 2692 __to_clock_t: total 000A 46 1763074 [main] bash 2692 __to_clock_t: dwHighDateTime 0, dwLowDateTime 200288 47 1763121 [main] bash 2692 __to_clock_t: total 0014 933 1764054 [main] bash 2692 pinfo::maybe_set_exit_code_from_windows: pid 2692, exit value - old 0x88B, windows 0xDEADBEEF, cygwin 0x88B 145 1764199 [main] bash 2692 pinfo::exit: Calling ExitThread hProcess 0x0, n 0x8B, exitcode 0x0 Sorry, I forgot to mention, can you please add the stackdump, too? Thanks, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
No problem. here is a new trace (similar to the original) and the related stackdump Exception: STATUS_ACCESS_VIOLATION at eip=610D6971 eax= ebx=10010248 ecx=F2FF edx=00245300 esi=0001 edi=00246000 ebp=0022EE68 esp=0022EE64 program=c:\cygwin\bin\bash.exe, pid 3572, thread main cs=001B ds=0023 es=0023 fs=003B gs= ss=0023 Stack trace: Frame Function Args 0022EE68 610D6971 (00245300, 0040, 6110E714, 6110E77B) 0022EEE8 6105390B (, , 0022EF18, 61090062) 0022EF18 6106EFC8 (, , 7C90E64E, 77DDDB0D) 0022EFC8 61004D1D (0022EFE0, , 0022EFC0, 77D4A303) 0022FF88 61005B6F (, , , ) End of stack trace ** Program name: c:\cygwin\bin\bash.exe (pid 3504, ppid 1) App version: 1005.18, api: 0.132 DLL version: 1005.19, api: 0.138 DLL build:20050916 00:00:39SNP OS version: Windows NT-5.1 Heap size:1073741824 Date/Time:2005-09-16 13:29:23 ** 27 319 [main] bash 3504 set_myself: myself-dwProcessId 3504 24 343 [main] bash 3504 time: 1126870163 = time (0) 442 785 [main] bash 3504 environ_init: 0x10010238: X$ 37 822 [main] bash 3504 environ_init: 0x10010248: ðH$ 97 919 [main] bash 3504 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x610D6971 sp 0x22EE64 24 943 [main] bash 3504 handle_exceptions: In cygwin_except_handler sig 11 at 0x610D6971 771020 [main] bash 3504 handle_exceptions: In cygwin_except_handler calling 0x0 231043 [main] bash 3504 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION 3531396 [main] bash 3504 try_to_debug: debugger_command '' 5191915 [main] bash 3504 open_stackdumpfile: Dumping stack trace to bash.exe.stackdump 861081 862996 [main] bash 3504 signal_exit: about to call do_exit (8B) 81 863077 [main] bash 3504 do_exit: do_exit (139), exit_state 0 67 863144 [main] bash 3504 void: 0x0 = signal (20, 0x1) 59 863203 [main] bash 3504 void: 0x0 = signal (1, 0x1) 55 863258 [main] bash 3504 void: 0x0 = signal (2, 0x1) 55 863313 [main] bash 3504 void: 0x0 = signal (3, 0x1) 79 863392 [main] bash 3504 sigproc_terminate: entering 59 863451 [main] bash 3504 sig_send: my_sendsig 0x0, myself-sendsig 0x0, exit_state 9 58 863509 [main] bash 3504 __set_errno: int sig_send(_pinfo*, siginfo_t, _cygtls*):548 val 11 59 863568 [main] bash 3504 sig_send: returning 0x1 from sending signal -42 57 863625 [main] bash 3504 proc_terminate: nprocs 0 56 863681 [main] bash 3504 proc_terminate: leaving 95 863776 [main] bash 3504 sigproc_terminate: already performed 59 863835 [main] bash 3504 __to_clock_t: dwHighDateTime 0, dwLowDateTime 0 57 863892 [main] bash 3504 __to_clock_t: total 58 863950 [main] bash 3504 __to_clock_t: dwHighDateTime 0, dwLowDateTime 200288 57 864007 [main] bash 3504 __to_clock_t: total 0014 912 864919 [main] bash 3504 pinfo::maybe_set_exit_code_from_windows: pid 3504, exit value - old 0x88B, windows 0xDEADBEEF, cygwin 0x88B 156 865075 [main] bash 3504 pinfo::exit: Calling ExitThread hProcess 0x0, n 0x8B, exitcode 0x0 -Original Message- The problem can be reproduced with cygwin1-20050916.dll Sorry, I forgot to mention, can you please add the stackdump, too? -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Sep 16 13:31, Hommersom, Fred wrote: No problem. here is a new trace (similar to the original) and the related stackdump Exception: STATUS_ACCESS_VIOLATION at eip=610D6971 eax= ebx=10010248 ecx=F2FF edx=00245300 esi=0001 edi=00246000 ebp=0022EE68 esp=0022EE64 program=c:\cygwin\bin\bash.exe, pid 3572, thread main cs=001B ds=0023 es=0023 fs=003B gs= ss=0023 Stack trace: Frame Function Args 0022EE68 610D6971 (00245300, 0040, 6110E714, 6110E77B) 0022EEE8 6105390B (, , 0022EF18, 61090062) 0022EF18 6106EFC8 (, , 7C90E64E, 77DDDB0D) 0022EFC8 61004D1D (0022EFE0, , 0022EFC0, 77D4A303) 0022FF88 61005B6F (, , , ) End of stack trace Sure you've used the latest snapshot DLL? I tried to reproduce this with the latest snapshot, as well as a self-build DLL from CVS and I can't reproduce this behaviour. My test environment consisted of 1400 variables with a size of 98K, one of the variables taking roughly 31K alone. However, what's strange in your strace output is this: 442 785 [main] bash 3504 environ_init: 0x10010238: X$ 37 822 [main] bash 3504 environ_init: 0x10010248: ðH$ 97 919 [main] bash 3504 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x610D6971 sp 0x22EE64 The first two lines should contain some valid environment entries, but both of them look broken. There's no hint why this is, of course. I observerd that tcsh doesn't like variables with a length of 31K, though. ash, bash, zsh and pdksh could handle that long environment varibale just fine, tcsh on the other hand printed this: $ echo $VERY_LONG_ENV_VAR Word too long. Sigh, Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Sep 16 14:28, Corinna Vinschen wrote: I observerd that tcsh doesn't like variables with a length of 31K, though. ash, bash, zsh and pdksh could handle that long environment varibale just fine, tcsh on the other hand printed this: $ echo $VERY_LONG_ENV_VAR Word too long. Never mind, that's a normal limitation of tcsh. THe same happens on Linux. The actual limit seems to be somewhat below BUFSIZ. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Project Co-Leader cygwin AT cygwin DOT com Red Hat, Inc. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
Yes sure. You can see this in the header of the dump its says: DLL version: 1005.19, api: 0.138 DLL build:20050916 00:00:39SNP In order to be sure that we are talking about the same things: I have all these variables in DOS and start bash from a CMD window with command c:\cygwin\bin\strace -o fhbashtrace.txt c:\cygwin\bin\bash As indicated in previous mails the stackdump occurs only with an exact number of characters for all environment names and values together. My names and values have sizes of 1-1000 characters. No extremes. But this dll behaves differently from the production. Now if I have one character less then the 'dumping' number some commands (e.g. ls) works, others don't. I did not find a pattern in it. I'll keep trying. But if I add 10 characters to an enviroment variable ( so 10 more then the number that gives the dump) then bash exits immediately without dump. Strace gives: 27 324 [main] bash 3024 set_myself: myself-dwProcessId 3024 23 347 [main] bash 3024 time: 1126876171 = time (0) 422 769 [main] bash 3024 environ_init: 0x10010238: !C:=C:\Dat No more output here, no stackdump nothing. just exit It looks to me as if a buffer or stack is reused if some maximum is exceeded with effect that the system sometimes works. Fred -Original Message- Sure you've used the latest snapshot DLL? I tried to reproduce this with the latest snapshot, as well as a self-build DLL from CVS and I can't reproduce this behaviour. My test environment consisted of 1400 variables with a size of 98K, one of the variables taking roughly 31K alone. However, what's strange in your strace output is this: 442 785 [main] bash 3504 environ_init: 0x10010238: X$ 37 822 [main] bash 3504 environ_init: 0x10010248: ðH$ 97 919 [main] bash 3504 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x610D6971 sp 0x22EE64 The first two lines should contain some valid environment entries, but both of them look broken. There's no hint why this is, of course. I observerd that tcsh doesn't like variables with a length of 31K, though. ash, bash, zsh and pdksh could handle that long environment varibale just fine, tcsh on the other hand printed this: $ echo $VERY_LONG_ENV_VAR Word too long. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Fri, Sep 16, 2005 at 03:55:14PM +0200, Hommersom, Fred wrote: Yes sure. You can see this in the header of the dump its says: DLL version: 1005.19, api: 0.138 DLL build:20050916 00:00:39SNP In order to be sure that we are talking about the same things: I have all these variables in DOS and start bash from a CMD window with command c:\cygwin\bin\strace -o fhbashtrace.txt c:\cygwin\bin\bash As indicated in previous mails the stackdump occurs only with an exact number of characters for all environment names and values together. My names and values have sizes of 1-1000 characters. No extremes. But this dll behaves differently from the production. Now if I have one character less then the 'dumping' number some commands (e.g. ls) works, others don't. I did not find a pattern in it. I'll keep trying. But if I add 10 characters to an enviroment variable ( so 10 more then the number that gives the dump) then bash exits immediately without dump. Strace gives: 27 324 [main] bash 3024 set_myself: myself-dwProcessId 3024 23 347 [main] bash 3024 time: 1126876171 = time (0) 422 769 [main] bash 3024 environ_init: 0x10010238: !C:=C:\Dat No more output here, no stackdump nothing. just exit It looks to me as if a buffer or stack is reused if some maximum is exceeded with effect that the system sometimes works. From your strace output, it looks to me like windows itself is returning garbage when we ask it for the list of environment variables. If that is the case, we can guard against that but we can't make the passed in environment useful, unfortunately. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
From your strace output, it looks to me like windows itself is returning garbage when we ask it for the list of environment variables. If that is the case, we can guard against that but we can't make the passed in environment useful, unfortunately. Is it possible that 'asking for the environment' makes an error in some mapping activity? Anyway, a clear error is better as a garbled environment. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
It looks to me as if a buffer or stack is reused if some maximum is exceeded with effect that the system sometimes works. From your strace output, it looks to me like windows itself is returning garbage when we ask it for the list of environment variables. I don't think all places in Windows have the limitation. Corinna reported (and I have reproduced on Win2k, CYGWIN-NT-5.0) that it is quite easy to create an environment larger than 32k and see it in a child process: $ foo=`perl -e 'print ax31000'` $ export foo $ /bin/env | wc -c 34664 But it certainly does look like at least one version of Windows (the OP was using CYGWIN_NT-5.1), when manipulating the environment during .bat execution, is tracking total environment size in a signed short, then croaking as the variable wraps around. The output of 'export -p' just before the breaking point will not be exactly 32k, since Cygwin and bash both add variables to the environment before export has a chance to print it. Meanwhile, it also exposes a bug in xargs, using the above environment: $ xargs --help xargs: environment is too large for exec Oops - xargs was over-eager in not exceeding ARG_MAX, even though --help implied there was nothing to exec, and even though on cygwin the environ and command line do not share common storage in exec*(). -- Eric Blake -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Fri, Sep 16, 2005 at 04:36:00PM +, Eric Blake wrote: It looks to me as if a buffer or stack is reused if some maximum is exceeded with effect that the system sometimes works. From your strace output, it looks to me like windows itself is returning garbage when we ask it for the list of environment variables. I don't think all places in Windows have the limitation. Look at the code. We're inspecting a buffer returned from GetEnvironmentStrings. That is a windows function. The very first things returned from this are garbage. Corinna reported (and I have reproduced on Win2k, CYGWIN-NT-5.0) that it is quite easy to create an environment larger than 32k and see it in a child process: $ foo=`perl -e 'print ax31000'` $ export foo $ /bin/env | wc -c 34664 $ /bin/env | wc -c 34664 You're not testing the same thing. Cygwin deals with environment variables on its own once you've started a cygwin process. It only relies on windows environment-passing mechanisms to pass environment variables to non-cygwin applications. I have a new version of a cygwin snapshot which has more debugging output and which should (temporarily?) fail with an error if environment processing fails. cgf -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
I don't think all places in Windows have the limitation. Look at the code. We're inspecting a buffer returned from GetEnvironmentStrings. That is a windows function. The very first things returned from this are garbage. OK, I stand corrected. $ /bin/env | wc -c 34664 $ cmd bash: /cygdrive/c/WINNT/system32/cmd: Invalid argument $ strace /bin/env bash: /usr/bin/strace: Invalid argument On the other hand, POSIX would claim that this usage should be failing with E2BIG, not EINVAL. So it looks like Windows does have a hard limit at total environment size of 32k (in spite of their documentation never mentioning it), but that cygwin could do better at reporting the error when trying to invoke a native process. -- Eric Blake -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
Perhaps I do not understand it. I was talking about invoking cygin from native. The native environment grows far over 32 k. It just does not show up in bash. If I can help by testing the new snapshot: please supply some hints. Fred On the other hand, POSIX would claim that this usage should be failing with E2BIG, not EINVAL. So it looks like Windows does have a hard limit at total environment size of 32k (in spite of their documentation never mentioning it), but that cygwin could do better at reporting the error when trying to invoke a native process. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
On Fri, Sep 16, 2005 at 08:04:57PM +0200, Hommersom, Fred wrote: Perhaps I do not understand it. I was talking about invoking cygin from native. The native environment grows far over 32 k. It just does not show up in bash. If I can help by testing the new snapshot: please supply some hints. AFAICT, Eric is getting sidetracked into issues that are not related to your problem. You can help by running the new snapshot under strace, like you did before. cgf On the other hand, POSIX would claim that this usage should be failing with E2BIG, not EINVAL. So it looks like Windows does have a hard limit at total environment size of 32k (in spite of their documentation never mentioning it), but that cygwin could do better at reporting the error when trying to invoke a native process. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
RE: Bash login breaks if too many environment variables are set
You can help by running the new snapshot under strace, like you did before. I have done three tests: below the maximum exactly the maximum over the maximum ** Here are the results of the test below the maximum ** Program name: c:\cygwin\bin\bash.exe (pid 3260, ppid 1) App version: 1005.18, api: 0.132 DLL version: 1005.19, api: 0.138 DLL build:20050916 12:02:10SNP OS version: Windows NT-5.1 Heap size:1073741824 Date/Time:2005-09-16 21:41:35 ** 23 240 [main] bash 3260 set_myself: myself-dwProcessId 3260 20 260 [main] bash 3260 time: 1126899695 = time (0) 496 756 [main] bash 3260 environ_init: GetEnvironmentStrings returned 0x2452F8 - =C:=C:\Data\locations\tc50_custy00 36 792 [main] bash 3260 environ_init: 0x10010238: !C:=C:\Data\locations\tc50_custy00 32 824 [main] bash 3260 environ_init: 0x10010260: ACD=C:\Data\TC\TC50\support\acd many more lines ** Here are the results for the exact number where it went wrong. ** *** internal error reading the windows environment - too many environment variables? ** Program name: c:\cygwin\bin\bash.exe (pid 3140, ppid 1) App version: 1005.18, api: 0.132 DLL version: 1005.19, api: 0.138 DLL build:20050916 12:02:10SNP OS version: Windows NT-5.1 Heap size:1073741824 Date/Time:2005-09-16 21:37:00 ** 26 538 [main] bash 3140 set_myself: myself-dwProcessId 3140 24 562 [main] bash 3140 time: 1126899420 = time (0) 14408 14970 [main] bash 3140 environ_init: GetEnvironmentStrings returned 0x2452F8 - x$ 46 15016 [main] bash 3140 environ_init: 0x10010238: X$ 35 15051 [main] bash 3140 environ_init: 0x10010248: ðH$ 76 15127 [main] bash 3140 handle_exceptions: In cygwin_except_handler exc 0xC005 at 0x610D6BD1 sp 0x22ED64 24 15151 [main] bash 3140 handle_exceptions: In cygwin_except_handler sig 11 at 0x610D6BD1 21 15172 [main] bash 3140 handle_exceptions: In cygwin_except_handler calling 0x0 ** Here are the results if the environment size is increased by 10 characters No error message in the window. bash quits immediately. The environmentstring is truncated. ** Program name: c:\cygwin\bin\bash.exe (pid 568, ppid 1) App version: 1005.18, api: 0.132 DLL version: 1005.19, api: 0.138 DLL build:20050916 12:02:10SNP OS version: Windows NT-5.1 Heap size:1073741824 Date/Time:2005-09-16 21:44:55 ** 23 282 [main] bash 568 set_myself: myself-dwProcessId 568 20 302 [main] bash 568 time: 1126899895 = time (0) 408 710 [main] bash 568 environ_init: GetEnvironmentStrings returned 0x2552F8 - =C:=C:\Data\location 35 745 [main] bash 568 environ_init: 0x10010238: !C:=C:\Data\location no more lines in this trace -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 According to Reid Thompson on 9/12/2005 5:15 PM: Hommersom, Fred wrote: The file bigsetup.bat contains a huge amount environment variables. For a medium number (~ 600) everything works fine For a larger number the output is: bash: /usr/bin/id: Resource temporarily unavailable ... i would think that you should declare your env variables in either your .bashrc or your .bash_profile rather than in a .bat file. It may solve your problem. I haven't had a chance to look at this further, but it is on my list. I tried looking in the Windows documentation to see if there is a limit on environment size for CreateProcess that you might be exceeding - all I can find is that an individual variable can be no more than 32k, but nothing about the overall environment size. Does anyone else know what limits windows imposes on the environment? POSIX only specifies ARG_MAX, which is the combination of command line and environment together. You might also want to experiment with 'export -p' in a bash where you are experiencing failures, to see if bash's list of environment variables is shorter than what you thought should have been inherited into bash. Being a builtin, it won't have then invocation problems like you are having with id, find, or sort. I also agree with Reid's suggestion - try sticking things into the environment AFTER bash is started, not beforehand. - -- Life is short - so eat dessert first! Eric Blake [EMAIL PROTECTED] volunteer cygwin bash maintainer -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Cygwin) Comment: Public key at home.comcast.net/~ericblake/eblake.gpg Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDKCRA84KuGfSFAYARAtiUAJ9/7QwrUG1TFnoqaMX2VRJo/K8ATQCg1G+/ vkJkRCSGt6r1iKUGhp2tmvU= =tPWq -END PGP SIGNATURE- -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Bash login breaks if too many environment variables are set
My startup of cygwin is a .bat file with contents call bigsetup.bat set HOME=C:\Data\locations\tc50_custy00 c:\cygwin\bin\bash --login -i The file bigsetup.bat contains a huge amount environment variables. For a medium number (~ 600) everything works fine For a larger number the output is: bash: /usr/bin/id: Resource temporarily unavailable bash: /bin/find: Resource temporarily unavailable bash: /usr/bin/sort: Resource temporarily unavailable bash: /usr/bin/tr: Resource temporarily unavailable bash: /usr/bin/uname: Resource temporarily unavailable bash: /usr/bin/id: Resource temporarily unavailable and for more variables no error messages show up but the HOME directory is not correctly set. What is the underlying cause of this behavior? Is it possible to tune bash or cygwin to cope with this large number of variables? Thanks Fred -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
Re: Bash login breaks if too many environment variables are set
Hommersom, Fred wrote: My startup of cygwin is a .bat file with contents call bigsetup.bat set HOME=C:\Data\locations\tc50_custy00 c:\cygwin\bin\bash --login -i The file bigsetup.bat contains a huge amount environment variables. For a medium number (~ 600) everything works fine For a larger number the output is: bash: /usr/bin/id: Resource temporarily unavailable bash: /bin/find: Resource temporarily unavailable bash: /usr/bin/sort: Resource temporarily unavailable bash: /usr/bin/tr: Resource temporarily unavailable bash: /usr/bin/uname: Resource temporarily unavailable bash: /usr/bin/id: Resource temporarily unavailable and for more variables no error messages show up but the HOME directory is not correctly set. What is the underlying cause of this behavior? Is it possible to tune bash or cygwin to cope with this large number of variables? Thanks Fred -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/ i would think that you should declare your env variables in either your .bashrc or your .bash_profile rather than in a .bat file. It may solve your problem. -- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/
