Re: IFS not fixing carriage returns
Eric Blake eblake at redhat.com writes: On 03/25/2011 10:35 AM, Tim McDaniel wrote: Although it's not documented in man bash on the latest Cygwin, I did find set -o igncr and it seems to work well. I documented it in the cygwin bash release notes, so it is in /usr/share/doc/Cygwin/bash.README. Patches welcome if you want to see it somewhere else like the man page, because that takes more effort. But I'm just curious about why my first attempt didn't work. IFS only affects word splitting _after words have been parsed and expansions performed_. Bash does _not_ ignore CR in the input stream while parsing words, regardless of the IFS settings. Let's use a simpler example: If you set IFS=., then echo a.b will still only echo the one word a.b and not split into two words a and b, because '.' is not special in determining word boundaries (a.b is a single word), and there was no expansion going on. And it would be prohibitively expensive to make shells honor random IFS while parsing out words, so bash _only_ uses space, tab, and newline to determine word boundaries, not carriage return. That's why igncr is more powerful than IFS - it strips the CR prior to the point that bash is even trying to parse the line into words. Can I urge this option be added into the upstream bash, so it's available under Linux versions too? I want to run scripts that work with some text files that have come from Windows and it would be extremely useful. If I use it now however, in bash version 4.0.33(1)-release (i486-pc-linux-gnu) I get: + set -o igncr set: 1: Illegal option -o igncr So the script is not portable between Windows and Linux. luke -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
IFS not fixing carriage returns
I have tried Googling for info on this, but there are a lot of false hits ... I want to run bash scripts on a Cygwin-running system. The problem is that (so far as I know) I cannot control the format of the scripts -- Rational Build Forge writes each line of the script with a trailing carriage return and then invokes the script with the environment that it wants to put out there. Although it's not documented in man bash on the latest Cygwin, I did find set -o igncr and it seems to work well. But I'm just curious about why my first attempt didn't work. This is a test script with CR-LF lines that I ran on my own box. #! /bin/bash -- set -x # echo $IFS | od -t x1 -a # IFS=$IFS$'\r' # echo $IFS | od -t x1 -a # # set -o igncr 2/dev/null || true # echo Hello, world! pwd ls -l | head exit 0 The trailing #s are to prevent problems with carriage returns before I can do something about them. But the output is $ ./128.bash + echo ' ' + od -t x1 -a 000 20 09 0a 0a sp ht nl nl 004 + IFS=' ' + echo ' ' + od -t x1 -a 000 20 09 0a 0d 0a sp ht nl cr nl 005 + $'\r' ./128.bash: line 8: $'\r': command not found ' echo 'Hello, world! Hello, world! + $'pwd\r' ./128.bash: line 10: $'pwd\r': command not found + ls -l + $'head\r' ./128.bash: line 11: $'head\r': command not found + exit $'0\r' : numeric argument required0 The od calls are intended to show that the carriage returns are getting in there, but the following error messages are showing that IFS apparently has no effect, despite the man page saying IFS The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``spacetabnewline''. Anyone have any ideas on why IFS doesn't work to deal with carriage returns? (Note that adding carriage return to IFS is still an inferior notion to set -o igncr. IFSing would effectively put a space at the end of each line, so I expect that line arg1 arg2 arg3 \ arg4 arg5 would not work. I would also exspect here-documents like some pipe EOF line1 line2 EOF would need their carriage returns stripped. In contrast, set -o igncr strips carriage returns even in these contexts, so they work as-is. I'm asking about IFS merely because I'm curious.) -- Tim McDaniel, t...@panix.com -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: IFS not fixing carriage returns
On 3/25/2011 11:35 AM, Tim McDaniel wrote: I have tried Googling for info on this, but there are a lot of false hits ... [...] Although it's not documented in man bash on the latest Cygwin, I did find set -o igncr and it seems to work well. But I'm just curious about why my first attempt didn't work. [...] Anyone have any ideas on why IFS doesn't work to deal with carriage returns? (Note that adding carriage return to IFS is still an inferior notion to set -o igncr. IFSing would effectively put a space at the end of each line, so I expect that line arg1 arg2 arg3 \ arg4 arg5 would not work. I would also exspect here-documents like some pipe EOF line1 line2 EOF would need their carriage returns stripped. In contrast, set -o igncr strips carriage returns even in these contexts, so they work as-is. I'm asking about IFS merely because I'm curious.) bug-b...@gnu.org might be a better place to ask questions like this. It is essentially a question on how the bash internally works. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: IFS not fixing carriage returns
On 03/25/2011 10:35 AM, Tim McDaniel wrote: Although it's not documented in man bash on the latest Cygwin, I did find set -o igncr and it seems to work well. I documented it in the cygwin bash release notes, so it is in /usr/share/doc/Cygwin/bash.README. Patches welcome if you want to see it somewhere else like the man page, because that takes more effort. But I'm just curious about why my first attempt didn't work. IFS only affects word splitting _after words have been parsed and expansions performed_. Bash does _not_ ignore CR in the input stream while parsing words, regardless of the IFS settings. Let's use a simpler example: If you set IFS=., then echo a.b will still only echo the one word a.b and not split into two words a and b, because '.' is not special in determining word boundaries (a.b is a single word), and there was no expansion going on. And it would be prohibitively expensive to make shells honor random IFS while parsing out words, so bash _only_ uses space, tab, and newline to determine word boundaries, not carriage return. That's why igncr is more powerful than IFS - it strips the CR prior to the point that bash is even trying to parse the line into words. -- Eric Blake ebl...@redhat.com+1-801-349-2682 Libvirt virtualization library http://libvirt.org signature.asc Description: OpenPGP digital signature
