Re: why must non-standard $IFS members be treated so differently ?
On Sun, Jul 29, 2012 at 10:23 AM, Jason Vas Dias
wrote:
>
> function count_colons { IFS=':' ;v=($@); echo ${#v[@]}; }
>
>
count_colons() { local n=${*//[!:]}; printf '%s\n' "${#n}"; }
Re: why must non-standard $IFS members be treated so differently ?
On Sun, Jul 29, 2012 at 03:04:21PM -0400, Chris F.A. Johnson wrote:
> On Sun, 29 Jul 2012, Jason Vas Dias wrote:
> >function count_args {v=($@); echo ${#v[@]}; }
>
>Always quote $@. Without quotes, it's the same as $*
>
> function count_args {v=( "$@" ); echo ${#v[@]}; }
count_args() { echo $#; }
However, if you actually wanted to stuff the function's positional
parameters into an array for some other reason, Chris is absolutely
correct.
Re: why must non-standard $IFS members be treated so differently ?
Thanks Andreas -
I guess your answer mostly explains my issue - except for one thing:
>> And shouldn't '3\ 4' be a single string in any case, regardless of IFS ?
>
> It is. But if field splitting is applied to it it will be split in two
> words when $IFS contains a space.
This was really the point of my question - why, if escaping is permitted and
an escape is shell syntax, does word-splitting not honor escapes, ESPECIALLY
if the character being escaped is a character in $IFS ?
Would it be much work to make word-splitting honor escapes ?
Why is this issue of escaping not being enabled during word-splitting
not documented anywhere ?
Thanks & Regards,
Jason
On Sun, Jul 29, 2012 at 9:05 PM, Andreas Schwab wrote:
> Jason Vas Dias writes:
>
>> Thanks Dan -
>> The plot thickens - Yes, you're right, I had $IFS mistakenly set to ':' in
>> the
>> shell in which I ran 'count_args' . Without this IFS setting, I get
>> a count of 4:
>> $ env -i PATH=/bin:/usr/bin HOME=${HOME} /bin/bash --norc
>> $ count_args 1 2 3\ 4
>> 4
>> $ IFS=: count_args 1 2 3\ 4
>> 3
>> This to me is strange , as I've asked bash not to use ' ' as a
>> delimiter, when $IFS==: , but it is doing so !
>
> IFS does not change the shell syntax. It only controls field splitting
> as applied to the result of expansions. Compare:
>
> $ bash -c 'IFS=:; echo a:b:c'
> a:b:c
> $ bash -c 'IFS=:; a=a:b:c; echo "$a" $a'
> a:b:c a b c
> $ bash -c 'IFS=:; a=a:b:c; b=$a; echo "$b" $b'
> a:b:c a b c
>
> In the last example the assignment "b=$a" doesn't undergo field
> splitting, so the colons are still preserved.
>
>> And shouldn't '3\ 4' be a single string in any case, regardless of IFS ?
>
> It is. But if field splitting is applied to it it will be split in two
> words when $IFS contains a space.
>
>> If word splitting is not doing any escaping, why not - shouldn't it
>> be doing so?
>
> Escape characters are part of the shell syntax. They are never special
> when they result from expansions, unless they are reinterpreted as shell
> input through eval.
>
> Andreas.
>
> --
> Andreas Schwab, sch...@linux-m68k.org
> GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
> "And now for something completely different."
Re: why must non-standard $IFS members be treated so differently ?
Jason Vas Dias writes:
> Thanks Dan -
> The plot thickens - Yes, you're right, I had $IFS mistakenly set to ':' in
> the
> shell in which I ran 'count_args' . Without this IFS setting, I get
> a count of 4:
> $ env -i PATH=/bin:/usr/bin HOME=${HOME} /bin/bash --norc
> $ count_args 1 2 3\ 4
> 4
> $ IFS=: count_args 1 2 3\ 4
> 3
> This to me is strange , as I've asked bash not to use ' ' as a
> delimiter, when $IFS==: , but it is doing so !
IFS does not change the shell syntax. It only controls field splitting
as applied to the result of expansions. Compare:
$ bash -c 'IFS=:; echo a:b:c'
a:b:c
$ bash -c 'IFS=:; a=a:b:c; echo "$a" $a'
a:b:c a b c
$ bash -c 'IFS=:; a=a:b:c; b=$a; echo "$b" $b'
a:b:c a b c
In the last example the assignment "b=$a" doesn't undergo field
splitting, so the colons are still preserved.
> And shouldn't '3\ 4' be a single string in any case, regardless of IFS ?
It is. But if field splitting is applied to it it will be split in two
words when $IFS contains a space.
> If word splitting is not doing any escaping, why not - shouldn't it
> be doing so?
Escape characters are part of the shell syntax. They are never special
when they result from expansions, unless they are reinterpreted as shell
input through eval.
Andreas.
--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5
"And now for something completely different."
Re: why must non-standard $IFS members be treated so differently ?
Thanks Dan -
The plot thickens - Yes, you're right, I had $IFS mistakenly set to ':' in the
shell in which I ran 'count_args' . Without this IFS setting, I get
a count of 4:
$ env -i PATH=/bin:/usr/bin HOME=${HOME} /bin/bash --norc
$ count_args 1 2 3\ 4
4
$ IFS=: count_args 1 2 3\ 4
3
This to me is strange , as I've asked bash not to use ' ' as a
delimiter, when $IFS==: , but it is doing so !
And shouldn't '3\ 4' be a single string in any case, regardless of IFS ?
If word splitting is not doing any escaping, why not - shouldn't it
be doing so?
Escaping works in filenames, so why not in word-splitting ?
Thanks & Regards,
Jason
On Sun, Jul 29, 2012 at 4:19 PM, Dan Douglas wrote:
> On Sunday, July 29, 2012 03:23:29 PM Jason Vas Dias wrote:
>> echo $(count_args 1 2 3\ 4)
>
> I should also have mentioned that I couldn't reproduce this case. You should
> be getting 4 here in your example, not 3. I have the same Bash version. Are
> you sure you were echoing `${#v[@]} ' and not `${#@}', and also that you did
> not set IFS=: for count_args? If you use exactly the function you sent with
> the default IFS then you should get 4 here.
> --
> Dan Douglas
RE:
>why must non-standard $IFS members be treated so differently ?
>Jason Vas Dias
>3:23 PM (4 hours ago)
>
>Good day Chet, list -
> I'm concerned about the difference in output of these functions with
>the example input
> given on the '$' prefixed line below (with 4.2.29(2)-release
> (x86_64-unknown-linux-gnu)):
>
>function count_args {v=($@); echo ${#v[@]}; }
>
>function count_colons { IFS=':' ;v=($@); echo ${#v[@]}; }
>
> $ echo $(count_args 1 2 3\ 4) $(count_colons 1:2:3\:4)
> 3 4
> It appears to be impossible for an item delimited by 'X' to contain
> an escaped 'X' ('\X') if 'X' is not
> a standard delimiter (' ', '') . Quoting doesn't seem to help either:
>
> $ echo $(count_args 1 2 3\ 4) $(count_colons 1:2:3':4')
> 3 4
>
>To me, this appears to be a bug.
>
>But I bet you're going to tell me it is a feature ?
>Please explain.
>
>Thanks & Regards,
>Jason
>
>BTW, documentation on $IFS does not appear to mention this issue:
>
> Word Splitting
> The shell scans the results of parameter expansion, command
> substitution, and arithmetic expansion that did not occur within
> double quotes for word splitting.
>
> The shell treats each character of IFS as a delimiter, and
> splits the results of the other expansions into words on these
> characters. If IFS is unset, or its value is exactly
> , the default, then sequences of ,
> , and at the beginning and end of the results of the
> previous expansions are ignored, and any sequence of IFS charac-
> ters not at the beginning or end serves to delimit words. If
> IFS has a value other than the default, then sequences of the
>whitespace characters space and tab are ignored at the
>beginning and end of the word, as long as the whitespace character is
>in the value of IFS (an IFS whitespace character). Any
>character in IFS that is not IFS whitespace, along with any adjacent
>IFS whitespace characters, delimits a field. A sequence of IFS
> whitespace characters is also treated as a delimiter. If the
> value of IFS is null, no word splitting occurs.
Re: why must non-standard $IFS members be treated so differently ?
On Sun, 29 Jul 2012, Jason Vas Dias wrote:
Good day Chet, list -
I'm concerned about the difference in output of these functions with
the example input
given on the '$' prefixed line below (with 4.2.29(2)-release
(x86_64-unknown-linux-gnu)):
function count_args {v=($@); echo ${#v[@]}; }
Always quote $@. Without quotes, it's the same as $*
function count_args {v=( "$@" ); echo ${#v[@]}; }
--
Chris F.A. Johnson,
Re: why must non-standard $IFS members be treated so differently ?
On Sunday, July 29, 2012 03:23:29 PM Jason Vas Dias wrote:
> echo $(count_args 1 2 3\ 4)
I should also have mentioned that I couldn't reproduce this case. You should
be getting 4 here in your example, not 3. I have the same Bash version. Are
you sure you were echoing `${#v[@]} ' and not `${#@}', and also that you did
not set IFS=: for count_args? If you use exactly the function you sent with
the default IFS then you should get 4 here.
--
Dan Douglas
signature.asc
Description: This is a digitally signed message part.
Re: why must non-standard $IFS members be treated so differently ?
On Sunday, July 29, 2012 03:23:29 PM Jason Vas Dias wrote:
> Good day Chet, list -
> I'm concerned about the difference in output of these functions with
> the example input
> given on the '$' prefixed line below (with 4.2.29(2)-release
> (x86_64-unknown-linux-gnu)):
>
> function count_args {v=($@); echo ${#v[@]}; }
>
> function count_colons { IFS=':' ;v=($@); echo ${#v[@]}; }
>
> $ echo $(count_args 1 2 3\ 4) $(count_colons 1:2:3\:4)
> 3 4
>
> It appears to be impossible for an item delimited by 'X' to contain
> an escaped 'X' ('\X') if 'X' is not
> a standard delimiter (' ', '') . Quoting doesn't seem to help either:
>
> $ echo $(count_args 1 2 3\ 4) $(count_colons 1:2:3':4')
> 3 4
>
> To me, this appears to be a bug.
>
> But I bet you're going to tell me it is a feature ?
> Please explain.
Bash doesn't re-parse the results of expansions for quotes or escaping. When
you expand something unquoted, the entire result is always subject to
word-splitting.
In the case of `count_args 1 2 3\ 4', you are passing 3 arguments. The
backslash is not the result of expansion , so it gets treated as escaping a
space. Note this is because bash is pass-by-value, this escaping/expansion is
processed prior to calling the function.
In the case of `count_colons 1:2:3\:4', you are passing one argument. The shell
strips away the backslash when the function is called (just as it did in the
first example), so the argument being passed is actually '1:2:3:4'
$ printf '%s\n' 1:2:3\:4
1:2:3:4
If you wanted to pass the backslash, you would have to either quote the
argument, or use \\.
However in either case, you're going to have 4 arguments, because as previously
stated, escape characters resulting from expansions are not treated as escapes.
$ f() { IFS=: local -a 'v=( $@ )'; printf '<%s> ' "${v[@]}"; echo; }; f
1:2:3\:4
<1> <2> <3> <4>
$ f() { IFS=: local -a 'v=( $@ )'; printf '<%s> ' "${v[@]}"; echo; }; f
1:2:3\\:4
<1> <2> <3\> <4>
See also my answer to this recent question: http://superuser.com/a/454564/78905
--
Dan Douglas
signature.asc
Description: This is a digitally signed message part.
why must non-standard $IFS members be treated so differently ?
Good day Chet, list -
I'm concerned about the difference in output of these functions with
the example input
given on the '$' prefixed line below (with 4.2.29(2)-release
(x86_64-unknown-linux-gnu)):
function count_args {v=($@); echo ${#v[@]}; }
function count_colons { IFS=':' ;v=($@); echo ${#v[@]}; }
$ echo $(count_args 1 2 3\ 4) $(count_colons 1:2:3\:4)
3 4
It appears to be impossible for an item delimited by 'X' to contain
an escaped 'X' ('\X') if 'X' is not
a standard delimiter (' ', '') . Quoting doesn't seem to help either:
$ echo $(count_args 1 2 3\ 4) $(count_colons 1:2:3':4')
3 4
To me, this appears to be a bug.
But I bet you're going to tell me it is a feature ?
Please explain.
Thanks & Regards,
Jason
BTW, documentation on $IFS does not appear to mention this issue:
Word Splitting
The shell scans the results of parameter expansion, command
substitution, and arithmetic expansion that did not occur within
double quotes for word splitting.
The shell treats each character of IFS as a delimiter, and
splits the results of the other expansions into words on these
characters. If IFS is unset, or its value is exactly
, the default, then sequences of ,
,
and at the beginning and end of the results of the
previous expansions are ignored, and any sequence of IFS charac-
ters not at the beginning or end serves to delimit words. If
IFS has a value other than the default, then sequences of the
whitespace characters space and tab are ignored at the
beginning and end of the word, as long as the whitespace character is
in the value of IFS (an IFS whitespace character). Any
character in IFS that is not IFS whitespace, along with any adjacent
IFS whitespace characters, delimits a field. A sequence of IFS
whitespace characters is also treated as a delimiter. If the
value of IFS is null, no word splitting occurs.
