Re: Indices of array variables are sometimes considered unset (or just display an error).
On 11/5/18 8:44 PM, Great Big Dot wrote:
> What's actually happening here is that the *indirection* expansion
> "${!foo}", and not the *indices* expansion "${!foo[@]}", is what is being
> preformed on something like "${!array[@]-}". Both expansions, while
> unrelated, happen to use the same syntax, with the exception that
> indirections apply to normal variables and index expansions apply to array
> variables. For some reason, adding on the "${foo-default}" expansion causes
> the former to be used instead of the latter.
The `some reason' is that the current behavior is documented and has been
the way bash has worked since I added indirect expansion (bash-2.0) and the
array keys expansion (bash-3.0).
> This can be seen here:
I'm going to use bash-4.4 for my explanation.
>
> $ array=(foo)
> $ printf -- '%s\n' "${!foo[@]-unset}"
> unset
Of course. There is no variable "foo"; the indirect expansion results in a
null string.
> $ foo='hello world'
> $ printf -- '%s\n' "${!foo[@]-unset}"
> hello world
(I don't get that result with any version of bash. I went back to bash-3.0
before I quit trying.)
This is a case where bash is trying to be helpful, maybe more so than is
desired. `foo' isn't an array variable, but the !foo[@] is first identified
as a candidate for indirect expansion, then checked to see whether or not
it is the entire expansion between ${ and }. Since it's not, it's not a
candidate for array keys expansion (this is as documented).
Since the array keys expansion isn't valid, the attempt to perform variable
indirection holds. This is where the helpful part comes in. Bash variables
can be referenced as arrays, even if they are not, using `0', `@', or `*'
as subscripts. That means that foo[@] gets expanded into "hello world",
which the shell tries to use as a variable name, resulting in:
$ ../bash-4.4-patched/bash ./x16
./x16: line 2: hello world: bad substitution
The error message in bash-5.0 is a little better:
$ ../bash-5.0-beta/bash ./x16
./x16: line 2: hello world: invalid variable name
>> This pattern of behavior is apparently unaffected by changes to IFS[...]
>
> Upon further examination, and in light of the above realization, this
> actually isn't true. In particular, iff the first character of IFS is
> alphanumeric or an underscore (or if IFS is the empty string), and if you
> use the "${array[*]}" form instead, then the expansion doesn't throw an
> error when the array contains more than one element.
Sure, since a double-quoted expansion using `*' separates words using the
first character of IFS.
> $ foo_bar='Beto2018'
> $ printf -- '%s\n' "${!array[*]-Warning: unset}"
> Beto2018
Nice. Let's hope he pulls it off today.
> Is there a good reason for treating "${!array[@]-}" and "${!array[*]-}"
> like indirections instead of index expansions (or just throwing an error)?
When I added array variables and the array keys expansion, I used the ksh93
syntax (${!var[sub]}) and tried to avoid conflict with the existing
indirect expansion as much as possible (that was back when we still thought
there was a chance that POSIX would standardize arrays and it was useful to
have consistent implementations). The ksh93 expansion syntax made it
invalid to use the array keys expansion as part of the ${param:-word}
expansion, so that's how I made it work. Since it wasn't a candidate for
that family of expansions, the existing variable indirection syntax
controls.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Indices of array variables are sometimes considered unset (or just display an error).
On 11/5/18 4:42 PM, Great Big Dot wrote:
> Description:
> The parameter expansion "${!var[@]}" expands to the indices of an array
> (whether linear or associative). The expansion "${var-string}"
> returns "${var}" iff var is set and 'string' otherwise. These two
> features do not play well together:
You seem to have neglected a significant section of the documentation:
"If the first character of parameter is an exclamation point (!), and
parameter is not a nameref, it introduces a level of indirection. Bash
uses the value formed by expanding the rest of parameter as the new
parameter; this is then expanded and that value is used in the rest of
the expansion, rather than the expansion of the original parameter.
This is known as indirect expansion. The value is subject to tilde
expansion, parameter expansion, command substitution, and arithmetic
expansion. If parameter is a nameref, this expands to the name of the
parameter referenced by parameter instead of performing the complete
indirect expansion. The exceptions to this are the expansions of
${!prefix*} and ${!name[@]} described below. The exclamation point
must immediately follow the left brace in order to introduce indirec-
tion."
> $ declare -a -- array=([0]='helloworld')
> $ printf -- '%s\n\n' "${!array[@]-Warning: unset}"
> Warning: unset
This happens to "work" because there is a single array element set, and it
expands to a single word.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Indices of array variables are sometimes considered unset (or just display an error).
On Mon, Nov 05, 2018 at 08:04:49PM -0500, Great Big Dot wrote:
> Hold up... when I view this email on the public archives, all of my
> "${array[@]}"'s (that is, "${array[]}"'a) got turned to
> "address@hidden"'s.
Sadly, there's nothing we can do about that. The maintainers of the
list archive would have to make their anti-spam measure a bit smarter
than it currently is.
Re: Indices of array variables are sometimes considered unset (or just display an error).
On Mon, Nov 5, 2018 at 10:38 PM Eduardo Bustamante
wrote:
> Sorry, I'm having a hard time following this email thread.
I *think* the point is that OP expected that:
(a) ${!var[@]-foo} expands to the indexes of var if ${var[@]} if set, else
to `foo'
whereas the behavior they observed is:
(b) ${!var[@]-foo} expands to the value of the variable whose name is
stored
in ${var[@]} or to `foo' if that variable is unset
Their expectation seems reasonable since "the variable whose name is stored
in ${var[@]}" is kind of a weird thing.
Re: Indices of array variables are sometimes considered unset (or just display an error).
On Mon, Nov 5, 2018 at 6:01 PM Great Big Dot wrote:
(...)
> > [... A]ccessing the index list of multiple-element arrays
> > fails when you append the unset expansion. With single-element
> > arrays, it fails iff the element in question contains any special
> > characters or whitespace, and thinks the array is unset otherwise.
> > (Further testing shows that a value of the empty string also throws
> > an error.) Finally, empty arrays are also considered unset[...]
>
> Oops, just realized what's causing this. I guess it isn't necessarily a
> bug? Debatable, I guess.
>
> What's actually happening here is that the *indirection* expansion
> "${!foo}", and not the *indices* expansion "${!foo[@]}", is what is being
> preformed on something like "${!array[@]-}". Both expansions, while
> unrelated, happen to use the same syntax, with the exception that
> indirections apply to normal variables and index expansions apply to array
> variables. For some reason, adding on the "${foo-default}" expansion causes
> the former to be used instead of the latter. This can be seen here:
Sorry, I'm having a hard time following this email thread.
What is your ultimate goal or the actual problem you're trying to solve?
(BTW, I would recommend against trying to do three expansions in one.
It might be more terse, but it's hard to read and as you found out,
leads to weird behavior)
Re: Indices of array variables are sometimes considered unset (or just display an error).
> On Mon, Nov 5, 2018 at 4:56 PM Great Big Dot
wrote:
> [... A]ccessing the index list of multiple-element arrays
> fails when you append the unset expansion. With single-element
> arrays, it fails iff the element in question contains any special
> characters or whitespace, and thinks the array is unset otherwise.
> (Further testing shows that a value of the empty string also throws
> an error.) Finally, empty arrays are also considered unset[...]
Oops, just realized what's causing this. I guess it isn't necessarily a
bug? Debatable, I guess.
What's actually happening here is that the *indirection* expansion
"${!foo}", and not the *indices* expansion "${!foo[@]}", is what is being
preformed on something like "${!array[@]-}". Both expansions, while
unrelated, happen to use the same syntax, with the exception that
indirections apply to normal variables and index expansions apply to array
variables. For some reason, adding on the "${foo-default}" expansion causes
the former to be used instead of the latter. This can be seen here:
$ array=(foo)
$ printf -- '%s\n' "${!foo[@]-unset}"
unset
$ foo='hello world'
$ printf -- '%s\n' "${!foo[@]-unset}"
hello world
So first the array is expanded, and then it's treated as a redirection, and
then the unset part kicks in if the array's value isn't an extant variable
name. This explains all the observations I made.
I still think it makes more sense if the "!" in "${!array[@]}" triggered
index expansion instead. At the very least, surely it should be one of
those expansion combinations that just isn't allowed, like
"${#foo[@]-default}" (actually, why is that disallowed?). Anyways, I don't
really see the point of the current behavior.
> This pattern of behavior is apparently unaffected by changes to IFS[...]
Upon further examination, and in light of the above realization, this
actually isn't true. In particular, iff the first character of IFS is
alphanumeric or an underscore (or if IFS is the empty string), and if you
use the "${array[*]}" form instead, then the expansion doesn't throw an
error when the array contains more than one element. E.g.:
$ array=(foo bar)
$ printf -- '%s\n' "${!array[*]-Warning: unset}"
bash: foo bar: bad substitution
$ IFS='_'
$ printf -- '%s\n' "${!array[*]-Warning: unset}"
Warning: unset
$ foo_bar='Beto2018'
$ printf -- '%s\n' "${!array[*]-Warning: unset}"
Beto2018
$ IFS=''
$ printf -- '%s\n' "${!array[*]-Warning: unset}"
Warning: unset
$ foobar='Hello, world'
$ printf -- '%s\n' "${!array[*]-Warning: unset}"
Hello, world
Though I understand it now, the above behavior doesn't seem especially
motivated to me. I mean, the variables that end up getting expanded don't
actually have their names stored anywhere, yet the indirection points to
them.
Is there a good reason for treating "${!array[@]-}" and "${!array[*]-}"
like indirections instead of index expansions (or just throwing an error)?
Re: Indices of array variables are sometimes considered unset (or just display an error).
On Mon, Nov 5, 2018 at 4:56 PM Great Big Dot wrote:
> The parameter expansion "${!var[@]}" expands to the indices of an array
(whether linear or associative).
Hold up... when I view this email on the public archives, all of my
"${array[@]}"'s (that is, "${array[]}"'a) got turned to
"address@hidden"'s. Was I supposed to use some escape sequence or
something? Is everyone who's subscribed to the mailing list able to see the
actual text? Or should I resend this bug report with all \@-signs escaped
somehow?
Testing...
test...@example.com
testing＠example.com
testing﹫example.com
testing\@example.com
testing @ example.com
