Re: "unset var" pops var off variable stack instead of unsetting it
Op 21-03-17 om 16:38 schreef Stephane Chazelas:
> IOW, the work around I was mentioning earlier (of using "local"
> before "unset" to make sure "unset" unsets) doesn't work in that
> case. You'd need to use the same work around as for mksh/yash
> (call unset in a loop until the variable is really unset (with
> the nasty side effect of unsetting the variable in a scope
> you're need meant to tamper with) so you'd want to do it in a
> subshell).
Note that this workaround needs to be applied conditionally on
cross-platform POSIX scripts because you'd get an infinite loop on
recent-ish versions of ksh93 (as of 2010, IIRC). Those ksh93 versions
have BUG_IFSISSET: it is not possible to determine in any normal way if
IFS is set, neither with "${IFS+set}" nor with [[ -v IFS ]]. They always
act as if IFS is set, even when it is unset. This applies to IFS only.
Upon detecting BUG_IFSISSET, modernish applies a workaround to isset()*
that involves analysing field splitting behaviour to distinguish between
empty IFS and unset IFS. So 'isset IFS' is fully cross-platform.
- M.
* https://github.com/modernish/modernish#working-with-variables
Re: "unset var" pops var off variable stack instead of unsetting it
> The effect of unset on a local was what I had in mind, but really the > manual says very little about scope. All it says right now is: > > "Variables local to the function may be declared with the local builtin > command. Ordinarily, variables and their values are shared between the > function and its caller." > > Which doesn't exactly describe dynamic scope even for those that know > what that means. Here's what I have to start: Variables local to the function may be declared with the local builtin command. Ordinarily, variables and their values are shared between the function and its caller. If a variable is declared local, the vari- able's visible scope is restricted to that function and its children (including the functions it calls). Local variables "shadow" variables with the same name declared at previous scopes. For instance, a local variable declared in a function hides a global variable of the same name: references and assignments refer to the local variable, leaving the global variable unmodified. When the function returns, the global variable is once again visible. The shell uses dynamic scoping to control a variable's visibility within functions. With dynamic scoping, visible variables and their values are a result of the sequence of function calls that caused exe- cution to reach the current function. The value of a variable that a function sees depends on its value within its caller, if any, whether that caller is the "global" scope or another shell function. This is also the value that a local variable declaration "shadows", and the value that is restored when the function returns. For example, if a variable var is declared as local in function func1, and func1 calls another function func2, references to var made from within func2 will resolve to the local variable var from func1, shadow- ing any global variable named var. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/21/17 3:19 AM, Dan Douglas wrote: > Also not documented is how a variable declared with declare/typeset is > distinct from an unset variable. I don't know, I think this is pretty clear: "A parameter is set if it has been assigned a value." The previous paragraph discusses attributes. The subsequent paragraph discusses how values are assigned. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-20 16:32:10 -0400, Chet Ramey:
[...]
> > See also:
> >
> > $ bash -c 'f() { unset a; echo "$a";}; a=1; a=2 f'
> > 1
> >
> > already mentioned.
>
> A distinction without a difference; the behavior is explicitly the same.
[...]
One I haven't mentioned yet is:
$ bash -c 'f() { local a; unset a; echo "$a";}; a=1; a=2 f'
1
IOW, the work around I was mentioning earlier (of using "local"
before "unset" to make sure "unset" unsets) doesn't work in that
case. You'd need to use the same work around as for mksh/yash
(call unset in a loop until the variable is really unset (with
the nasty side effect of unsetting the variable in a scope
you're need meant to tamper with) so you'd want to do it in a
subshell).
--
Stephane
Re: "unset var" pops var off variable stack instead of unsetting it
On 03/18/2017 12:19 PM, Chet Ramey wrote: > On 3/17/17 6:35 PM, Dan Douglas wrote: > >> The problem is the non-obvious nature of unset's interaction with scope, >> (and the lack of documentation). Not much can be done about the former, >> as it is with so many things. > > How would you suggest improving the documentation? I can see highlighting > the fact that unset applied to a local variable at the same scope > preserves the local attribute. What else? > The effect of unset on a local was what I had in mind, but really the manual says very little about scope. All it says right now is: "Variables local to the function may be declared with the local builtin command. Ordinarily, variables and their values are shared between the function and its caller." Which doesn't exactly describe dynamic scope even for those that know what that means. Also not documented is how a variable declared with declare/typeset is distinct from an unset variable.
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/20/17 4:29 PM, Stephane Chazelas wrote:
>> I believe he means the behavior of `a=0; a=1 eval unset a', which Posix
>> implicitly requires affect the global scope and results in a being unset
>> when the statement completes.
> [...]
>
> See also:
>
> $ bash -c 'f() { unset a; echo "$a";}; a=1; a=2 f'
> 1
>
> already mentioned.
A distinction without a difference; the behavior is explicitly the same.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-20 14:47:17 -0400, Chet Ramey:
> On 3/20/17 2:30 PM, Eric Blake wrote:
> > On 03/17/2017 07:21 PM, Stephane Chazelas wrote:
> >
> >>> The problem is the non-obvious nature of unset's interaction with scope,
> >>
> >> the main problem to me is an unset command that doesn't unset.
> >>
> >> As shown in my original post, there's also a POSIX conformance
> >> issue.
> >
> > As POSIX has not yet specified 'local', any use of 'local' already
> > renders the script non-conformant, so it shouldn't matter what bash does
> > in that situation (although if POSIX is ever going to standardize
> > 'local', it requires some concerted effort to make all shells with
> > 'local' to settle on a lowest common denominator).
>
> I believe he means the behavior of `a=0; a=1 eval unset a', which Posix
> implicitly requires affect the global scope and results in a being unset
> when the statement completes.
[...]
See also:
$ bash -c 'f() { unset a; echo "$a";}; a=1; a=2 f'
1
already mentioned.
In any case, those are corner cases I'm not too worried about.
I'm more concerned about "unset var" not unsetting var in real
life cases when "local"/"typeset" is being used (regardless of
POSIX).
The other POSIX related concern I was also mentioning is the
fact that the work around implies using non-POSIX constructs,
the fact that libraries of POSIX functions can't be used in
bash/mksh/yash.
The particular case that affects me directly, is that
recommendations I'm giving online about POSIX compatible
constructs such as:
(unset IFS; set -f; cmd -- $var)
Which is (or at least I though should be) *the* Bourne idiom to
split, already mentioned several times are *wrong* for bash (or
mksh or yash) in the general case, as "unset" doesn't do what it
says on the can there.
Another case of "principle of least astonishment" not being
followed IMO.
--
Stephane
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/20/17 2:30 PM, Eric Blake wrote: > On 03/17/2017 07:21 PM, Stephane Chazelas wrote: > >>> The problem is the non-obvious nature of unset's interaction with scope, >> >> the main problem to me is an unset command that doesn't unset. >> >> As shown in my original post, there's also a POSIX conformance >> issue. > > As POSIX has not yet specified 'local', any use of 'local' already > renders the script non-conformant, so it shouldn't matter what bash does > in that situation (although if POSIX is ever going to standardize > 'local', it requires some concerted effort to make all shells with > 'local' to settle on a lowest common denominator). I believe he means the behavior of `a=0; a=1 eval unset a', which Posix implicitly requires affect the global scope and results in a being unset when the statement completes. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/ signature.asc Description: OpenPGP digital signature
Re: "unset var" pops var off variable stack instead of unsetting it
It seems to me this matter is specific to the IFS variable (and possibly few others like CDPATH). Unsetting IFS restores the default field splitting behavior, but unsetting PATH does not restore the default command search path. As Peter suggests, you can locally re-define any variables you want and that should work in any situation. -- Yuki
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-20 08:04:33 -0400, Greg Wooledge: [...] > > Credits to Dan Douglas > > (https://www.mail-archive.com/miros-mksh@mirbsd.org/msg00707.html) > > for finding the bug. He did find a use for it though (get the > > value of a variable from the caller's scope). > > Isn't this exactly the same as the "upvar" trick that Freddy Vulto > discovered? http://www.fvue.nl/wiki/Bash:_Passing_variables_by_reference Hi Greg, Yes, I hadn't realised initially that the issue had already been discussed before (and not fixed). You'll find that that "upvar" and the link above has since been mentioned in this thread (see also https://www.mail-archive.com/bug-bash@gnu.org/msg19445.html) At this point, I have little hope that bash will be fixed. But mksh/oksh and yash still might. -- Stephane
Re: "unset var" pops var off variable stack instead of unsetting it
On Fri, Mar 17, 2017 at 09:51:34PM +, Stephane Chazelas wrote: > Then, the "unset", instead of unsetting IFS, actually pops a > layer off the stack. > Credits to Dan Douglas > (https://www.mail-archive.com/miros-mksh@mirbsd.org/msg00707.html) > for finding the bug. He did find a use for it though (get the > value of a variable from the caller's scope). Isn't this exactly the same as the "upvar" trick that Freddy Vulto discovered? http://www.fvue.nl/wiki/Bash:_Passing_variables_by_reference
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-20 12:30:09 +0900, 渡邊裕貴:
> It seems to me this matter is specific to the IFS variable (and possibly
> few others like CDPATH). Unsetting IFS restores the default field splitting
> behavior, but unsetting PATH does not restore the default command search
> path. As Peter suggests, you can locally re-define any variables you
> want and that should work in any situation.
[...]
Hi Yuki,
you seem to be concurring with me that unset is broken and that
the work around is to not use it.
Note that unsetting PATH generally *does* restore a default
command search path. However, on many systems, not everything
agrees on the default search path (for instance on my Debian
system, for execvp(), it's ":/bin:/usr/bin" (yes, current
directory first!), for bash and dash it seems to be only the
current directory (as if PATH was set to the empty string), for
yash it seems it's nothing, for mksh "/usr/bin:/bin", for ksh93
"/bin:/usr/bin"... behaviour left "implementation defined" by
POSIX) so unsetting PATH is not useful.
Now, there are many reasons one may want to use unset.
For instance, unsetting LC_* restores LANG, one may want to
unset LD_LIBRARY_PATH, GCONV_PATH, LOCPATH, PERL5LIB,
PYTHON_PATH... for security reasons or to get a consistent
behaviour. In POSIX sh language, unsetting a variable is the
only way to unexport it. Same for changing the type of a
variable to scalar in bash without declaring it local.
zsh/yash/mksh have "typeset -g" for that, but in bash typeset -g
affects the variable in the global scope instead of preventing
the restricting the scope in other shells.
unset is also commonly used to make sure variables /have a
default value of / like in things like:
rmv() (
unset OPTIND force interactive verbose
while getopts :ivf o; do
(f) force=1;;
...
esac
shift "$((OPTIND - 1))"
exec rm ... ${force+"-f"} "$@"
)
Replacing the "unset force" with "force=" (and use
${force:+"-f"}) to work around the unset bug would not be enough
with bash/mksh as $force might have been defined as "integer" in
a parent scope. So one would need to use "typeset force=", or
just "typeset foo" which declares it with an value,
but that would make it no longer standard sh code (so that
function can no longer be used in sh scripts).
--
Stephane
Re: "unset var" pops var off variable stack instead of unsetting it
On 20/03/2560 04:51, Stephane Chazelas wrote: > On comp.unix.shell ot http://unix.stackexchange.com, I've posted > many articles describing how to do splitting in POSIX-like > shells: > > ( # subshell for local scope > unset -v IFS # restore default splitting behaviour > set -o noglob # disable globbing > cmd -- $var # split+glob with default IFS and glob disabled > ) > > I'm now considering adding a note along the lines of: > > "Beware that with current versions of bash, pdksh and yash, > the above may not work if used in scripts that otherwise use > typeset/declare/local on $IFS or call a function with > `IFS=... my-function' (or IFS=... eval... or IFS=... > source...)" > Wouldn't it be better to avoid using a function like 'unset' (that works in the way you purport to expect) in a dynamically scoped language as a matter of principle?? If unset works like you would want/expect it, it would also discard all values of all higher scopes. Would it not be better to set IFS to the value desired (whatever default splitting behaviour you need)?? -- Peter
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/19/17 6:22 PM, Stephane Chazelas wrote:
>> $ cat x2
>> function foo
>> {
>> (
>> unset -v IFS
>> recho "${IFS-unset}"
>> )
>> }
>>
>> IFS=':|'
>> foo
>> echo after IFS = "$IFS"
>> $ ../bash-4.4-patched/bash ./x2
>> argv[1] =
>> after IFS = :|
>
> Yes, that one is fine but it is not the issue that is being
> discussed here. There's no variable to pop off a stack above.
>
> the issue is when that "foo" function is called in a context
> where IFS had been declared locally. Like in:
>
> IFS=1
> function example {
> typeset IFS=2
> foo
> }
>
> Where "foo" would output "1", because then "unset -v IFS" would
> *not* have unset IFS but instead would have restored the value
> it had before the "typeset" (in that case, the global scope).
>
Yeah, that's how local variables and dynamic scoping in bash have always
worked. That ship really has sailed.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-19 18:05:19 -0400, Chet Ramey:
> On 3/19/17 5:51 PM, Stephane Chazelas wrote:
>
> > On comp.unix.shell ot http://unix.stackexchange.com, I've posted
> > many articles describing how to do splitting in POSIX-like
> > shells:
> >
> > ( # subshell for local scope
> > unset -v IFS # restore default splitting behaviour
> > set -o noglob # disable globbing
> > cmd -- $var # split+glob with default IFS and glob disabled
> > )
> >
> > I'm now considering adding a note along the lines of:
> >
> > "Beware that with current versions of bash, pdksh and yash,
> > the above may not work if used in scripts that otherwise use
> > typeset/declare/local on $IFS or call a function with
> > `IFS=... my-function' (or IFS=... eval... or IFS=...
> > source...)"
>
> You can, of course, do whatever you want. You might want to read my
> message from yesterday about what happens when you do that, or look
> at the following examples, after which you may decide that the situation
> is not as dire.
>
> $ cat x2
> function foo
> {
> (
> unset -v IFS
> recho "${IFS-unset}"
> )
> }
>
> IFS=':|'
> foo
> echo after IFS = "$IFS"
> $ ../bash-4.4-patched/bash ./x2
> argv[1] =
> after IFS = :|
Yes, that one is fine but it is not the issue that is being
discussed here. There's no variable to pop off a stack above.
the issue is when that "foo" function is called in a context
where IFS had been declared locally. Like in:
IFS=1
function example {
typeset IFS=2
foo
}
Where "foo" would output "1", because then "unset -v IFS" would
*not* have unset IFS but instead would have restored the value
it had before the "typeset" (in that case, the global scope).
--
Stephane
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/19/17 5:51 PM, Stephane Chazelas wrote:
> On comp.unix.shell ot http://unix.stackexchange.com, I've posted
> many articles describing how to do splitting in POSIX-like
> shells:
>
> ( # subshell for local scope
> unset -v IFS # restore default splitting behaviour
> set -o noglob # disable globbing
> cmd -- $var # split+glob with default IFS and glob disabled
> )
>
> I'm now considering adding a note along the lines of:
>
> "Beware that with current versions of bash, pdksh and yash,
> the above may not work if used in scripts that otherwise use
> typeset/declare/local on $IFS or call a function with
> `IFS=... my-function' (or IFS=... eval... or IFS=...
> source...)"
You can, of course, do whatever you want. You might want to read my
message from yesterday about what happens when you do that, or look
at the following examples, after which you may decide that the situation
is not as dire.
$ cat x2
function foo
{
(
unset -v IFS
recho "${IFS-unset}"
)
}
IFS=':|'
foo
echo after IFS = "$IFS"
$ ../bash-4.4-patched/bash ./x2
argv[1] =
after IFS = :|
$ cat x2a
function foo
{
(
unset -v IFS
recho "${IFS-unset}"
foo='a:b:c:d'
recho $foo
)
}
IFS=':|'
foo
echo after IFS = "$IFS"
$ ../bash-4.4-patched/bash ./x2a
argv[1] =
argv[1] =
after IFS = :|
or
$ cat x2b
function foo
{
typeset IFS='+'
unset -v IFS
recho "${IFS-unset}"
}
IFS=':|'
foo
echo after IFS = "$IFS"
$ ../bash-4.4-patched/bash ./x2b
argv[1] =
after IFS = :|
or even
$ cat x2c
function foo
{
typeset MIFS='+'
unset -v MIFS
recho "${MIFS-unset}"
}
MIFS=':|'
foo
echo after MIFS = "$MIFS"
$ ../bash-4.4-patched/bash ./x2c
argv[1] =
after MIFS = :|
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-18 13:16:56 -0400, Chet Ramey:
> On 3/17/17 5:51 PM, Stephane Chazelas wrote:
>
> > Now, if that "split" functions is called from within a function
> > that declares $IFS local like:
> [...]
> > because after the "unset IFS", $IFS is not unset (which would
> > result in the default splitting behaviour) but set to ":" as it
> > was before "bar" ran "local IFS=."
[...]
For bash, it looks like the boat has sailed as the issue has
been discussed before, but let me at least offer my opinion, and
also add the maintainer of yash and mksh in Cc so they can
comment as they have similar issues in their shell which they
may want to address (at least in the documentation). It's even
worse for mksh and yash as it's harder to work around there.
For Yuki and Thorsten, see the start of the discussion at
https://www.mail-archive.com/bug-bash@gnu.org/msg19431.html
(and
https://www.mail-archive.com/miros-mksh@mirbsd.org/msg00697.html
before that)
In short, the issue is that "unset var" does not always leave
$var unset (contrary to what the documentation or the name of
the command suggest) but may instead restore a previous value.
Reproducer for bash/pdksh/yash:
$ f()(unset a; echo "$a"); g() { typeset a=2; f; }; a=1; g
1
For bash, also:
$ f()(unset a; echo "$a"); a=1; a=2 f
1
One work around for bash is:
f()(local a; unset a; echo "$a"); g() { typeset a=2; f; }; a=1; g
as long as we want "$a" to be unset only for the local
function (already enforced by the subshell in this case) or with
bash/mksh/yash:
f()(while [ "${a+set}" ]; do unset a; done
echo "$a"); g() { typeset a=2; f; }; a=1; g
(again, not likely to do what we want when not called in a
subshell)
(see also
f()(unset "a[0]"; echo "$a"); g() { typeset a=2; f; }; a=1; g
in pdksh that doesn't unset "$a" but makes it an array with no
element).
>
> This is how local variables work. Setting a local variable shadows
> any global with the same name; unsetting a local variable reveals the
> global variable (or really, because bash has dynamic scoping, the value
> at a previous scope) since there is no longer a local variable to shadow
> it.
[...]
Chet, the behaviour you describe above would be that of a "popvar"
(not "unset") command, an arcane command for an arcane feature:
pop a variable off a stack to restore the value (and attributes)
it had in an outer scope. A feature I would probable never need
in a million years. The only known usage of it being that hack
(http://www.fvue.nl/wiki/Bash:_Passing_variables_by_reference)
to be able to return a value into a variable passed as argument
to a function while still being able to use a local variable
with the same name in the function.
There is no way any sane person would write
unset IFS
and mean anything else than unsetting the IFS variable (make
sure $IFS is not set afterwards so word splitting revers to the
default).
There's no way any sane person would expect that to mean
"restore the variable from an outer scope I don't known about"
(yash/pdksh) or in the case of bash: "restore the variable from
an outer scope unless I've declared it in the current function
context".
unsetting variables is an essential feature in shells as many
variables especially those in the environment have a special
meaning, affect the environment when set.
I can't imagine it being anything other than an unintended
accident of implementation, certainly not an intentional feature
of the language (at least not initially).
In all other languages that have a "unset"/"undef"/"delete"
similar feature (tcl, perl, php, ksh88 (dynamic scoping), ksh93
(static scoping), zsh, dash at least), unset unsets the variable
in the innest scope it has been declared in. I don't know of any
language that has a "popvar" feature to allow the user to unravel
the variable stack behind the back of the interpreter.
Several languages with static scoping (tcl with upvar, ksh93
with "typeset -n", python3 with nonlocal at least) have a way to
access variables in a parent scope, but with dynamic scoping,
there's no need for that. Child functions already have access to
the parent variables.
The issue (that there's no notion of variable reference in
those shells) that
http://www.fvue.nl/wiki/Bash:_Passing_variables_by_reference
tries to hack around is better addressed IMO with namespacing
(like return the value in a dedicated variable (REPLY for
instance is already used for that internally in bash and several
other shells) or make sure utility functions that modify
arbitrary variables use a dedicated prefix for their own
variables))
In any case, even if that was an essential feature, it would not
be a good reason for breaking the "unset" command or at least
subvert its meaning. Implementing "typeset -n" like in ksh93 or
an "upvar" builtin a la tcl would make a lot more sense IMO.
On comp.unix.shell ot http://unix.stackexchange.com, I've posted
many articles describing how to do
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/17/17 8:21 PM, Stephane Chazelas wrote:
> I don't expect the need to have to add "local var" in
>
> (
>unset -v var
>echo "${var-OK}"
> )
>
> would be obvious to many people beside you though.
Try the following:
function foo
{
(
unset -v IFS
recho "${IFS-unset}"
foo='a:b:c:d'
recho $foo
)
}
IFS=':|'
foo
echo after IFS = "$IFS"
Bash and ksh93 echo '', '', and ':|'.
> People writing function libraries meant to be used by several
> POSIX-like shells need to change their code to:
>
> split() (
> [ -z "$BASH_VERSION" ] || local IFS # WA for bash bug^Wmisfeature
> unset -v IFS
> set -f
> split+glob $1
> )
>
> if they want them to be reliable in bash.
Well, there's the fact that you've left the realm of standardization when
you start talking about local variables, so "Posix-like" doesn't have
much meaning.
>
>> The problem is the non-obvious nature of unset's interaction with scope,
>
> the main problem to me is an unset command that doesn't unset.
So you don't like dynamic scoping. This is not new.
> As shown in my original post, there's also a POSIX conformance
> issue.
Yeah, it looks like there is differing behavior that has to do with the
"special builtin" nature of eval and unset. In Posix, these two statements
are essentially equivalent:
a=1 eval unset a
a=1; eval unset a
I am not a fan of the special builtin behavior rules.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/17/17 6:35 PM, Dan Douglas wrote: > The problem is the non-obvious nature of unset's interaction with scope, > (and the lack of documentation). Not much can be done about the former, > as it is with so many things. How would you suggest improving the documentation? I can see highlighting the fact that unset applied to a local variable at the same scope preserves the local attribute. What else? -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
On 3/17/17 5:51 PM, Stephane Chazelas wrote:
> Now, if that "split" functions is called from within a function
> that declares $IFS local like:
[...]
> because after the "unset IFS", $IFS is not unset (which would
> result in the default splitting behaviour) but set to ":" as it
> was before "bar" ran "local IFS=."
This is how local variables work. Setting a local variable shadows
any global with the same name; unsetting a local variable reveals the
global variable (or really, because bash has dynamic scoping, the value
at a previous scope) since there is no longer a local variable to shadow
it.
>
> A simpler reproducer:
>
> $ bash -c 'f()(unset a; echo "$a"); g(){ local a=1; f;}; a=0; g'
> 0
>
> Or even with POSIX syntax:
>
> $ bash -c 'f()(unset a; echo "$a"); a=0; a=1 eval f'
> 0
In this case, the unset unsets the variable provided in the builtin's
`environment'. It's essentially the same thing.
>
> A work around is to change the "split" function to:
>
> split() (
> local IFS
> unset -v IFS # default splitting
> set -o noglob # disable glob
>
> set -- $1 # split+(no)glob
> [ "$#" -eq 0 ] || printf '<%s>\n' "$@"
> )
>
> For some reason, in that case (when "local" and "unset" are
> called in the same function context), unset does unset the
> variable.
There's special code in bash to preserve the locally-declared nature of
a variable if you use local and unset in the same function scope. That's
been in bash forever (at least as far back as bash-2.0, which is when I
quit looking), for compatibility -- that's how ksh93 behaves -- and user
expectations.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: "unset var" pops var off variable stack instead of unsetting it
On 03/17/2017 09:16 PM, Dan Douglas wrote:
> Why
> would a subshell just make the call stack go away?
I guess slight correction, it's unset itself, because:
> In fact, mksh prints "global" even without the subshell, despite it
> using dynamic scope for either function definition syntax.
Another "not-sure-if-bug-or-feature". It is a way to guarantee reaching
the global scope, which is impossible in bash, short of calling unset
${#FUNCNAME[@]} times.
If feature, I'm not instantly in love with it.
Re: "unset var" pops var off variable stack instead of unsetting it
On 03/17/2017 07:21 PM, Stephane Chazelas wrote:
> I don't expect the need to have to add "local var" in
>
> (
>unset -v var
>echo "${var-OK}"
> )
True. I would pretty much never use a subshell command group when I know
that locals are available though. And if I know locals are available then
(except dash) I know arrays are available, in which case I'd almost never
use field splitting. This is only like the millionth screwy gotcha with
IFS. Everybody knows IFS is broken beyond repair :o)
Then again, you could easily write a similar bug with any other special
variable that has a side-effect.
> would be obvious to many people beside you though.
> very
> People writing function libraries meant to be used by several
> POSIX-like shells need to change their code to:
>
> split() (
> [ -z "$BASH_VERSION" ] || local IFS # WA for bash bug^Wmisfeature
> unset -v IFS
> set -f
> split+glob $1
> )
>
> if they want them to be reliable in bash.
Even if the inconsistent effect of unset isn't obvious, it should be
obvious that a subshell isn't equivalent to setting a local, because it
doesn't just make dynamic scope go away.
I'm far more surprised by the behavior of mksh and dash, in which it's
the subshell rather than the unset builtin that's inconsistent. Why
would a subshell just make the call stack go away? That makes no sense,
and a subshell isn't supposed to do that. Dash and mksh don't even agree
with one another on how that works:
(cmd) ~ $ mksh /dev/fd/3 3<<\EOF
function f { typeset x=f; g; }
function g { ( unset x; echo "${x-unset}"; ) }
x=global; f
EOF
global
(ins) ~ $ dash /dev/fd/3 3<<\EOF
alias typeset=local function=
function f() { typeset x=f; g; }
function g() { ( unset x; echo "${x-unset}"; ) }
x=global; f
EOF
unset
In fact, mksh prints "global" even without the subshell, despite it
using dynamic scope for either function definition syntax.
At least bash's output in this case (empty) can be fully explained as
a combination of quirks with unset and hidden locals (neither being
documented), plus dynamic scope being what it is.
We're pretty much arguing over which is the less counter-intuitive
inconsistency here. If mksh's subshells worked consistently as in bash,
you'd have written the same bug as bash in your example. And it would
be even easier to do so without the unset quirk since this could happen
within a single function call too:
(cmd) ~ $ mksh /dev/fd/3 3<<\EOF
function f {
typeset x=f
( unset x; echo "${x-unset}"; )
}
x=global; f
EOF
global
This could really bite if x and f are defined in separate files so the
initial state of x isn't necessarily known.
> So what should the documentation be? With my "eval" case in
> mind, it's hard to explain without getting down to how stacking
> variables work. Maybe something like:
>
> [...]
All that touches on several issues in addition to scope, such as
the various states that a variable can be in, and the exact nature
of references to variables like 'a[@]'. That's some of the least-well
documented stuff, but some of that should also probably be left subject
to change due to the great inconsistency across shells and other issues
just within bash. (Ugh also have to mention the stupid 'a[0]' with
associative arrays - that's one where "consistency" is itself a bug).
> It might be worth pointing out that "unset -v", contrary to the
> default behaviour, won't unset functions so it's a good idea to
> use "unset -v" instead of "unset" if one can't guarantee that
> the variable was set beforehand (like the common case of using
> unset to remove a variable which was potentially imported from
> the environment).
Yeah. I believe POSIX mentions that as well.
Re: "unset var" pops var off variable stack instead of unsetting it
2017-03-17 17:35:36 -0500, Dan Douglas:
> The need to localize IFS is pretty obvious to me - of course that's
> given prior knowledge of how it works.
[...]
I don't expect the need to have to add "local var" in
(
unset -v var
echo "${var-OK}"
)
would be obvious to many people beside you though.
People writing function libraries meant to be used by several
POSIX-like shells need to change their code to:
split() (
[ -z "$BASH_VERSION" ] || local IFS # WA for bash bug^Wmisfeature
unset -v IFS
set -f
split+glob $1
)
if they want them to be reliable in bash.
> The problem is the non-obvious nature of unset's interaction with scope,
the main problem to me is an unset command that doesn't unset.
As shown in my original post, there's also a POSIX conformance
issue.
> (and the lack of documentation). Not much can be done about the former,
> as it is with so many things.
So what should the documentation be? With my "eval" case in
mind, it's hard to explain without getting down to how stacking
variables work. Maybe something like:
after unset -v var
- if var had been declared (without -g) in the current
function scope (not the global scope), $var becomes unset in
the current scope (not in parent scopes). Futher unset
attempts will not affect the variable in parent scopes.
- otherwise, the previous var value (and type and attributes)
is popped from a stack. That stack is pushed every time the
variable is declared without -g in a new function scope or
when the "." or "eval" special builtins are invoked as var=x
eval 'code' or var=x . sourced-file. If the stack was empty,
the variable is unset.
There's also missing documentation for:
unset -v 'var[x]' (note the need to quote that glob)
can only be used if "var" is an array or hash variable and unsets
the array/hash element of key x. Unsetting the last element
does not unset the variable. For arrays, negative subscripts
are relative to the greatest assigned subscript in the array.
unset "a[-1]" "a[-1]" unsets the 2 elements with the greatest
subscript, but that's not necessarily the case for unset
"a[-2]" "a[-1]" if the array was sparse.
unset "var[@]" or unset "var[*]" can be used to unset all the
elements at once. For associative arrays, use unset 'a[\*]' or
unset 'a[\@]' to unset the elements of key * and @. It is not
possible [AFAICT] to unset the element of key "]" or where the
key consists only of backslash characters [btw, it also looks
like bash hashes (contrary to zsh or ksh93 ones) can't have an
element with an empty key]
It is not an error to attempt to "unset" a variable or array
element that is not set, except when using negative subscripts.
Also, the doc says:
> The -v flag specifies that NAME refers to parameters.
> This is the default behaviour.
It might be worth pointing out that "unset -v", contrary to the
default behaviour, won't unset functions so it's a good idea to
use "unset -v" instead of "unset" if one can't guarantee that
the variable was set beforehand (like the common case of using
unset to remove a variable which was potentially imported from
the environment).
--
Stephane
Re: "unset var" pops var off variable stack instead of unsetting it
The need to localize IFS is pretty obvious to me - of course that's given prior knowledge of how it works. The problem is the non-obvious nature of unset's interaction with scope, (and the lack of documentation). Not much can be done about the former, as it is with so many things.
Re: "unset var" pops var off variable stack instead of unsetting it
It's probably easiest to find previous discussions here on this topic by searching for 'upvar' (common name for a function that takes advantage of this behavior). Latest I think was this one [1] but a much earlier discussion is here [2]. [1] https://lists.gnu.org/archive/html//bug-bash/2015-01/msg00018.html [2] https://lists.gnu.org/archive/html/bug-bash/2008-10/msg00107.html
