Re: Return status of command substitution with $(...) "gets lost"

[Chet Ramey]

On 3/11/10 11:32 AM, Eric Blake wrote:

> If you'd like 'local' to be standardized in the next version of POSIX,
> several years from now, that would also be a good goal; help in
> submitting the proposal to the Austin group would be appreciated.

I thought an early draft of Posix.2 (in those days) included a definition
of `local' and its behavior (d9, probably).  I just can't lay my hands
on a copy right now.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Return status of command substitution with $(...) "gets lost"

[Eric Blake]

On 03/11/2010 09:27 AM, Marc Herbert wrote:
> Chet Ramey wrote:
> 
>> To Posix, assignment statements never fail -- assignment errors cause
>> non- interactive shells to exit, period.  In that case, it's possible
>> to reflect the exit status of a command substitution in the exit
>> status of a command consisting only of assignment statements,...
> 
> It's possible... and required by POSIX, right? 

POSIX says nothing about local.  But yes, POSIX requires that:

a=$(b)

set $? to the status of running the command substitution.

> Here you seem to imply this is actually an "advanced" and thus
> non-standard feature (why I have no clue). An unreliable feature would
> never have been standardized for sure...

The question at hand is whether:

local a=$(b)

should set the exit status to the command substitution (a change in
behavior), or to the status of running the 'local' command (the current
behavior).  I threw out the question about change as a discussion point,
but I'm perfectly fine with Chet's response that the current behavior
will stay as is.

If you'd like 'local' to be standardized in the next version of POSIX,
several years from now, that would also be a good goal; help in
submitting the proposal to the Austin group would be appreciated.

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Return status of command substitution with $(...) "gets lost"

[Marc Herbert]

Chet Ramey wrote:

> To Posix, assignment statements never fail -- assignment errors cause
> non- interactive shells to exit, period.  In that case, it's possible
> to reflect the exit status of a command substitution in the exit
> status of a command consisting only of assignment statements,...

It's possible... and required by POSIX, right? 

At least this is my understanding of section "2.9.1 Simple Commands".
More precisely: "... with the exit status of the LAST command
substitution performed"

Just to make things less abstract, here is some (simplified) code I
often find useful:

local _res
if _res=$( some_code... ); then
   echo $_res
elif _res=$( some_other_code... );
   echo $_res
else
   return 1
fi


> ... though that has problems of its own and is only reliable on commands
> consisting of a single assignment.

Here you seem to imply this is actually an "advanced" and thus
non-standard feature (why I have no clue). An unreliable feature would
never have been standardized for sure...

Re: Return status of command substitution with $(...) "gets lost"

[Chet Ramey]

On 3/11/10 4:24 AM, Marc Herbert wrote:
> Chet Ramey a écrit :
>>> On the other hand, local is not specified by POSIX.  What would it hurt
>>> to redefine the exit status of local to reflect the status of any
>>> command substitutions performed during the local assignments (short of
>>> compatibility with zsh)?
>>
>> I think that would be a bad idea, and am not inclined to do it.
> 
> Care to elaborate a bit? (preferably on the public list)

Because that's not what local and its siblings (declare/typeset/export/
readonly) do.  These builtins exist to assign and modify variable
attributes.  As an added feature, they support value assignment at the
same time, but the important function is the attribute setting.  They
don't need to know how the value was computed.

(Yes, a secondary function is to insulate the shell programmer from
assignment errors that might otherwise cause the shell to exit, and the
ability to create variables with local scope is clearly a hole in Posix.)

Since the function is setting the attribute or value, the exit status
should reflect whether or not that succeeded.  For instance, attempting
to modify a read-only variable, or to assign a variable with an invalid
name, which would cause a Posix shell to exit, should be reflected in the
exit status.

For instance, should this succeed or fail?

local -r foo=$(echo a; exit 1)

After all, the variable was correctly created, and ends up with the desired
attribute.  Is that not the intent of the statement?

To Posix, assignment statements never fail -- assignment errors cause non-
interactive shells to exit, period.  In that case, it's possible to
reflect the exit status of a command substitution in the exit status of a
command consisting only of assignment statements, though that has problems
of its own and is only reliable on commands consisting of a single
assignment.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Return status of command substitution with $(...) "gets lost"

[Chet Ramey]

> On the other hand, local is not specified by POSIX.  What would it hurt
> to redefine the exit status of local to reflect the status of any
> command substitutions performed during the local assignments (short of
> compatibility with zsh)?

I think that would be a bad idea, and am not inclined to do it.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Return status of command substitution with $(...) "gets lost"

[Eric Blake]

On 03/10/2010 10:28 AM, Marc Herbert wrote:
> Chet Ramey a écrit :
>> On 3/4/10 2:36 AM, Ettelbrueck, Heiko wrote:
>>> Problem: The $? variable is always 0 after that statement. (If,
>>> on the other hand, I separate the declaration and the
>>> definition of the variable as shown in the example below, the
>>> $? variable is really set to the exit status of the external
>>> tool.)
>>
>> The exit status is the status of the command you run: local.  In the
>> absence of a command, when there are only assignment statements, the
>> exit status can be the exit status of a command substitution:
> 
> I have been bitten by this a few times. Since then I never, ever write this:
>   local foo=...
> 
> But always:
>   local foo; foo=...

On the other hand, local is not specified by POSIX.  What would it hurt
to redefine the exit status of local to reflect the status of any
command substitutions performed during the local assignments (short of
compatibility with zsh)?

-- 
Eric Blake   ebl...@redhat.com+1-801-349-2682
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: Return status of command substitution with $(...) "gets lost"

[Marc Herbert]

Chet Ramey a écrit :
> On 3/4/10 2:36 AM, Ettelbrueck, Heiko wrote:
>> Problem: The $? variable is always 0 after that statement. (If,
>> on the other hand, I separate the declaration and the
>> definition of the variable as shown in the example below, the
>> $? variable is really set to the exit status of the external
>> tool.)
> 
> The exit status is the status of the command you run: local.  In the
> absence of a command, when there are only assignment statements, the
> exit status can be the exit status of a command substitution:

I have been bitten by this a few times. Since then I never, ever write this:
  local foo=...

But always:
  local foo; foo=...


It's overkill, but simple. I tend to even forget why I am doing this,
which is nice: now I can focus on the real problems to solve.

Re: Return status of command substitution with $(...) "gets lost"

[Chet Ramey]

On 3/4/10 2:36 AM, Ettelbrueck, Heiko wrote:

> Important detail: The local variable is declared and defined in
> the same step with "local VARNAME=$(do something)".
> 
> Problem: The $? variable is always 0 after that statement. (If,
> on the other hand, I separate the declaration and the
> definition of the variable as shown in the example below, the
> $? variable is really set to the exit status of the external
> tool.)

The exit status is the status of the command you run: local.  In the
absence of a command, when there are only assignment statements, the
exit status can be the exit status of a command substitution:

"If there is no command name, but the command contained a command
substitution, the command shall complete with the exit status of the last
command substitution performed. Otherwise, the command shall complete with
a zero exit status."

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Return status of command substitution with $(...) "gets lost"

[Chris F.A. Johnson]

On Thu, 4 Mar 2010, Ettelbrueck, Heiko wrote:

> Configuration Information [Automatically generated, do not change]:
> Machine: x86_64
> OS: linux-gnu
> Compiler: gcc
> Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
> -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu'
> -DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale'
> -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib
>   -g -O2
> uname output: Linux wdfd00221495a 2.6.27.42-0.1-xen #1 SMP 2010-01-06
> 16:07:25 +0100 x86_64 x86_64 x86_64 GNU/Linux
> Machine Type: x86_64-unknown-linux-gnu
> 
> Bash Version: 4.1
> Patch Level: 0
> Release Status: release
> 
> Description:
> There is a function which runs some external tool. Both the
> tool's output and its exit status are relevant for further
> execution of the bash script.
> -> The tool's is stored into a (new) local variable, which is
>then processed further.
> -> The tool's exit status is retrieved from the $? variable.
> 
> Important detail: The local variable is declared and defined in
> the same step with "local VARNAME=$(do something)".
> 
> Problem: The $? variable is always 0 after that statement.

   The return status is that of the command, local.

-- 
   Chris F.A. Johnson  
   ===
   Author:
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
   Pro Bash Programming: Scripting the GNU/Linux Shell (2009, Apress)

Return status of command substitution with $(...) "gets lost"

[Ettelbrueck, Heiko]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-unknown-linux-gnu'
-DCONF_VENDOR='unknown' -DLOCALEDIR='/usr/local/share/locale'
-DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include -I./lib
  -g -O2
uname output: Linux wdfd00221495a 2.6.27.42-0.1-xen #1 SMP 2010-01-06
16:07:25 +0100 x86_64 x86_64 x86_64 GNU/Linux
Machine Type: x86_64-unknown-linux-gnu

Bash Version: 4.1
Patch Level: 0
Release Status: release

Description:
There is a function which runs some external tool. Both the
tool's output and its exit status are relevant for further
execution of the bash script.
-> The tool's is stored into a (new) local variable, which is
   then processed further.
-> The tool's exit status is retrieved from the $? variable.

Important detail: The local variable is declared and defined in
the same step with "local VARNAME=$(do something)".

Problem: The $? variable is always 0 after that statement. (If,
on the other hand, I separate the declaration and the
definition of the variable as shown in the example below, the
$? variable is really set to the exit status of the external
tool.)

I realized this on version 4.1 (as this bug is reported for
now) as well as on older versions (4.0, 3.x), if I remember it
correctly.

Please check whether that behavior is intended or whether it's
really a bug.

Repeat-By:
The following script contains two functions, the first one
showing the scenario where the issue occurs (testfunc_fail),
the second one showing the working scenario (testfunc_ok).

testfunc_fail() {
  local OUTPUT=$(touch /does/not/exist 2>&1)
  echo "testfunc_fail() returned $? with message '$OUTPUT'."
}

testfunc_ok() {
  local OUTPUT
  OUTPUT=$(touch /does/not/exist 2>&1)
  echo "testfunc_ok() returned $? with message '$OUTPUT'."
}

testfunc_fail
testfunc_ok