Re: $? in a simple command with no command name
Date:Wed, 1 Sep 2021 19:04:12 +0100 From:Harald van Dijk Message-ID: <837d3b5b-ac61-98eb-2741-d667a78e2...@gigawatt.nl> | $? is defined as "Expands to the decimal exit status of the most recent | pipeline (see Pipelines)." I suspect this is just a slightly more precisely specified version of what the original Bourne shell (7th edition) man page said of $? .B ? The value returned by the last executed command in decimal. "The value returned" has turned into "exit status" which is clearly what is intended (not something sent to stdout or whatever), and "command" has been replaced by "pipeline" (of which a command, as specified in the 7th edition man page, is just the degenerate case), which is correct. The 7th edition man page says nothing about the exit status of a command substitution, and nothing about commands without command words, or what exit status they might have. It also does not say that command substitutions are executed in a subshell environment. So one could read its man page as implying that $? is set whenever a command terminates, including commands for command substititions (and since those are, and always were, actually executed in a subshell environment, the final status of that environment instead of the individual commands (or pipelines) that form the command substitution). Further, we now know from what has been reported here in other messages that that was how the implementation worked (I assume, that whenever it did a wait(2) for a child, it set $? to the exit status (or signal number modified so that can be detected) (as well as generating the exit status from the built-in commands). It seems likely that when this was all specified for the older POSIX/SUS standards, the wording for $? was taken from the sh man page, corrected as much as was known it should be, and simply then forgotten. That is, there was probably never any intent to explicitly say that $? can only be set when a pipeline in the current execution environment terminates (and is waited upon), that most probably simply is the way things ended up written. Now, because of that, we can no longer reasonably say that $? is set as each command substitution is performed, and then (normally) overridden by the exit status of the command (if there is one), as the current wording, with its (probably unintended) implications has been around for too long now - even if it didn't match shells of the time. But we can say that in the presence of command substitutions, when the value of $? used in the same command line is unspecified (used elsewhere there's no issue). kre
Re: $? in a simple command with no command name
On 9/1/21 4:59 PM, (Joerg Schilling) wrote: "Chet Ramey via austin-group-l at The Open Group" wrote: Given the following: (exit 42) a=$? b=`false` b=$? echo $? $a $b Bash prints 1 42 1. The original (v7) bourne shell and the rest of the research line through v9 prints 1 1 (b is set to the empty string). That implies that it executes the assignment statements in reverse order, in addition to carrying $? through the sequence of assignments. You are right, the original Bourne Shell for unknown reasons did evaluate a series of shell variable assignments in reverse order. That was changed in ksh88 and in bosh. The SVR4.2 shell prints 1 42 1. I imagine the rest of the SVR4 line sh is the same. Something called SVR4.2 does not really exist. It was a minor change compared to SvR4 announced by Novell short before they sold the Copyright to SCO. > I know of no customers for SVR4.2... even SCO seems to only used it internally in their project Monterey that was abandoned by IBM. And yet it existed as a product. Univel probably had Unixware customers they didn't tell you about. You can find it for download if you look for it, I suspect. There have been major changes in the Bourne Shell for SvR4, but the $? was not touched. So you are mistaken. Sure, I didn't have SVR4 to test against when I wrote that. The important thing to know here is that the Bourne Shell has some checkpoints that update the intermediate value of $?. Since that changed in ksh88 and since POSIX requires a different behavior compared to the Bourne Shell, I modified one checkpoint in bosh to let it match POSIX. Interesting, since ksh88 (Solaris 10 11/16/88i) and ksh93 (93u+ 2012-08-01) both print 1 42 1 Odd that POSIX would specify something different, isn't it? (exit 42); a=$? b=`false` b=$?; echo $? $a $b prints 1 42 42 in bosh and 1 1 in the SvR4 Bourne Shell. It echoes 255 255 with the Solaris 10 /bin/sh (b is again null). It looks like /bin/false exits with status 255, the Solaris 10 sh still performs the assignments in reverse order, and the Solaris 10 version of the SVR4 sh sets $? from the result of each command substitution. In any case, kre's point stands: the original Bourne shell (and, for that matter, the POSIX base implementation) set $? as each command substitution finishes. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: $? in a simple command with no command name
Scott Lurndal wrote: > On Wed, Sep 01, 2021 at 10:59:46PM +0200, Joerg Schilling via austin-group-l > at The Open Group wrote: > > > > Something called SVR4.2 does not really exist. It was a minor change > > compared > > to SvR4 announced by Novell short before they sold the Copyright to SCO. > > SVR4.2 ES/MP became Unixware 2.01. Sure? I had such a machine for a while because I enhanced the cdrom filesystem driver to support Joliet and ISO-9660:1999, but I did not realize that the shell was changed. Well, this of course is a change that needs testing to discover. Jörg -- EMail:jo...@schily.net Jörg Schilling D-13353 Berlin Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/
Re: $? in a simple command with no command name
On Wed, Sep 01, 2021 at 10:59:46PM +0200, Joerg Schilling via austin-group-l at The Open Group wrote: > > Something called SVR4.2 does not really exist. It was a minor change compared > to SvR4 announced by Novell short before they sold the Copyright to SCO. SVR4.2 ES/MP became Unixware 2.01. They both were fairly widely used back in the day. scott
Re: $? in a simple command with no command name
"Chet Ramey via austin-group-l at The Open Group" wrote: > Given the following: > > (exit 42) > a=$? b=`false` b=$? > > echo $? $a $b > > Bash prints 1 42 1. > > The original (v7) bourne shell and the rest of the research line through v9 > prints 1 1 (b is set to the empty string). That implies that it executes > the assignment statements in reverse order, in addition to carrying $? > through the sequence of assignments. You are right, the original Bourne Shell for unknown reasons did evaluate a series of shell variable assignments in reverse order. That was changed in ksh88 and in bosh. > The SVR4.2 shell prints 1 42 1. I imagine the rest of the SVR4 line sh is > the same. Something called SVR4.2 does not really exist. It was a minor change compared to SvR4 announced by Novell short before they sold the Copyright to SCO. I know of no customers for SVR4.2... even SCO seems to only used it internally in their project Monterey that was abandoned by IBM. There have been major changes in the Bourne Shell for SvR4, but the $? was not touched. So you are mistaken. The important thing to know here is that the Bourne Shell has some checkpoints that update the intermediate value of $?. Since that changed in ksh88 and since POSIX requires a different behavior compared to the Bourne Shell, I modified one checkpoint in bosh to let it match POSIX. (exit 42); a=$? b=`false` b=$?; echo $? $a $b prints 1 42 42 in bosh and 1 1 in the SvR4 Bourne Shell. Jörg -- EMail:jo...@schily.net Jörg Schilling D-13353 Berlin Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sourceforge.net/projects/schilytools/files/
Re: $? in a simple command with no command name
On 9/1/21 2:23 PM, Robert Elz via austin-group-l at The Open Group wrote: > Date:Wed, 1 Sep 2021 19:04:12 +0100 > From:Harald van Dijk > Message-ID: <837d3b5b-ac61-98eb-2741-d667a78e2...@gigawatt.nl> > > | Is there any statement that overrides the general definition to > | explicitly make this unspecified? If not, the general definition applies > | and $? must expand to 0 both times it appears on line 2. > > Perhaps as currently written that's correct, but if so, the standard > probably needs to be updated, as it is fairly clear that shells which > set $? as each command substitution finishes have always existed (in > fact, that might have been what the original Bourne shell did, I haven't > checked) and the standard should allow for that. Given the following: (exit 42) a=$? b=`false` b=$? echo $? $a $b Bash prints 1 42 1. The original (v7) bourne shell and the rest of the research line through v9 prints 1 1 (b is set to the empty string). That implies that it executes the assignment statements in reverse order, in addition to carrying $? through the sequence of assignments. The SVR4.2 shell prints 1 42 1. I imagine the rest of the SVR4 line sh is the same. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: $? in a simple command with no command name
Date:Wed, 1 Sep 2021 19:04:12 +0100 From:Harald van Dijk Message-ID: <837d3b5b-ac61-98eb-2741-d667a78e2...@gigawatt.nl> | Is there any statement that overrides the general definition to | explicitly make this unspecified? If not, the general definition applies | and $? must expand to 0 both times it appears on line 2. Perhaps as currently written that's correct, but if so, the standard probably needs to be updated, as it is fairly clear that shells which set $? as each command substitution finishes have always existed (in fact, that might have been what the original Bourne shell did, I haven't checked) and the standard should allow for that. kre
Re: $? in a simple command with no command name
On 01/09/2021 18:48, Robert Elz via austin-group-l at The Open Group wrote: Date:Wed, 1 Sep 2021 16:38:00 +0300 From:"=?UTF-8?B?T8SfdXo=?= via austin-group-l at The Open Group" Message-ID: | true | a=$? b=`exit 1` b=$? >`echo /dev/null; exit 2` | echo $? $a $b | Now, I wonder, what did I miss? That $? (the exit status) is defined only when a command completes. What it is in the middle of a command execution isn't specified. $? is defined as "Expands to the decimal exit status of the most recent pipeline (see Pipelines)." During the a=$? and b=$? assignments, the most recent pipeline in that shell execution environment was 'true' at line 1. Is there any statement that overrides the general definition to explicitly make this unspecified? If not, the general definition applies and $? must expand to 0 both times it appears on line 2. Cheers, Harald van Dijk
Re: $? in a simple command with no command name
1 Eylül 2021 Çarşamba tarihinde Robert Elz yazdı: > Date:Wed, 1 Sep 2021 16:38:00 +0300 > From:"=?UTF-8?B?T8SfdXo=?= via austin-group-l at The Open > Group" > Message-ID: qvsysq5nfzfxetr7yeq...@mail.gmail.com> > > | true > | a=$? b=`exit 1` b=$? >`echo /dev/null; exit 2` > | echo $? $a $b > > | Now, I wonder, what did I miss? > > That $? (the exit status) is defined only when a command completes. > What it is in the middle of a command execution isn't specified. > > | Where does it say in the standard that > | the value of parameter `?' is affected by redirections, > > It isn't the redirection, but the command substitution that generates > the file name for the redirection. > > | and that its > | value depends on assignments that appear earlier in the same command? > > All this stuff is just unspecified, the order in which all of this is > carried out, and precisely when $? gets updated, isn't specified, if you > want to achieve meaningful results, you need to split the one > multi-assignemnt > with redirect command into multiple commands. > > You get 1 0 0 from a shell which only sets $? when a command completes > (and which does the redirect before the arg expansions). > > Getting 2 for $? comes because it all depends which of the two command > substitutions is executed first (the last one executed provides $?), which > depends upon whether the redirect happens first or the arg expansions, and > for a command with no actual command word, that's unspecified > (implementations > are allowed to do it either way - which the standard allows, as as you see, > different implementations do it different ways, and users need to be aware > of that). > > Getting a non-zero value for a or b just means that $? is being internally > updated when a value for it becomes available, rather than only when a > command completes, about which as best I remember, nothing is said in the > standard at all. > Oh, okay then. Thanks > kre > > -- Oğuz
Re: $? in a simple command with no command name
Date:Wed, 1 Sep 2021 16:38:00 +0300 From:"=?UTF-8?B?T8SfdXo=?= via austin-group-l at The Open Group" Message-ID: | true | a=$? b=`exit 1` b=$? >`echo /dev/null; exit 2` | echo $? $a $b | Now, I wonder, what did I miss? That $? (the exit status) is defined only when a command completes. What it is in the middle of a command execution isn't specified. | Where does it say in the standard that | the value of parameter `?' is affected by redirections, It isn't the redirection, but the command substitution that generates the file name for the redirection. | and that its | value depends on assignments that appear earlier in the same command? All this stuff is just unspecified, the order in which all of this is carried out, and precisely when $? gets updated, isn't specified, if you want to achieve meaningful results, you need to split the one multi-assignemnt with redirect command into multiple commands. You get 1 0 0 from a shell which only sets $? when a command completes (and which does the redirect before the arg expansions). Getting 2 for $? comes because it all depends which of the two command substitutions is executed first (the last one executed provides $?), which depends upon whether the redirect happens first or the arg expansions, and for a command with no actual command word, that's unspecified (implementations are allowed to do it either way - which the standard allows, as as you see, different implementations do it different ways, and users need to be aware of that). Getting a non-zero value for a or b just means that $? is being internally updated when a value for it becomes available, rather than only when a command completes, about which as best I remember, nothing is said in the standard at all. kre
Re: $? in a simple command with no command name
On 01/09/2021 17:22, Geoff Clare via austin-group-l at The Open Group wrote: Harald van Dijk wrote, on 01 Sep 2021: The second problem is the redirection. Based on the above, command substitutions in a redirection are supposed to affect the exit status just as any other command substitution, but the standard says: 3. Redirections shall be performed as described in Redirection. 4. Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value. This means the command substitution in your redirection is supposed to be performed prior to that in the variable assignment. After the list, the standard says: In the preceding list, the order of steps 3 and 4 may be reversed if no command name results from step 2 or if the command name matches the name of a special built-in utility Ah, thank you. That means the outputs "1 0 0" and "2 0 0" are both equally correct. I assume the outputs "1 2 1" and "2 0 1" do indicate shell bugs. Cheers, Harald van Dijk
Re: $? in a simple command with no command name
Harald van Dijk wrote, on 01 Sep 2021: > > The second problem is the redirection. Based on the above, command > substitutions in a redirection are supposed to affect the exit status just > as any other command substitution, but the standard says: > > 3. Redirections shall be performed as described in Redirection. > > 4. Each variable assignment shall be expanded for tilde expansion, > parameter expansion, command substitution, arithmetic expansion, > and quote removal prior to assigning the value. > > This means the command substitution in your redirection is supposed to be > performed prior to that in the variable assignment. After the list, the standard says: In the preceding list, the order of steps 3 and 4 may be reversed if no command name results from step 2 or if the command name matches the name of a special built-in utility -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: $? in a simple command with no command name
Hi, On 01/09/2021 14:38, Oğuz via austin-group-l at The Open Group wrote: Consider the following: true a=$? b=`exit 1` b=$? >`echo /dev/null; exit 2` echo $? $a $b Having read the relevant sections of the standard a couple times, I would expect the output to be `1 0 0'; I agree. but that's not the case with most shells. Below are what each shell I have on my computer output for that script. ? a b bash2 0 1 bosh2 0 0 dash1 0 0 ksh 2 0 1 mksh1 0 0 yash2 0 0 zsh 1 2 1 Now, I wonder, what did I miss? Where does it say in the standard that the value of parameter `?' is affected by redirections, and that its value depends on assignments that appear earlier in the same command? There are two problems here. The first is the value of $? during expansion. The standard says: If there is no command name, but the command contained a command substitution, the command shall complete with the exit status of the last command substitution performed. Otherwise, the command shall complete with a zero exit status. Some shells implement this by modifying $? as soon as a command substitution finishes. As all command substitutions are processed in order, the end result is that $? is based on the last command substitution performed. However, there is nothing that allows $? to be modified earlier, so this looks like a bug (#1) in those shells. The second problem is the redirection. Based on the above, command substitutions in a redirection are supposed to affect the exit status just as any other command substitution, but the standard says: 3. Redirections shall be performed as described in Redirection. 4. Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value. This means the command substitution in your redirection is supposed to be performed prior to that in the variable assignment. Some shells do not do this in the specified order, which looks like a bug (#2) in those shells. This can be confirmed by running under 'set -x'. Bug #1 leads to the output "1 2 1". Bug #2 leads to the output "2 0 0". Bug #1 & bug #2 combined leads to the output "2 0 1". Cheers, Harald van Dijk
$? in a simple command with no command name
Consider the following: true a=$? b=`exit 1` b=$? >`echo /dev/null; exit 2` echo $? $a $b Having read the relevant sections of the standard a couple times, I would expect the output to be `1 0 0'; but that's not the case with most shells. Below are what each shell I have on my computer output for that script. ? a b bash2 0 1 bosh2 0 0 dash1 0 0 ksh 2 0 1 mksh1 0 0 yash2 0 0 zsh 1 2 1 Now, I wonder, what did I miss? Where does it say in the standard that the value of parameter `?' is affected by redirections, and that its value depends on assignments that appear earlier in the same command? Regards, Oğuz
What does the standard say about whether assignments are visible for subsequent expansions in a simple command without a command name?
Hello, the current shell standard says in Section 2.9.1: --- If no command name results, variable assignments shall affect the current execution environment. If the command name is not a special built-in utility or function, the variable assignments (...) shall not affect the current execution environment (...) In this case it is unspecified: Whether or not the assignments are visible for subsequent expansions in step 4 --- As you can see, if we do have a command, the standard mentions that it is unspecified if the assignments are subsequently visible. So one can say, it is needed to discuss the notion of subsequent visibility. Yet if we don't have a command, the standard does not discuss that at all... One could perhaps interpret that to mean, it is unspecified in that case also. That would mean, there is no need to discuss the notion of subsequent visibility. Thus we apparently have a contradiction - one one hand there is a need to discuss some X, on the other, there is not. As an aside, for dash such assignments are not visible: $ A=; A=a B=$A; echo $B $ but for bash, they are: ~>A=; A=a B=$A; echo $B a ~> Questions: 1. Is that apparent contradiction a bug in the standard that should be fixed, either by adding the visibility clause for both cases, or deleting it for both cases? (in that case, I will make a report). 2. In the larger sense, if the standard does not specify some behaviour that a shell must implement, does that automatically mean it is "unspecified" or "undefined", in the sense of Base Definitions, Section 1.5? 3. In this specific case, is it unspecified what the behaviour is? Or is it specified (which would mean dash or bash have a bug), and what is it? Thank you Mark
