Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
2020-04-20 1:42 Robert Elz : > Probably not, bosh is derived from that shell (more or less) and it is > also A > > [...] > > So are the FreeBSD and NetBSD shells (which is not surprising, as like > dash, they're descendants of ash). > > You can also add zsh to A: Thank you for the information. I have also tested ksh93, pdksh and oksh (OpenBSD KornShell) on FreeBSD. `ksh93' is (B), and `pdksh' and `oksh' do not implement the special treatment in trap handlers. Here is the current list: (A) zsh (zsh-5.7.1, zsh-5.6.2) ash variants (dash-0.5.10.2, busybox-1.28.3, FreeBSD sh, NetBSD sh), Bourne sh variants (heirloom-sh-050706-4, bosh-2020/04/18) (B) bash-4.4, gwsh, ksh variants (ksh-2020.0.0, ksh93.u_1, mksh-R57, mksh-R56) (C) yash-2.49 (D) none (Not Implemented) bash-4.3, pdksh-5.2.14.2, oksh-6.6.1, posh-0.13.2 I think now it is rather clear that the current wording of POSIX is somehow ambiguous and there can be different interpretations. In fact, there is a split, (A) vs (B), in shell implementations. >> 2020-04-19 23:00 Harald van Dijk : >>> It does still mean that anyone writing a function needs to beware that >>> exit and exit $? do not have the same effect, if there is any >>> possibility that the function will be invoked from a trap action. I >>> suspect most people writing functions will not be aware of that. Yes, that is the problem. Actually, the original problem is that I just want to perform `eval "$PROMPT_COMMAND"` in a trap action in Bash script where `$PROMPT_COMMAND' is provided by users. If everything is under the control, I can just always write `return $?', but the commands in `$PROMPT_COMMAND' are specified by users who are unlikely to care about this problem. This specific case is just my personal one, but I think the behavior (B) can possibly cause similar problems in other shell scripts in general. >> 2020-04-19 20:51 Robert Elz : >>> [...] >>> >>> The end result, unless we can get agreement that some >>> implementations are buggy, and will be fixed (which given the >>> split seems an unlikely outcome) is likely to simply be that all >>> of this simply becomes unspecified (or perhaps we could hope, >>> implementation defined) which will mean even more cases where it >>> becomes more difficult to write portable reliable code. >>> >>> kre >> 2020-04-19 23:00 Harald van Dijk : >>> True, and if the intent is that exit and return behave differently >>> and the standard is updated to clearly require that, I have no >>> problem changing the shell back to the prior behaviour. According to these comments, it seems like a deadlock. The standard will not change until shells change their behavior to match with each other, and shells will not change their behavior until the standard is clarified. Actually I can understand both sides, but I think we can find a point of compromise. There are already different implementations so it is already difficult to write portable and reliable code. Even some shells such as `pdksh', `oksh' and `posh' do not implement the special behavior of `return' in trap actions at all, so the standard does not describe the current situation properly. Thus I think the side effect of making it unspecified is limited. Maybe we can first let it be unspecified and then wait to see whether the shells will switch the behavior from the literal reading (B) to more sensible interpretation (A) or not. - It might be difficult to change the behavior (B) of `ksh' and `mksh' because its behavior is unchanged at least since 1993. - While, `bash' implemented the behavior (B) in bash-4.4 which is relatively recently in its long history, so maybe we can hope for a change. - Harald: The remaining shell with (B) in the list is `gwsh'. For example, if the standard changes its description to `unspecified' (instead of clearly requiring (A) or other interpretation), do you think you have a chance to change the behavior back? - I think `yash' with the interpretation (C) will follow the standard anytime if the standard clarifies the intended behavior because `yash' aims to strictly support POSIX. Thank you. -- Koichi
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
Date:Sun, 19 Apr 2020 22:02:49 +0900
From:Koichi Murase
Message-ID:
| Thank you for pointing out this. After reading your comment, I was
| interested in the behavior of the Bourne shell, so I have tried with
| the Heirloom Bourne shell (which is a port of the SVR4 Bourne shell
| according to its page) in AUR. Actually, the Heirloom Bourne shell
| turns out to follow the interpretation (A) [ although it might be
| possible that the behavior is changed by the port ].
Probably not, bosh is derived from that shell (more or less) and it is
also A
bosh $ $SHELL /tmp/TTT
In trap argument: last command preceding the trap action
In direct function call: last command in the trap action
In indirect function call: last command in the trap action
bosh $ $SHELL -c 'echo ${.sh.version}'
version bosh 2020/04/10 a+ (amd64-unknown-netbsd8.99.30)
(it is really version 2020/04/18 without the version string being updated).
So are the FreeBSD and NetBSD shells (which is not surprising, as like
dash, they're descendants of ash).
You can also add zsh to A:
zsh $ $SHELL /tmp/TTT
In trap argument: last command preceding the trap action
In direct function call: last command in the trap action
In indirect function call: last command in the trap action
zsh $ echo $SHELL
/usr/pkg/bin/zsh
zsh $ $SHELL -c 'echo ${ZSH_VERSION}'
5.6.2
(/tmp/TTT is your script - but with a newline removed. It might have just
been an e-mail artifact, and was in the middle of one of the messages).
| Thank you for suggesting another possibility. Let me call the above
| one the interpretation (D).
| But there was actually no implementation that follows (D)
I am not all that surprised, it would be a rather bizarre interpretation.
kre
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
On 19/04/2020 14:32, Koichi Murase wrote:
2020-04-19 21:55 Harald van Dijk :
My reading was that interpretation B must be what is intended, which
is why I had modified my shell, a fork of dash, to change dash's A
behaviour to B in late 2018.
Thank you. Is your shell this https://github.com/hvdijk/gwsh ? I
tried gwsh-0.5.9.1. Here is the updated list:
Yes, that is my shell. Pay no attention to that version number, that is
dash's version number that I have not updated because I never did a
proper release yet. :)
(A) `zsh', `dash', `busybox', `heirloom-sh'
(B) `bash-4.4', `mksh', `ksh', `gwsh'
(C) `yash'
(D) none
Not Implemented: `bash-4.3', `posh'
My reasoning for that is that the description of the return
commanhd ("When return is executed in a trap action, the last
command is considered to be the command that executed immediately
preceding the trap action.") is identical to that of the exit
command ("When exit is executed in a trap action, the last command
is considered to be the command that executed immediately preceding
the trap action.")
Thank you. This is a good point. Maybe this is the origin of the
current wording.
and for the exit command, almost all shells, including dash, are in
agreement that it applies even when the exit command is invoked
indirectly. [...]
It is reasonable that indirect `exit' in function calls in trap
handlers are affected because it actually brings the completion of
trap action which is more consistent with the following description of
`trap'. While, the trap action will not be completed by indirect
`return', so it is not surprising that the behavior can be different
between `exit' and `return'.
True, and if the intent is that exit and return behave differently and
the standard is updated to clearly require that, I have no problem
changing the shell back to the prior behaviour.
It does still mean that anyone writing a function needs to beware that
exit and exit $? do not have the same effect, if there is any
possibility that the function will be invoked from a trap action. I
suspect most people writing functions will not be aware of that.
Another question is what the behaviour should be in a subshell. That is:
f() { false; return; } # or exit
trap '(f; echo $?)' EXIT
Is the return or exit here still considered to execute within a trap
action, or does the fact that subshells reset traps mean that whatever
was executing in the parent shell can no longer be considered a trap
action within the subshell?
Cheers,
Harald van Dijk
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
2020-04-19 21:55 Harald van Dijk :
> My reading was that interpretation B must be what is intended, which
> is why I had modified my shell, a fork of dash, to change dash's A
> behaviour to B in late 2018.
Thank you. Is your shell this https://github.com/hvdijk/gwsh ? I
tried gwsh-0.5.9.1. Here is the updated list:
(A) `zsh', `dash', `busybox', `heirloom-sh'
(B) `bash-4.4', `mksh', `ksh', `gwsh'
(C) `yash'
(D) none
Not Implemented: `bash-4.3', `posh'
> My reasoning for that is that the description of the return
> commanhd ("When return is executed in a trap action, the last
> command is considered to be the command that executed immediately
> preceding the trap action.") is identical to that of the exit
> command ("When exit is executed in a trap action, the last command
> is considered to be the command that executed immediately preceding
> the trap action.")
Thank you. This is a good point. Maybe this is the origin of the
current wording.
> and for the exit command, almost all shells, including dash, are in
> agreement that it applies even when the exit command is invoked
> indirectly. [...]
It is reasonable that indirect `exit' in function calls in trap
handlers are affected because it actually brings the completion of
trap action which is more consistent with the following description of
`trap'. While, the trap action will not be completed by indirect
`return', so it is not surprising that the behavior can be different
between `exit' and `return'.
[XCU 2.14 Special Built-In Utilities - trap - DESCRIPTION/paragraph 2]
[...] The value of "$?" after the trap action completes shall be
the value it had before trap was invoked.
--
Koichi
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
2020-04-19 20:51 Robert Elz :
> Before I make any other comments, I should point out that quite a
> bit of what is in the shell standard (and the rest of POSIX, and for
> that matter, many other standards) isn't there because there's a
> good reason for it, but just because the original shells implemented
> it that way, often for expediency purposes (the original Bourne
> shell had to run in the 64KB address space (code, data, and stack)
> of a PDP-11).
Thank you for pointing out this. After reading your comment, I was
interested in the behavior of the Bourne shell, so I have tried with
the Heirloom Bourne shell (which is a port of the SVR4 Bourne shell
according to its page) in AUR. Actually, the Heirloom Bourne shell
turns out to follow the interpretation (A) [ although it might be
possible that the behavior is changed by the port ].
2020-04-19 20:51 Robert Elz :
> Note that another intrepretation that you didn't list is that this
> magic return applies to return in functions called directly from the
> trap action string (which are more likely to be functions written
> expressly with the intent of being called this way) but not to
> functions called indirectly.
Thank you for suggesting another possibility. Let me call the above
one the interpretation (D). I have updated the script to distinguish
(D) from (B):
setexit() { return "$1"; }
invoke() { kill -USR1 $$; return 222; }
trap 'setexit 111; return' USR1
invoke
case $? in
0) echo 'In trap argument: last command preceding the trap action' ;;
111) echo 'In trap argument: last command in the trap action' ;;
222) echo 'In trap argument: (failed to exit the function)' ;;
*) echo 'In trap argument: (unexpected)' ;;
esac
stat=99
handler() { setexit 111; return; }
trap 'handler; stat=$?; return' USR1
invoke
case $stat in
0) echo 'In direct function call: last command preceding the trap action' ;;
111) echo 'In direct function call: last command in the trap action' ;;
*) echo 'In direct function call: (unexpected)' ;;
esac
stat=99
utility2() { setexit 111; return; }
handler2() { utility2; stat=$?; }
trap 'handler2' USR1
invoke
case $stat in
0) echo 'In indirect function call: last command preceding the
trap action' ;;
111) echo 'In indirect function call: last command in the trap action' ;;
*) echo 'In indirect function call: (unexpected)' ;;
esac
But there was actually no implementation that follows (D) among the
shells that I tested so far. Here is the updated list:
(A) `zsh', `dash', `busybox', `heirloom-sh'
(B) `bash-4.4', `mksh', `ksh'
(C) `yash'
(D) none
Not Implemented: `bash-4.3', `posh'
--
Koichi
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
Date:Sun, 19 Apr 2020 17:12:42 +0900
From:Koichi Murase
Message-ID:
| Currently, I do not see any rationale for the behavior (B) or (C).
Before I make any other comments, I should point out that quite a bit
of what is in the shell standard (and the rest of POSIX, and for that
matter, many other standards) isn't there because there's a good reason
for it, but just because the original shells implemented it that way,
often for expediency purposes (the original Bourne shell had to run in
the 64KB address space (code, data, and stack) of a PDP-11).
So that's not really a meaningfull question to ask in this context,
the standard might very well require something totally irrational,
simply because that is how it has been implemented.
If something is implemented a particular way, applications (scripts)
may come to depend upon it, so changing it later can be difficult.
All that aside, I agree that the wording in the standard is ambiguous.
My first reading (my immediate impression) is that your interpretation A
is what is intended, and only a return command executing directly from the
trap action itself, and not one in a function that happens to be called,
should be affected. (And of course, the former is only specified if the
trap occurs while running a function or dot script).
That is, with code like
long_func() {
t=$(trap -p)
trap 'eval "$T"; return' INT
while sleep 10
do
printf .
done
}
That is, it should not apply in functions called from a trap action,
for them, return should act just like it would in a function called
from any other source.
Note that another intrepretation that you didn't list is that this magic
return applies to return in functions called directly from the trap action
string (which are more likely to be functions written expressly with the
intent of being called this way) but not to functions called indirectly.
Next thing to note is that all of this wording is to be changed in the
next version of the standard - there's no longer any "exit status of the
last command" stuff for return (or exit), rather these are to be specified
to return the value of $? when no arg is given.
When we do that, all we need to do is specify how traps affect $?, which
should be (IMO), on entry, $? is the status of the last command (ie: starting
a trap action changes nothing wrt $? - it holds the same value it would have
had if there were no trap action called, and the next command simply started.
On return from a trap action to the command that would have been executed
next, had no trap action intervened, the exit status should be retained at
the same value it had before the trap action was executed (the trap should
not alter $?).
If a trap action causes the control flow to be altered (that is, by
executing a return/exit/break/continue outside of a context that is local
to the trap action) then the exit status should be whatever the trap
action causes to happen (that of the last command executed there).
But that I believe this is the most appropriate specification does not
mean that is what can go into the standard, that depends upon what the
implementations actually do - and as there seems to be a diversity of
implementations (including as to what is the command before the trap,
when the trap is caused by a kill ... and possibly for async signals as
well - some shells seem to use the status of the command before the kill
command as the relevant one (that would be the status of the last command
executed before the signal arrived), even though the standard is clear that
the trap action cannot be run while a foreground command is running. Here
the trap cannot run before the kill command starts (it is that which sends
the signal) and assuming it is run in foreground, that means not until
after the kill command completes, which means $? should be the status from
he kill command. But some shells seem to run the trap action with $? being
the status of the commane before the kill command (which would be the
value of $? when the signal arrived).
The end result, unless we can get agreement that some implementations are
buggy, and will be fixed (which given the split seems an unlikely outcome)
is likely to simply be that all of this simply becomes unspecified (or
perhaps we could hope, implementation defined) which will mean even more
cases where it becomes more difficult to write portable reliable code.
kre
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
19 Nisan 2020 Pazar tarihinde Koichi Murase yazdı: > 2020-04-19 15:21 Oğuz : > > The same document you linked says: > > > > > If the shell is not currently executing a function or dot script, the > > > results are unspecified. > > > > in DESCRIPTION section; it's unspecified what those returns do. > > Thank you for the comment, but I am confused. In both two examples > that I provided, it actually running a function when `kill' is called, > i.e. `kill' is called when `invoke' is executed. Then the trap > handler is invoked while executing the function. > > So was I, obviously overlooked that part. > If that is not the correct interpretation [i.e., the function is not > considered currently executed while the trap handler is processed, and > the description on `return' applies only to `return's appearing in the > function calls inside the trap action], then it is the third > interpretation: > > (C) The `return's in the function-call tree in trap processing are > affected, but the behavior of `return' directly called from the trap > argument is unspecified. > > However, in this case, the same question for the interpretation (B) > also applies to the interpretation (C): > > 2020-04-19 13:32 Koichi Murase : > > > - If the literal interpretation (B) is correct, what is the use case > > > of this behavior, or what is the rationale for this behavior? > > Currently, I do not see any rationale for the behavior (B) or (C). > Does anyone know something which explains the necessity of the > behavior (B) or (C)? > > 2020-04-19 15:21 Oğuz : > > However some shells (dash for instance) print 42. I think it's a > > misreading of the standard. > > It seems to me that (A) is the natural behavior because it is more > consistent with the following description in `trap' section. I can > imagine that the special case for the `return' exit status is added to > make it more consistent with this section: > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V > 3_chap02.html#tag_18_28_03 > > [XCU 2.14 Special Built-In Utilities - trap - DESCRIPTION/paragraph 2] > > > > [...] The value of "$?" after the trap action completes shall be the > value > > it had before trap was invoked. Apparently in trap actions return without an operand is reserved for retaining exit status of the last command preceding the trap action. Thus (B) sounds like the correct interpretation. However this way functions invoked from trap actions have to use return $? to deliver exit status of the last command they executed. I agree that the standard should elaborate on this. Maybe I am wrong, but I cannot stop suspecting that this is a wording > problem that the original intension is (A), but it just reads like (B) > or (C) due to not enough description in the standard. > > -- > Koichi > -- Oğuz
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
2020-04-19 15:21 Oğuz : > The same document you linked says: > > > If the shell is not currently executing a function or dot script, the > > results are unspecified. > > in DESCRIPTION section; it's unspecified what those returns do. Thank you for the comment, but I am confused. In both two examples that I provided, it actually running a function when `kill' is called, i.e. `kill' is called when `invoke' is executed. Then the trap handler is invoked while executing the function. If that is not the correct interpretation [i.e., the function is not considered currently executed while the trap handler is processed, and the description on `return' applies only to `return's appearing in the function calls inside the trap action], then it is the third interpretation: (C) The `return's in the function-call tree in trap processing are affected, but the behavior of `return' directly called from the trap argument is unspecified. However, in this case, the same question for the interpretation (B) also applies to the interpretation (C): 2020-04-19 13:32 Koichi Murase : > > - If the literal interpretation (B) is correct, what is the use case > > of this behavior, or what is the rationale for this behavior? Currently, I do not see any rationale for the behavior (B) or (C). Does anyone know something which explains the necessity of the behavior (B) or (C)? 2020-04-19 15:21 Oğuz : > However some shells (dash for instance) print 42. I think it's a > misreading of the standard. It seems to me that (A) is the natural behavior because it is more consistent with the following description in `trap' section. I can imagine that the special case for the `return' exit status is added to make it more consistent with this section: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_28_03 > [XCU 2.14 Special Built-In Utilities - trap - DESCRIPTION/paragraph 2] > > [...] The value of "$?" after the trap action completes shall be the value > it had before trap was invoked. Maybe I am wrong, but I cannot stop suspecting that this is a wording problem that the original intension is (A), but it just reads like (B) or (C) due to not enough description in the standard. -- Koichi
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
Oh, sorry. It should be
g() { f; return; }
not
g() { return; }
On Sun, Apr 19, 2020 at 9:20 AM Oğuz wrote:
>
> > I have a question on the POSIX behavior of `return' shell builtin
> > without arguments when used in a shell `trap' action.
> > [...]
> > trap 'setexit 111; return' USR1
> > [...]
> > trap 'handler; stat=$?; return' USR1
>
> The same document you linked says:
>
> > If the shell is not currently executing a function or dot script, the
> > results are unspecified.
>
> in DESCRIPTION section; it's unspecified what those returns do.
>
> I got what you're actually asking though. According to the
> standard, application below should print kill's exit status bacause
> it is *the command that executed immediately preceding the trap
> action*.
>
> f() { return 42; }
> g() { return; }
> trap 'g; echo $?' USR1
> kill -USR1 $$
>
> However some shells (dash for instance) print 42. I think it's a
> misreading of the standard.
>
> --
> Oğuz
--
Oğuz
Re: XCU 2.14: Exit status by no-argument `return' in shell trap handlers
> I have a question on the POSIX behavior of `return' shell builtin
> without arguments when used in a shell `trap' action.
> [...]
> trap 'setexit 111; return' USR1
> [...]
> trap 'handler; stat=$?; return' USR1
The same document you linked says:
> If the shell is not currently executing a function or dot script, the
> results are unspecified.
in DESCRIPTION section; it's unspecified what those returns do.
I got what you're actually asking though. According to the
standard, application below should print kill's exit status bacause
it is *the command that executed immediately preceding the trap
action*.
f() { return 42; }
g() { return; }
trap 'g; echo $?' USR1
kill -USR1 $$
However some shells (dash for instance) print 42. I think it's a
misreading of the standard.
--
Oğuz
