Re: Possible bug in bash

[Dale R. Worley]

Greg Wooledge  writes:
> On Fri, May 13, 2022 at 10:36:56PM -0400, Dale R. Worley wrote:
>> Reading your message, I believe that the rule can be stated as follows,
>> and I'd thank you to check it:  && and || have the same precedence, and
>> they both "associate left".  So for example
>> x && yy || zz
>> is equivalent (as a control structure) to
>> { x && yy ;} || zz
>
> Not really.  Let's say you have a bunch of commands strung together like
> this:
>
> a && b || c && d || e && f || g
[ most of the exposition snipped ]
> And so on, until the entire line has been processed.  Each simple command
> in the line is either executed, or not, depending on the current value
> of $? and the operator which precedes it.
>
> That's why this has no equivalence to a regular "if/then/else" command.
> The implementation is just entirely different.

Uh, I didn't say it was equivalent to a 'regular "if/then/else"'
command, I said it was equivalent to

   { { { { { a && b ;} || c ;} && d ;} || e ;} && f ;} || g

which indeed has the same effect as you described in your message.

Dale

Re: Possible bug in bash

[Chet Ramey]

On 1/15/23 7:42 AM, anonymous4feedb...@outlook.com wrote:

For the follow script
alias al=' '
alias foo=bar
al for foo in v
do echo foo=$foo bar=$bar
done
bash (version 5.1.16) prints foo=v bar=, while all other shells I tested (dash, 
ksh, zsh, and yash) all prints foo= bar=v.


That's strange. I get the same results you do for bash, but I tried the
same shells you did and got the same results as bash (for different
reasons, I suspect).

$ cat x4
shopt -s expand_aliases 2>/dev/null
alias al=' '
alias foo=bar
al for foo in v
do echo foo=$foo bar=$bar
done
$ ./bash ./x4
foo=v bar=
$ dash ./x4
foo=v bar=
$ ksh93 ./x4
foo=v bar=
$ yash ./x4
foo=v bar=
$ mksh ./x4
foo=v bar=
$ zsh ./x4
foo=v bar=
$ ./bash --version
GNU bash, version 5.2.15(54)-maint (x86_64-apple-darwin21.6.0)
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.



Apparently bash substitutes foo for bar in line 3 because the previous alias al 
ends with a space. But it is unintuitive that the word after for is checked for 
alias.


This is the opposite of what happens. In default mode, which you're
probably using, bash checks `for' for aliases, and, finding none, stops
checking. The other shells probably *don't* check whether `for' has an
alias, because POSIX says reserved words can't be alias expanded ("However,
reserved words in correct grammatical context shall not be candidates for
alias substitution.") Either way, the `check the next word for alias
expansion' flag gets turned off.


According to the posix standard,
If the value of the alias replacing the word ends in a , the shell shall check 
the next command word for alias substitution; this process shall continue until a word is 
found that is not a valid alias or an alias value does not end in a .




But “command word” is not defined. It is ambiguous whether “for” in this 
context is a command word, or whether tokens other than command word is allowed 
between the first alias and the next command word.


`Command word' refers to the previous paragraph: since alias expansion is
attempted on "a resulting word that is identified to be the command name
word of a simple command," the "next command word" is presumably the word
following that one.



The same is true for case
alias al=' '
alias foo=bar
al case foo in foo) echo foo;; bar) echo bar;; esac


Everybody prints `foo'.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Possible bug in bash

[Chet Ramey]

On 5/13/22 10:36 PM, Dale R. Worley wrote:

Robert Elz  writes:

Note particularly that there is no operator precedence between
&& and || - they are the same (unlike in C for example)


Reading your message, I believe that the rule can be stated as follows,
and I'd thank you to check it:  && and || have the same precedence, and
they both "associate left".  So for example
 x && yy || zz
is equivalent (as a control structure) to
 { x && yy ;} || zz


Yes, this is correct.


--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Possible bug in bash

[Robert Elz]

Date:Fri, 13 May 2022 23:27:50 -0400
From:Greg Wooledge 
Message-ID:  

  | Not really.

What's the difference?

  | Let's say you have a bunch of commands strung together like this:
  |
  | a && b || c && d || e && f || g
  |
  | We start with the shell's command parser pointing to "a", which I'll
  | represent like this:
  |
  | a && b || c && d || e && f || g
  | ^

Or instead
{ { { { { a && b ;} || c ;} && d ;} || e ;} && f ;} || g

We start by executing the LHS of the outer || operator
  ^
which we execute by executing the LHS of the && operator in that
^^^
which we execute by executing the LHS of the || operator in that
  ^
which we execute by executing the LHS of the && operator in that
 ^^
which we execute by executing the LHS of the || operator in that
  ^
which we execute by executing the LHS of the && operator in that
  ^
which is a

  | So, "a" is executed, and this sets the $? special parameter to some value,
  | either 0 or nonzero.

The same for both variants.  Though while $? does get set, I believe it is
more correct to say that the exit status (which $? is defined to fetch) is
set.

  | The parser moves to the right and sees "&& b":

It isn't the parser, this whole thing must have already been parsed before
anything is executed, it is the format that the parser produces (usually a
tree, but it could be executable code for some processor, or pseudo-code for
an interpreter) which is being used here - but this is not much more than a
pedantic detail.

  | a && b || c && d || e && f || g
  |   
  |
  | So it checks $?.  If $? is 0, then b will be executed.

In the alternative case, the shell is executing the a && b sub-expression,
and has just executed a - it does exactly the same to do that as you
described.

  | Otherwise, the parser will keep reading to the right.

Again, not the parser...

  | Next it sees "|| c":
  |
  | a && b || c && d || e && f || g

In our case, we now have an exit status from the { a && b; } group.
If a exited with status 0, then the group's status is the status with
which b exited, otherwise it is a's exit status, and b was never evaluated.
(In either case, the group's status is that of the last command executed
in the group, which here is the (short) and-or list, and its status is that
of whichever of its two commands was the final one executed.)

  | If $? is nonzero (either from "a" or from "b"), then "c" will be executed.

So now we can return one step up our parse tree, where we now
have
{ $? || c ;}
which is not (anything like) real code - I am using $? there to indicate
that the sub-expression that was there has already been evaluated, and we
have its exit code.   In this, with the || operator, if $? is non-zero
(either because a exeited non-zero, or it exited 0, and b exited non-zero)
we evaluate the RHS (c), otherwise (both a and b exited 0) we already have
the result (0) and we are done with this sub-expression, without c being
executed.

  | And so on, until the entire line has been processed.

Exactly.

  | Each simple command in the line is either executed, or not,
  | depending on the current value of $? and the operator which precedes it.

The apparent mechanism appears to be slightly different when we have
only binary && and || operators, and groups, but what actually gets
executed is identical.   That's what makes them equivalent.

  | That's why this has no equivalence to a regular "if/then/else" command.
  | The implementation is just entirely different.

The implementation isn't what matters.   They are specified (defined) to
behave differently, that's what matters.

  | Here's an actual example:
  |
  | unicorn:~$ false && true || true && echo a || false && echo b
  | a
  | b

And the same example in the other format:

jinx$ { { { { false && true ;} || true ;} && echo a ;} || false ;} && echo b
a
b

It takes more chars to write that way, but it is much easier to examine
and determine what is going to happen.

  | In real life, there is no reason to write code like this.  It's horrible
  | and confusing.  Just don't do it.

Agreed.And even in the simple cases, just to reinforce things, don't
write && when what you mean is if.   It isn't cool, it is dangerous.

kre

Re: Possible bug in bash

[Robert Elz]

Date:Fri, 13 May 2022 22:36:56 -0400
From:"Dale R. Worley" 
Message-ID:  <87ilq8hmbb@hobgoblin.ariadne.com>

  | Reading your message, I believe that the rule can be stated as follows,
  | and I'd thank you to check it:

OK

  | && and || have the same precedence, and they both "associate left".

That is correct, I believe ... I sometimes have trouble wrapping
my head around formal mathematical definitions.   But if you are
going to state it that way, you should probably also add that
short circuit evaluation is required, that is, as binary operators,
the RHS is never evaluated if the result is known from the LHS.
(This is not just an optimisation, it is a requirement.)

  | So for example

A better example might be, that for any pipelines a b c d
and where X Y and Z each represents one of the control
operators && or || in any combination
a X b Y c Z d
is equiv to:
{ { a X b; } Y c; } Z d

And just as a hint, far and away the best/safest way to make
use of these operators is when only one of them is used
(any number of times) in one and-or list.  So:

a && b && c && d && e && ...

causes a b c d e ... to be executed, in sequence, until one of them
"fails" (that is, exits with a status that is not zero), after which
none of the rest is executed (or evaluated at all).  The exit status
of the list is (as always in and-or lists) that of the last command
executed, so is 0 here only if every command was executed, and all
exited with status 0.   And:

a || b || c || d || e || ...

causes a b c d e ... to be executed, in sequence, until one of them
"succeeds" (exits with status 0), after which none of the rest is
executed (or evaluated).   The exit status of the list works the
same way, but here will be 0 unless every command in the list
was executed with non-zero status returned every time.

The idiom
command || :
is how one says "I don't care about the exit status of command".

Mixing && and || in the same and-or list takes very careful
thought, even very experienced sh programmers mess that up
more often than you'd imagine.  Much better to use the equiv
form, with the explicit grouping, the:
a X b Y c Z d
form is one and-or list, and if X Y and Z are not all the
same, will often, if you are not very careful, have unanticipated
results.  Write it as:
{ { a X b; } Y c; } Z d
and now there are 3 independant (binary) and-or lists, which is
much easier to reason about.  (It is reasonable to expand any of
those 3 and-or lists to more terms, provided each makes use of only
one of the two possible operators, in the innermost, every operator
is X, etc.)

kre

Bonus extra: A common error is to end a function with something like:
$verbose && echo function finished

That is almost always wrong, as it causes the function to return
non-zero status if verbose=false.  That is, while you are debugging
the script, with verbose=true it all works as intended (perhaps add
"eventually" there) but as soon as you set verbose=false, things change.
If written as the seemingly equivalent:
if $verbose; then echo function finished; fi
we do not get that issue, the function will always return a
0 status if it gets that far (and that is its final command).
Please do not treat this as an endorsement of "echo", using
printf would be better, nor for not quoting var expansions.
It is intended as a reinforcement of the:
if you mean 'if' write 'if' not '&&'
message.

Re: Possible bug in bash

[Greg Wooledge]

On Fri, May 13, 2022 at 10:36:56PM -0400, Dale R. Worley wrote:
> Reading your message, I believe that the rule can be stated as follows,
> and I'd thank you to check it:  && and || have the same precedence, and
> they both "associate left".  So for example
> x && yy || zz
> is equivalent (as a control structure) to
> { x && yy ;} || zz

Not really.  Let's say you have a bunch of commands strung together like
this:

a && b || c && d || e && f || g

We start with the shell's command parser pointing to "a", which I'll
represent like this:

a && b || c && d || e && f || g
^

So, "a" is executed, and this sets the $? special parameter to some value,
either 0 or nonzero.  The parser moves to the right and sees "&& b":

a && b || c && d || e && f || g
  

So it checks $?.  If $? is 0, then b will be executed.  Otherwise, the
parser will keep reading to the right.  Next it sees "|| c":

a && b || c && d || e && f || g
   

If $? is nonzero (either from "a" or from "b"), then "c" will be executed.
Otherwise, the parser keeps reading to the right:

a && b || c && d || e && f || g


And so on, until the entire line has been processed.  Each simple command
in the line is either executed, or not, depending on the current value
of $? and the operator which precedes it.

That's why this has no equivalence to a regular "if/then/else" command.
The implementation is just entirely different.

Here's an actual example:

unicorn:~$ false && true || true && echo a || false && echo b
a
b

The first "false" is executed no matter what; this sets $? to 1.  The
parser looks at "&& true" and skips it because $? is nonzero.  So it
looks at "|| true", and runs that because $? is nonzero.  Now $? is 0.
Next up is "&& echo a", and since $? is 0, bash runs that echo.  This
sets $? to 0 (because the echo succeeded), so "|| false" is skipped.
The parser finally ends on "&& echo b", and since $? is 0, that echo
is also executed.

In real life, there is no reason to write code like this.  It's horrible
and confusing.  Just don't do it.

Re: Possible bug in bash

[Dale R. Worley]

Robert Elz  writes:
> Note particularly that there is no operator precedence between
> && and || - they are the same (unlike in C for example)

Reading your message, I believe that the rule can be stated as follows,
and I'd thank you to check it:  && and || have the same precedence, and
they both "associate left".  So for example
x && yy || zz
is equivalent (as a control structure) to
{ x && yy ;} || zz

Dale

Re: Possible bug in bash

[flyingrhino]

> 
> Have a thorough look around at:
> http://mywiki.wooledge.org/BashPitfalls
> http://mywiki.wooledge.org/BashFAQ
> http://mywiki.wooledge.org/BashGuide
> 
> and you will find a lot of useful knowledge that is
> explained with considerable effort by many people.

Pity I didn't find this page when I searched for solutions before writing this 
post.
Again, thanks for the links - will help vastly in my scripting.


-- 
Rhinos can fly,

It's just a case of mind over matter ...
... And you need a lot of mind to control that much matter ...

Re: Possible bug in bash

[David]

On Fri, 13 May 2022 at 18:18, flyingrhino  wrote:

> Before opening the bug I looked online for if-then-else vs [[ and no
> proper information was available, definitely not to the extent you explain 
> here.

Have a look here:

http://mywiki.wooledge.org/BashPitfalls#cmd1_.26.26_cmd2_.7C.7C_cmd3

> This is very useful and rare knowledge and the effort
> you took to explain this is highly appreciated.

Have a thorough look around at:
http://mywiki.wooledge.org/BashPitfalls
http://mywiki.wooledge.org/BashFAQ
http://mywiki.wooledge.org/BashGuide

and you will find a lot of useful knowledge that is
explained with considerable effort by many people.

Re: Possible bug in bash

[flyingrhino]

Thanks Lawrence for the response.
Between you and Robert I now have a clear understanding on this and I'll go 
back and fix the bug in my code which used this construct.

Ken.


On Fri, 13 May 2022 00:51:49 -0400
Lawrence Velázquez  wrote:

> On Thu, May 12, 2022, at 11:34 PM, flyingrhino wrote:
> > Should the "else" condition after the:  ||  run if the last command in 
> > the:  &&  section exits non zero?  
> 
> Yes.  This behavior is not a bug; ''A && B || C'' is simply not
> equivalent to ''if A then B; else C; fi''.
> 
> https://mywiki.wooledge.org/BashPitfalls#pf22
> 
> > Script:
> >
> >
> > #!/bin/bash
> >
> > [[ "a" == "a" ]] && \
> >  {
> >  echo "equal"
> >  ls x
> >  } || {
> >  echo "* SHOULD NOT DISPLAY 4"
> >  }
> >
> >
> > Result:
> >
> > ./moo.sh
> > equal
> > ls: cannot access 'x': No such file or directory
> > * SHOULD NOT DISPLAY 4  
> 
> This behavior is expected and correct.  There is no bug here.
> 
> > BTW, I've checked other conditions as follows and they look ok:
> >
> > [...]
> >
> > [[ "a" == "a" ]] && \
> >  {
> >  echo "equal"
> >  } || {
> >  echo "* SHOULD NOT DISPLAY 1"
> >  }  
> 
> This would display "* SHOULD NOT DISPLAY 1" if ''echo "equal"''
> failed for some reason.  (Unlikely but possible.)
> 



-- 
Rhinos can fly,

It's just a case of mind over matter ...
... And you need a lot of mind to control that much matter ...

Re: Possible bug in bash

[flyingrhino]

Hi,

Thank you very much for the detailed description of this scenario. 
Before opening the bug I looked online for if-then-else vs [[ and no proper 
information was available, definitely not to the extent you explain here.
This is very useful and rare knowledge and the effort you took to explain this 
is highly appreciated.

Ken.


On Fri, 13 May 2022 12:39:10 +0700 (ICT)
Robert Elz  wrote:

> Not a bug.
> 
> Do not use && || as if they were a replacement for if then else fi
> they aren't. In some simple cases it all works out OK, but not in
> general, as you discovered.   If you mean if x; then y; else z; fi
> then write that,  not x && y || z
> 
> The way and-or lists work, is that the first command (whatever is
> to the left of the first && or || at the the current nesting
> level ... that is any that occur nested inside some other construct
> are not yet seen, but will eventually be used following these
> same rules, if that other construct is evaluated) is executed.
> 
> To this point it is just like an if statement, in that the list
> between the "if" and the corresponding "then" is evaluated.
> 
> Then, forever (until the entire and-or list is consumed, there
> can be many && and || operators, in any order) when the
> operator is reached, if the current exit status is 0, then
> if the operator is && we execute the next command in the and-or
> list, the exit status changes to the result of that command.
> If the operator was || then simply skip to the next && or ||
> operator in this and-or list, the exit status does not change,
> and continue.
> 
> If, when one of those operators is reached the current exit status
> is non-zero, we do just the samd, but in reverse, execute the next
> command (updating the exit status as that happens) if the operator
> is || and skip it leaving the exit status unchanged, if &&
> 
> So, in your x && y || z command, x (the [[ command) is executed,
> exit status is 0 (true) the && is reached, so we go on and
> execute y (the brace group command in your example).  Executing
> a group sets the exit status of the group to the status of
> the last command in the group that was executed.  In your example
> the ls command, which in your case was not zero.
> So, when we get to the || operator, the exit status is not
> zero, which means we go on and execute the next command z (echo
> in your example).
> 
> Note particularly that there is no operator precedence between
> && and || - they are the same (unlike in C for example)
> 
> This is completely different from if x ;  then y ; else z ; fi
> whers what happens in y (if executed) has no bearing at all
> upon whether z is executed - there exactly one of y or z
> gets executed, never both.  Not true of and-or lists.
> 
> The exit status of the whole construct is also different.
> In an and-or list, the exit status of that list is that
> of whichever command (x, y, or z) was executed last.  The
> exit status of the if form is that of whatever command in
> the then or else lists (y or z -- cannot be both) was
> executed, or 0 if neither was (that is, if x exited non-zero
> and there was no else part).  The actual exit status of
> x is never used as the exit status of the if command.
> 
> So if you mean "if" write "if" not && - using an and-or
> list when an if should have been used does not show you
> as being a "cool power programmer, using advanced syntax
> that the boring newbies do not understand" rather it shows
> that you are an incompetant beginner, trying to master
> concepts not yet understood, and making a mess of it.
> 
> Use the proper construct for what is needed, always.
> 
> kre

Re: Possible bug in bash

[Robert Elz]

Not a bug.

Do not use && || as if they were a replacement for if then else fi
they aren't. In some simple cases it all works out OK, but not in
general, as you discovered.   If you mean if x; then y; else z; fi
then write that,  not x && y || z

The way and-or lists work, is that the first command (whatever is
to the left of the first && or || at the the current nesting
level ... that is any that occur nested inside some other construct
are not yet seen, but will eventually be used following these
same rules, if that other construct is evaluated) is executed.

To this point it is just like an if statement, in that the list
between the "if" and the corresponding "then" is evaluated.

Then, forever (until the entire and-or list is consumed, there
can be many && and || operators, in any order) when the
operator is reached, if the current exit status is 0, then
if the operator is && we execute the next command in the and-or
list, the exit status changes to the result of that command.
If the operator was || then simply skip to the next && or ||
operator in this and-or list, the exit status does not change,
and continue.

If, when one of those operators is reached the current exit status
is non-zero, we do just the samd, but in reverse, execute the next
command (updating the exit status as that happens) if the operator
is || and skip it leaving the exit status unchanged, if &&

So, in your x && y || z command, x (the [[ command) is executed,
exit status is 0 (true) the && is reached, so we go on and
execute y (the brace group command in your example).  Executing
a group sets the exit status of the group to the status of
the last command in the group that was executed.  In your example
the ls command, which in your case was not zero.
So, when we get to the || operator, the exit status is not
zero, which means we go on and execute the next command z (echo
in your example).

Note particularly that there is no operator precedence between
&& and || - they are the same (unlike in C for example)

This is completely different from if x ;  then y ; else z ; fi
whers what happens in y (if executed) has no bearing at all
upon whether z is executed - there exactly one of y or z
gets executed, never both.  Not true of and-or lists.

The exit status of the whole construct is also different.
In an and-or list, the exit status of that list is that
of whichever command (x, y, or z) was executed last.  The
exit status of the if form is that of whatever command in
the then or else lists (y or z -- cannot be both) was
executed, or 0 if neither was (that is, if x exited non-zero
and there was no else part).  The actual exit status of
x is never used as the exit status of the if command.

So if you mean "if" write "if" not && - using an and-or
list when an if should have been used does not show you
as being a "cool power programmer, using advanced syntax
that the boring newbies do not understand" rather it shows
that you are an incompetant beginner, trying to master
concepts not yet understood, and making a mess of it.

Use the proper construct for what is needed, always.

kre

Re: Possible bug in bash

[Lawrence Velázquez]

On Thu, May 12, 2022, at 11:34 PM, flyingrhino wrote:
> Should the "else" condition after the:  ||  run if the last command in 
> the:  &&  section exits non zero?

Yes.  This behavior is not a bug; ''A && B || C'' is simply not
equivalent to ''if A then B; else C; fi''.

https://mywiki.wooledge.org/BashPitfalls#pf22

> Script:
>
>
> #!/bin/bash
>
> [[ "a" == "a" ]] && \
>  {
>  echo "equal"
>  ls x
>  } || {
>  echo "* SHOULD NOT DISPLAY 4"
>  }
>
>
> Result:
>
> ./moo.sh
> equal
> ls: cannot access 'x': No such file or directory
> * SHOULD NOT DISPLAY 4

This behavior is expected and correct.  There is no bug here.

> BTW, I've checked other conditions as follows and they look ok:
>
> [...]
>
> [[ "a" == "a" ]] && \
>  {
>  echo "equal"
>  } || {
>  echo "* SHOULD NOT DISPLAY 1"
>  }

This would display "* SHOULD NOT DISPLAY 1" if ''echo "equal"''
failed for some reason.  (Unlikely but possible.)

-- 
vq

Re: possible bug in Bash 5.0

[Eli Schwartz]

On 12/25/19 5:41 PM, Xin Wu wrote:
> Hi,
> 
> I found the single-bracket [ ... ] and double-bracket [[ ... ]] behave
> differently for string comparison in the follow simple Bash-script.
> 
> # comma (,) is before two (2) in ASCII
> a=,rst
> b=2rst
> if [ "$a" \> "$b" ]; then
>   echo "single-bracket"
> fi
> if [[ "$a" > "$b" ]]; then
>   echo "double-bracket"
> fi
> 
> According to some web pages I was expecting that single-bracket and
> double-bracket should give same result. I'm not a Bash-expert, therefore
> I'm not sure whether this is a real bug in Bash 5.0.
> 
> Merry Christmas!
> 
> Xin
> 
> PS: the web pages are:
> 
> * http://tldp.org/LDP/abs/html/comparison-ops.html
> * http://tldp.org/LDP/abs/html/refcards.html
> * http://mywiki.wooledge.org/BashFAQ/031

On the last of these three web pages is the note:

"As of bash 4.1, string comparisons using < or > respect the current
locale when done in [[, but not in [ or test. In fact, [ and test have
never used locale collating order even though past man pages said they
did. Bash versions prior to 4.1 do not use locale collating order for [[
either."

Try using the C locale, for example: LC_COLLATE=C

Then see what the double brackets does.

-- 
Eli Schwartz
Arch Linux Bug Wrangler and Trusted User



signature.asc
Description: OpenPGP digital signature

Re: Possible bug in bash... or maybe UFU?

[Chet Ramey]

On 6/4/19 3:09 AM, George R Goffe wrote:
> Hi,
> I'm trying to build the latest bash from ftp.gnu.org and am having some 
> problems. Are these a bug or am I doing something wrong?

This is what happens when the bash configure can't find a working termcap
or curses library. What does the Makefile have as a value for TERMCAP_LIB?

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: Possible bug in Bash Reference Manual - lists of commands

[Greg Wooledge]

On Wed, May 22, 2019 at 02:38:29PM +0800, Ralph Jensen wrote:
> The Bash Reference Manual, Edition 5 and earlier versions define lists of
> commands as follows:
> 
> "A list is a sequence of one or more pipelines separated by one of the
> operators ..." (Bash Reference Manual 3.2.3).
> 
> Shouldn't that say commands rather than pipelines?

Seems correct as is.

wooledg:~$ if true|false; false|true; true|true; then echo "yes"; fi
yes

There's a sequence of three pipelines between "if" and "then".  Works
as expected.

RE: Possible bug with BASH V4: The $!

[Lenga, Yair]

Chet,

Thanks for the detailed feedback. I double check the documentation (man, 
changelog)  issues, those seems to be local problems with our system (for some 
strange reason, the man pages were mirrored from RH4 image).

One last comment about the double subshell case:
 ( echo $!) 
 ( echo $!) 

The detail of the $! include reference to current subshell, and seems to 
conflict with the Shell Execution Environment, I'm not sure how to resolve 
those conflicts.

However, the implemented behavior can lead to strange bugs, as it is very 
unexpected. My 2 cents is that the Bash3 behavior or reseting the $! in sub 
shells is more likely to prevent errors, and will still be in compliance with 
the POSIX text. In the following (not very practical) example:

( DO_somthing ; if [ PROBLEM ] ; then kill $!; fi ; sleep 20) 
...
( DO_somthing ; if [ PROBLEM ] ; then kill $!; fi ; sleep 20)  

The user is very unlikely to expect the second subshell to kill the first 
subshell.

In any case, I've modified my scripts to use ${BASHPID:-$!} to get them to 
work in BASH4 and BASH3. Again, Many thanks.

Yair.


-Original Message-
From: Chet Ramey [mailto:chet.ra...@case.edu] 
Sent: Thursday, April 11, 2013 2:03 PM
To: Lenga, Yair [ICG-MKTS]
Cc: 'chet.ra...@case.edu'; 'Dan Douglas'; 'bug-bash@gnu.org'
Subject: Re: Possible bug with BASH V4: The $!

On 4/11/13 12:15 PM, Lenga, Yair wrote:
 Dan, Chet: Many thanks for the info about BASHPID.
 
  I've checked BASHPID on RH6, and it looks OK. Few comments related to making 
 the change visible, and POSIX compliance.
 
 + The 'set' command, does not print the BASHPID, this make it very hard to 
 find it (unless you read every line in the BASH info file !).
It will be very helpful to include  BASHPID there (if set).

It does, but BASHPID is one of the special shell variables that springs into 
existence only when it's referenced.  Run the following script; it should 
display two lines of output:

sleep 1
echo $BASHPID
set | grep ^BASHP

 + The man page list BASH_VERSION, etc., but no indication of BASHPID. It's 
 probably a good idea to put a note next to '$!' about BASHPID.

It's in there, and has been since bash-4.0.

 + Could not find any change log on my system (probably a problem with RH). 
 Not sure if it's documented.

That depends on your distribution.  There is a summary changelog (CHANGES), a 
file listing new features (NEWS), and a detailed change log
(CWRU/changelog) in the FSF bash distribution.

 Also, I have a question about POSIX mode (and POSIX compliance).
 The POSIX says  '$!' is most recent background command executed from the 
 current shell.
 
 So in :
 
 ( echo $!) 
 ( echo $!) 
 
 According to POSIX, both calls should print  (nothing). As the background 
 command does not come from the current shell.
 
 With BASH4, The first call will print  (nothing), second call will print 
 the PID of the first call.

The bash behavior is correct.  The Posix requirement that (...) subshells 
create a `subshell environment', which is an exact copy of the parent shell's 
state with a couple of minor exceptions, trumps the text you quoted.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_12

for the exact reference.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Possible bug with BASH V4: The $!

[Greg Wooledge]

On Thu, Apr 11, 2013 at 01:05:43PM +, Lenga, Yair  wrote:
 Is it possible to add back the functionality to allow the child process to 
 somehow retrieve it's OWN PID.

You are looking for the BASHPID variable, which expands to the process
ID of the current bash process (even in a subshell).

Re: Possible bug with BASH V4: The $!

[Dan Douglas]

On Thursday, April 11, 2013 01:05:43 PM Lenga, Yair wrote:
 With BASH 3.0, the backgrounded task could access the PID of the parent using
 $$, and the PID Of itself as $!.
 
 With BASH 4.0, the script fail, the $! is not available to the child
 process.
 
 Is it possible to add back the functionality to allow the child process to
 somehow retrieve it's OWN PID.  I assume that the behavior of  $$ cannot be
 modified at this time.

BASHPID was introduced in Bash 4. I would guess that prior to that, the
behavior of $! was either a nonstandard extension or unintentional bug. Check
the changelog.

-- 
Dan Douglas

Re: Possible bug with BASH V4: The $!

[Chet Ramey]

On 4/11/13 9:24 AM, Dan Douglas wrote:
 On Thursday, April 11, 2013 01:05:43 PM Lenga, Yair wrote:
 With BASH 3.0, the backgrounded task could access the PID of the parent using
 $$, and the PID Of itself as $!.

 With BASH 4.0, the script fail, the $! is not available to the child
 process.

 Is it possible to add back the functionality to allow the child process to
 somehow retrieve it's OWN PID.  I assume that the behavior of  $$ cannot be
 modified at this time.
 
 BASHPID was introduced in Bash 4. I would guess that prior to that, the
 behavior of $! was either a nonstandard extension or unintentional bug. Check
 the changelog.

According to the change log, I made this change in late November 2006 as
the result of a shell discussion on the austin-group list.  BASHPID came
in to replace it shortly after.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

RE: Possible bug with BASH V4: The $!

[Lenga, Yair]

Dan, Chet: Many thanks for the info about BASHPID.

 I've checked BASHPID on RH6, and it looks OK. Few comments related to making 
the change visible, and POSIX compliance.

+ The 'set' command, does not print the BASHPID, this make it very hard to find 
it (unless you read every line in the BASH info file !).
   It will be very helpful to include  BASHPID there (if set).
+ The man page list BASH_VERSION, etc., but no indication of BASHPID. It's 
probably a good idea to put a note next to '$!' about BASHPID.
+ Could not find any change log on my system (probably a problem with RH). Not 
sure if it's documented.

Also, I have a question about POSIX mode (and POSIX compliance).
The POSIX says  '$!' is most recent background command executed from the 
current shell.

So in :

( echo $!) 
( echo $!) 

According to POSIX, both calls should print  (nothing). As the background 
command does not come from the current shell.

With BASH4, The first call will print  (nothing), second call will print the 
PID of the first call.

This is very confusing, and probably conflict with POSIX standard which 
indicate that $! apply to CURRENT shell.

Thanks
Yair

-Original Message-
From: Chet Ramey [mailto:chet.ra...@case.edu] 
Sent: Thursday, April 11, 2013 10:14 AM
To: Lenga, Yair [ICG-MKTS]
Cc: Dan Douglas; bug-bash@gnu.org; chet.ra...@case.edu
Subject: Re: Possible bug with BASH V4: The $!

On 4/11/13 9:24 AM, Dan Douglas wrote:
 On Thursday, April 11, 2013 01:05:43 PM Lenga, Yair wrote:
 With BASH 3.0, the backgrounded task could access the PID of the 
 parent using $$, and the PID Of itself as $!.

 With BASH 4.0, the script fail, the $! is not available to the 
 child process.

 Is it possible to add back the functionality to allow the child 
 process to somehow retrieve it's OWN PID.  I assume that the behavior 
 of  $$ cannot be modified at this time.
 
 BASHPID was introduced in Bash 4. I would guess that prior to that, 
 the behavior of $! was either a nonstandard extension or unintentional 
 bug. Check the changelog.

According to the change log, I made this change in late November 2006 as the 
result of a shell discussion on the austin-group list.  BASHPID came in to 
replace it shortly after.

Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Possible bug with BASH V4: The $!

[Greg Wooledge]

On Thu, Apr 11, 2013 at 04:15:45PM +, Lenga, Yair  wrote:
 + The man page list BASH_VERSION, etc., but no indication of BASHPID. It's 
 probably a good idea to put a note next to '$!' about BASHPID.

It is certainly in the man page:

  BASHPID
   Expands to the process ID of the current bash process.  This
   differs from $$ under certain circumstances, such as subshells
   that do not require bash to be re-initialized.

 + Could not find any change log on my system (probably a problem with RH). 
 Not sure if it's documented.

Debian installs this file as /usr/share/doc/bash/CHANGES.gz (no idea where
Red Hat puts it).

The relevant entry is this one:

c.  There is a new variable, $BASHPID, which always returns the process id of
the current shell.

This appears in the section for bash-4.0-alpha.

Re: Possible bug with BASH V4: The $!

[Dan Douglas]

On Thursday, April 11, 2013 04:15:45 PM Lenga, Yair wrote:
 ( echo $!) 
 ( echo $!) 
 
 According to POSIX, both calls should print  (nothing). As the background
 command does not come from the current shell.
 
 With BASH4, The first call will print  (nothing), second call will print
 the PID of the first call.
 
 This is very confusing, and probably conflict with POSIX standard which
 indicate that $! apply to CURRENT shell.

The backgrounding of each list member is sequential. Presumably $! should be
set immediately after each fork and inherited by subshells like most other
variables. See Command Execution Environment in the manual or POSIX.

`x ', `{ x; } ' and, `(x) ' -- where `x' is a simple command are all
identical. The subshell group () is redundant in your example.

Note the wait doesn't impact these results. It just forces the output to be
ordered.

$!/usr/bin/env bash
for sh in {{b,d}a,z,{m,}k,po}sh bb sh; do
printf %-6s ${sh}:
$sh /dev/fd/0
echo
done \EOF
${ZSH_VERSION+false} || emulate sh
n=5
while [ $((n -= 1)) -ge 0 ]; do
printf %s  ${!:-empty}  wait
done
EOF

bash: empty 2787 2788 2789 2790
dash: empty 2793 2794 2795 2796
zsh:  0 2800 2801 2802 2803
mksh: empty 2807 2809 2810 2811
ksh:  empty 2814 2815 2816 2817
posh: empty 2820 2821 2822 2823
bb:   empty 2826 2827 2828 2829 # busybox
sh:   empty 2832 2833 2834 2835 # bash POSIX mode

-- 
Dan Douglas

Re: Possible bug with BASH V4: The $!

[Chet Ramey]

On 4/11/13 12:15 PM, Lenga, Yair wrote:
 Dan, Chet: Many thanks for the info about BASHPID.
 
  I've checked BASHPID on RH6, and it looks OK. Few comments related to making 
 the change visible, and POSIX compliance.
 
 + The 'set' command, does not print the BASHPID, this make it very hard to 
 find it (unless you read every line in the BASH info file !).
It will be very helpful to include  BASHPID there (if set).

It does, but BASHPID is one of the special shell variables that springs
into existence only when it's referenced.  Run the following script; it
should display two lines of output:

sleep 1
echo $BASHPID
set | grep ^BASHP

 + The man page list BASH_VERSION, etc., but no indication of BASHPID. It's 
 probably a good idea to put a note next to '$!' about BASHPID.

It's in there, and has been since bash-4.0.

 + Could not find any change log on my system (probably a problem with RH). 
 Not sure if it's documented.

That depends on your distribution.  There is a summary changelog (CHANGES),
a file listing new features (NEWS), and a detailed change log
(CWRU/changelog) in the FSF bash distribution.

 Also, I have a question about POSIX mode (and POSIX compliance).
 The POSIX says  '$!' is most recent background command executed from the 
 current shell.
 
 So in :
 
 ( echo $!) 
 ( echo $!) 
 
 According to POSIX, both calls should print  (nothing). As the background 
 command does not come from the current shell.
 
 With BASH4, The first call will print  (nothing), second call will print 
 the PID of the first call.

The bash behavior is correct.  The Posix requirement that (...) subshells
create a `subshell environment', which is an exact copy of the parent
shell's state with a couple of minor exceptions, trumps the text you
quoted.

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_12

for the exact reference.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/

Re: Possible Bug in BASH

[Andreas Schwab]

Matthew Strax-Haber strax-habe...@neu.edu writes:

 Perhaps it is not a bug, but if it is not then the documentation is 
 inconsistent with the behavior.

There is no inconsistency.  In the first example you define a function
with the name `clear', in the second example you define a function with
the name `c'.  Surely they behave differently.  Alias substitution is a
pure textual substution that changes the text to be parsed.

Andreas.

-- 
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
And now for something completely different.

Re: Possible Bug in BASH

[Chet Ramey]

On 2/25/10 2:05 AM, Matthew Strax-Haber wrote:

 Below is a simple demonstration of the unexpected behavior:

 SHELL 1:
 mattsh$ alias c=clear
 mattsh$ c () { echo foo; }
 mattsh$ clear
 foo

 Here c is the first word.  So it is replaced by the alias.  (I didn't
 know this behavior before.  This is actually really surprising
 behavior to me since I didn't expect a function definition to be a
 simple command.)

When `c' is parsed and delimited as a token, it is in a position to be the
first word of a simple command.  The shell does not look ahead or apply
any grammar rules before performing alias substitution, so `c' is a valid
candidate for replacement.  Since there is an alias defined for `c', the
text is replaced.

This is as Posix specifies and how Posix shells behave.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/