Re: Bash 5.2.21 segfaults when I feed it garbage
On 1/10/24 2:06 PM, Grisha Levit wrote: Rewriting the original report as: bash <<<'((X=([))' even after the last fix, there's still a similar issue with input like: bash <<<'((X=([))]' Thanks for the report. I debated whether or not to do error handling before restoring the parser state; this makes up my mind. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: nofork command substitution bugs
On 1/12/24 1:06 PM, Oğuz wrote:
Why would that be unexpected, since you're explicitly running something
in the calling shell's context, with the expected side effects to that
environment?
I wasn't clear. This doesn't exit the shell
bash-5.3$ exec foo
bash: exec: foo: not found
bash-5.3$
This does:
bash-5.3$ : ${ exec foo;}
bash: exec: foo: not found
$
Why would you expect either to cause an interactive shell to exit?
Oh, I see, an interactive shell. Think of funsubs in an interactive shell
as being temporarily non-interactive, like when the shell executes
./source. If you source a file from an interactive shell, a failed exec
causes the shell to exit.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: document that read built-in can't return zero-length string in the middle of input
On Sat, Jan 13, 2024 at 02:06:08AM +0700, Robert Elz wrote:
> Date:Fri, 12 Jan 2024 07:15:35 -0500
> From:Greg Wooledge
> Message-ID:
>
> | This was one of the things I tested:
>
> Perhaps intended to, but didn't, or not in this example:
>
> | { read -N1; read -r b c; } < <(printf \\nabc); declare -p REPLY a b c
>
> Rewrite that, correctly for the purpose, as:
>
> { read -N1; read -r b c; } < <(printf \\nabc); declare -p REPLY b c
>
> or a bit more cleanly as:
>
> { read -N1; read -r b c; } < <(printf '\\\nabc'); declare -p REPLY b c
Yes, I made an error there. Sorry for the confusion.
However, you're testing something slightly different from what I was
testing. Here's what I originally intended:
unicorn:~$ { read -N1; read -r b c; } < <(printf %s \\nabc); declare -p REPLY b
c
declare -- REPLY="n"
declare -- b="abc"
declare -- c=""
The important part of the result, for me, is that read -N1 still consumes
two characters of input, and stores one character into the variable.
The stored character is 'n' instead of a newline, after fixing the test.
And here's what you thought I was trying to test:
unicorn:~$ { read -N1; read -r b c; } < <(printf %s $'\\\nabc'); declare -p
REPLY b c
declare -- REPLY="a"
declare -- b="bc"
declare -- c=""
In this case, read -N1 consumes *three* characters of input, and stores
one character into the variable. So the character count difference is
even more dramatic.
> ps: the use of process substitution there is just silly, it would work
> just as well, and be easier to understand if written:
>
> printf '\\\nabc' | { read -N1; read -r b c; }; declare -p REPLY b c
That requires lastpipe, or moving the declare -p inside the curly braces.
> or{ read -N1; read -r b c; } <<< $'\\\nabc' ; declare -p REPLY b c
<<< unavoidably adds a newline character to the input, which isn't
actually harmful in this case, but isn't what I was trying to test.
I generally go with < <(printf %s ...) when I want to send input that
may not end with a newline. I simply forgot the %s.
I actually ran several tests before this, which I didn't show in the
previous email. The first was:
unicorn:~$ { read -r -N1 a b c; } < <(printf abc); declare -p a b c
declare -- a="a"
declare -- b=""
declare -- c=""
I kept making changes to this, ultimately ending up with the test with
the error in it. I should've started with printf %s in the first place,
but I was lazy. There's a lesson somewhere in there.
In any case, the important part of this thread is the proposed
documentation change. My wording may not be ideal, so if you have
any improvement suggestions, feel free to make them.
Re: document that read built-in can't return zero-length string in the middle of input
On Fri, Jan 12, 2024, at 2:06 PM, Robert Elz wrote:
> ps: the use of process substitution there is just silly, it would work
> just as well, and be easier to understand if written:
>
> printf '\\\nabc' | { read -N1; read -r b c; }; declare -p REPLY b c
Presumably Greg was just preventing the implicit pipeline subshell
from swallowing the variables.
bash-5.2$ printf '\\\nabc' | { read -N1; read -r b c; }; declare -p
REPLY b c
bash: declare: REPLY: not found
bash: declare: b: not found
bash: declare: c: not found
For these self-contained experiments, printing the values from
within the subshell is probably fine.
bash-5.2$ printf '\\\nabc' | { read -N1; read -r b c; declare -p REPLY
b c; }
declare -- REPLY="a"
declare -- b="bc"
declare -- c=""
--
vq
Re: document that read built-in can't return zero-length string in the middle of input
Date:Fri, 12 Jan 2024 07:15:35 -0500
From:Greg Wooledge
Message-ID:
| This was one of the things I tested:
Perhaps intended to, but didn't, or not in this example:
| { read -N1; read -r b c; } < <(printf \\nabc); declare -p REPLY a b c
Rewrite that, correctly for the purpose, as:
{ read -N1; read -r b c; } < <(printf \\nabc); declare -p REPLY b c
or a bit more cleanly as:
{ read -N1; read -r b c; } < <(printf '\\\nabc'); declare -p REPLY b c
and (both cases) you get:
declare -- REPLY="a"
declare -- b="bc"
declare -- c=""
bash is (as I assumed earlier it would) doing the correct thing, and
ignoring the elided newline completely. If it didn't, that would have
been a bug, but it does, so all is well.
In the case you tested, printf output $'\nabc' and read -N1 correctly
read the first char (\n) into REPLY (and then read abc into b, 'c' isn't
really needed for this example).
[Note: I deleted the output of 'a' as I didn't conveniently have a
variable called 'a' defined, so declare issued an error, which would
have just confused things.]
kre
ps: the use of process substitution there is just silly, it would work
just as well, and be easier to understand if written:
printf '\\\nabc' | { read -N1; read -r b c; }; declare -p REPLY b c
or { read -N1; read -r b c; } <<< $'\\\nabc' ; declare -p REPLY b c
I much prefer the former of those two.
Re: nofork command substitution bugs
On Fri, Jan 12, 2024 at 7:05 PM Chet Ramey wrote:
> Nofork command substitution freezes the jobs list, because you don't
> want jobs appearing and disappearing in the list while you're running
> word expansion. If the jobs list is frozen, wait -n doesn't even try
> waiting, since you don't want jobs removed from the list in, say, a
> SIGCHLD trap. I think it's ok to do the wait but not delete the job,
> regardless of what posix says, so I'll make that change for funsubs.
Sounds good, thanks.
> Why would that be unexpected, since you're explicitly running something
> in the calling shell's context, with the expected side effects to that
> environment?
I wasn't clear. This doesn't exit the shell
bash-5.3$ exec foo
bash: exec: foo: not found
bash-5.3$
This does:
bash-5.3$ : ${ exec foo;}
bash: exec: foo: not found
$
Why would you expect either to cause an interactive shell to exit?
Re: No filename expansion in shell-expand-line
On 1/12/24 3:28 AM, Martin Schulte wrote: Hello, from the documentation I understand that shell-expand-line (ESC CTRL-E) should do alias expansion, history expansion, brace expansion, tilde expansion, shell parameter expansion, command substitution, arithmetic expansion, process substitution (if supported by the operation system), word splitting, filename expansion and quote removal. It doesn't do filename expansion and never has, going all the way back to when it was first added in 1989, and the current man page doesn't include it in the enumerated list of expansions it performs: shell-expand-line (M-C-e) Expand the line by performing shell word expansions. This per- forms alias and history expansion, $'string' and $"string" quot- ing, tilde expansion, parameter and variable expansion, arith- metic expansion, word splitting, and quote removal. See HISTORY EXPANSION below for a description of history expansion. This is an update from the bash-5.2 man page, the result of https://lists.gnu.org/archive/html/bug-bash/2023-03/msg00031.html There's a separate bindable command to perform filename expansion. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/ OpenPGP_signature.asc Description: OpenPGP digital signature
Re: nofork command substitution bugs
On 1/4/24 8:36 AM, Oğuz wrote:
These bugs affect the development branch only.
1. `wait -n' doesn't work inside nofork command substitution. I think
it should, or wait without `-n' shouldn't work either, or what works
and what doesn't should be documented.
Nofork command substitution freezes the jobs list, because you don't
want jobs appearing and disappearing in the list while you're running
word expansion. If the jobs list is frozen, wait -n doesn't even try
waiting, since you don't want jobs removed from the list in, say, a
SIGCHLD trap. I think it's ok to do the wait but not delete the job,
regardless of what posix says, so I'll make that change for funsubs.
$ sleep 3 & sleep 1; echo ${| wait -n -p REPLY;}
[1] 433
$
[1]+ Donesleep 3
$
2. Sending a signal other than SIGTERM to an asynchronous process
created inside nofork command substitution kills the parent shell.
This should not happen.
Everything in a command substitution is in the same process group as the
parent shell, since word expansions are performed by the parent shell and
command substitution is a synchronous operation. As a result, asynchronous
processes in nofork command substitutions aren't run in a separate process
group when job control is active. If you send a signal to the process
group, the shell gets it, too. I'm not sure of the right approach here;
I'll have to think about it.
There are other cases where the parent shell exits unexpectedly, such
as when the exec command fails inside nofork command substitution, but
I'm not sure if those are bugs or intended behavior.
Why would that be unexpected, since you're explicitly running something
in the calling shell's context, with the expected side effects to that
environment?
3. Linking REPLY to a variable inside nofork command substitution
renders the target variable unset. I don't think this should be the
case; and if it has to, it should be documented.
I agree; I don't think the implicit creation of REPLY as a local variable
should follow namerefs here.
Chet
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
OpenPGP_signature.asc
Description: OpenPGP digital signature
Re: document that read built-in can't return zero-length string in the middle of input
On Fri, Jan 12, 2024 at 03:26:31PM +0700, Robert Elz wrote:
> Date:Thu, 11 Jan 2024 20:02:04 -0500
> From:Greg Wooledge
> Message-ID:
>
> | What actually counts is how many
> | characters are *stored* in the variable, not how many characters are
> | *read* from the input.
>
> I suspect that is the root of the issue here, you're considering
> the word "read" to mean read(2) whereas as it is in the documentation
> for the read command, I suspect it means that one, how many characters
> the read command obtains, rather than how many the read system call
> processes (like elided newlines. when -r is not used, would also not
> be counted, or expected to be, right?)
This was one of the things I tested:
unicorn:~$ { read -N1; read -r b c; } < <(printf \\nabc); declare -p REPLY a b
cdeclare -- REPLY=$'\n'
declare -- a=$'\n'
declare -- b="abc"
declare -- c=""
(The value of 'a' was from a previous test.) read -N1 consumed two
characters of the input, and stored one character in the variable.
As a script writer, those are the things I care about. I don't know
or care about the internals of the shell.
Also note that I'm assuming the implementation isn't going to change,
and that our sole goal in this thread is to document its behavior.
Therefore I don't speak about what's "expected" -- merely what is.
I would, however, strongly recommend that script writers who use
read -N *always* include the -r option.
Re: No filename expansion in shell-expand-line
Sorry, I falied to copy and paste R*, the last line is replaced by
ls --color=auto echo {a..c} ~root xterm-256color 2024-01-12 42 /dev/fd/63 hello
R*
> Hello,
>
> from the documentation I understand that shell-expand-line (ESC CTRL-E)
> should do alias expansion, history expansion, brace expansion, tilde
> expansion, shell parameter expansion, command substitution, arithmetic
> expansion, process substitution (if supported by the operation system), word
> splitting, filename expansion and quote removal.
>
> $ echo $BASH_VERSION
> 5.2.15(1)-release
> $ echo R*
> README
> $ echo
>
> $ ls !! {a..c} ~root $TERM $(date +%F) $((6*7)) <(cat /etc/passwd) "hello" R*
>
> After pressing ESC CTRL-E the last line is replaced by:
>
> ls --color=auto echo {a..c} ~root xterm-256color 2024-01-12 42 /dev/fd/63
> hello
>
> So, everything works fine execept filename expansion, which seems to be
> missing.
>
> Or do I understand something wrong?
>
> Best regards,
>
> Martin
No filename expansion in shell-expand-line
Hello,
from the documentation I understand that shell-expand-line (ESC CTRL-E) should
do alias expansion, history expansion, brace expansion, tilde expansion, shell
parameter expansion, command substitution, arithmetic expansion, process
substitution (if supported by the operation system), word splitting, filename
expansion and quote removal.
$ echo $BASH_VERSION
5.2.15(1)-release
$ echo R*
README
$ echo
$ ls !! {a..c} ~root $TERM $(date +%F) $((6*7)) <(cat /etc/passwd) "hello" R*
After pressing ESC CTRL-E the last line is replaced by:
ls --color=auto echo {a..c} ~root xterm-256color 2024-01-12 42 /dev/fd/63 hello
So, everything works fine execept filename expansion, which seems to be missing.
Or do I understand something wrong?
Best regards,
Martin
Re: document that read built-in can't return zero-length string in the middle of input
Date:Thu, 11 Jan 2024 20:02:04 -0500 From:Greg Wooledge Message-ID: | What actually counts is how many | characters are *stored* in the variable, not how many characters are | *read* from the input. I suspect that is the root of the issue here, you're considering the word "read" to mean read(2) whereas as it is in the documentation for the read command, I suspect it means that one, how many characters the read command obtains, rather than how many the read system call processes (like elided newlines. when -r is not used, would also not be counted, or expected to be, right?) kre
