Re: use-after-free in read_token_word
Also in parse_matched_pair:
diff --git a/parse.y b/parse.y
index 0a8c039a..1001ac1b 100644
--- a/parse.y
+++ b/parse.y
@@ -3906,14 +3906,13 @@ parse_matched_pair (int qc, int open, int
close, size_t *lenp, int flags)
/* Locale expand $"..." here. */
/* PST_NOEXPAND */
ttrans = locale_expand (nestret, 0, nestlen - 1, start_lineno, );
- free (nestret);
-
/* If we're supposed to single-quote translated strings,
check whether the translated result is different from
the original and single-quote the string if it is. */
if (singlequote_translations &&
((nestlen - 1) != ttranslen || STREQN (nestret, ttrans,
ttranslen) == 0))
{
+ free (nestret);
if ((rflags & P_DQUOTE) == 0)
nestret = sh_single_quote (ttrans);
else if ((rflags & P_DQUOTE) && (dolbrace_state ==
DOLBRACE_QUOTE2) && (flags & P_DOLBRACE))
@@ -3923,7 +3922,10 @@ parse_matched_pair (int qc, int open, int
close, size_t *lenp, int flags)
nestret = sh_backslash_quote_for_double_quotes (ttrans, 0);
}
else
- nestret = sh_mkdoublequoted (ttrans, ttranslen, 0);
+ {
+ free (nestret);
+ nestret = sh_mkdoublequoted (ttrans, ttranslen, 0);
+ }
free (ttrans);
nestlen = strlen (nestret);
retind -= 2; /* back up before the $" */
use-after-free in read_token_word
./bash --norc -O noexpand_translation -in <<<'$":"'
=ERROR: AddressSanitizer: heap-use-after-free on address 0x000108102b40
READ of size 1 thread T0
#0 read_token_word parse.y:5236
#1 read_token parse.y:3618
freed by thread T0 here:
#1 read_token_word parse.y:5231
#2 read_token parse.y:3618
diff --git a/parse.y b/parse.y
index e3516e2d..0a8c039a 100644
--- a/parse.y
+++ b/parse.y
@@ -5228,15 +5228,19 @@ read_token_word (int character)
/* PST_NOEXPAND */
/* Try to locale-expand the converted string. */
ttrans = locale_expand (ttok, 0, ttoklen - 1, first_line, );
- free (ttok);
-
/* Add the double quotes back (or single quotes if the user
has set that option). */
if (singlequote_translations &&
((ttoklen - 1) != ttranslen || STREQN (ttok, ttrans, ttranslen) == 0))
- ttok = sh_single_quote (ttrans);
+ {
+ free (ttok);
+ ttok = sh_single_quote (ttrans);
+ }
else
- ttok = sh_mkdoublequoted (ttrans, ttranslen, 0);
+ {
+ free (ttok);
+ ttok = sh_mkdoublequoted (ttrans, ttranslen, 0);
+ }
free (ttrans);
ttrans = ttok;
Re: Document that here strings don't support brace expansion.
Chet Ramey writes: > If they're linked, why wouldn't saying filename generation isn't performed > be enough to imply that brace expansion isn't performed either? Hmm, yes, good point. Dale
Re: Document that here strings don't support brace expansion.
On 3/15/23 12:52 AM, Alex Bochannek wrote: Chet, Thank you for the thoughtful responses. My thoughts below got a bit long, so here is a summary: - The here-string documentation should either spell out that brace expansion is not performed or simple only list the expansions that are. The original text didn't describe the specific expansions for here-strings at all, just deferred to the general description of redirections and here- documents. I changed to the current text after an August 2011 discussion where it was requested (in the midst of yet another riveting discussion of the value of `set -e'). The fact that pathname expansion is not performed was a particular sore point for at least one of the participants. In general, I don't like enumerating expansions that aren't performed, but it was the best way to resolve the issue here. - The same goes for the documentation for the shell-expand-line Readline command. Yes, I can make that explicit. - A Readline command to expand braces is a nice to have, but not essential. OK. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: Document that here strings don't support brace expansion.
On Wed, Mar 15, 2023, 8:07 AM Robert Elz wrote:
> While it is probably useful to have every place where expansions occur list
> which ones apply, perhaps it would (also) be useful to include the general
> principle which controls all of this.
>
> That is, expansions are performed, late (just as commands are about to
> be run, after all parsing has finished, etc) in one of two basic contexts.
>
> In one, like in arg lists to commands (also the list after "in" in a for
> statement, and probablty more places I have forgotten), any number of words
> can be produced. In those, all the expansions typically happen.
>
> In the other, exactly one word is required - like the word in var=${word}
> or the word in case ${word} in, or (in bash, and perhaps a couple of other
> shells) the word after a here string operator <<<${word} (and one more I
> will come to in a minute). [The word after a here-doc redirect operator
> (<<)
> is strange for hysterical (oops: historical) reasons.]
>
> In those, the syntax allows for only one word to be produced, so performing
> field splitting, globbing, or brace expansion, all make no sense at all,
> and so the basic assumption, unless specified otherwise, is that none of
> those will happen. (This is also why var="$@" is just plain stupid to
> write,
> and has unspecified consequences, though people keep doing it).
>
> quote removal always happens (everywhere).
>
> The one other place where just one word is required, and is all that
> makes sense, is after a redirect operator - the file name (or fd if the
> operator is >& or similar). That is >${word} (etc). For most shells
> none of the multi-word generating "expansions" happen there either, as
> there has to be exactly one filename to write into (in this example), not
> several. However, at least when interactive, some shells perform globbing
> at this point, but require that only a single filename match the pattern
> (making it an error if more than one do) - just because users are too lazy
> to type the entire filename -- or use a func like:
>
> setfn()
> {
> var=$1; shift
> set -- "$@"
> case "$#" in
> 1) test -e "$1" && {
> eval "$var='$1'"
>
declare -g "$var=$1"
> return 0
>}
>printf >&2 'No match\n'
>return 1;
>;;
> esac
> printf >&2 'Ambiguous: %d files matched\n', "$#"
> return 1
> }
>
> Used like
> setfn f a*.txt
> after which, if it worked, you can use "$f" instead of the
> filename you are too lazy to type, which is shorter than a
> pattern guaranteed to only match one file will often be.
>
> Further it doesn't risk failing later, if something you do
> causes the pattern to later match more than one file (say you
> create abc-new.txt as a new version of abc.txt or save the
> old one as abc.was.txt). "$f" doesn't change until you change it.
>
> Bash allows filename expansion after a redirect, at least in
> interactive shells (haven't tested it for non-interactive), it
> also performs brace expansion there - but that's largely guaranteed
> to fail, so I have no idea why - no field splitting though.
> Those exceptions to the general rule need to be clearly pointed out,
> but explaining every time something requires just one word, that
> some expansions don't happen, when in a context where it would be
> insane if they did, seems overdone.
>
> <<< requires a single word, as so all other redirect operators.
> Brace expansion, which has as its whole purpose, generating multiple
> words (as does field splitting, though it doesn't always succeed)
> (globbing merely can) should be obviously inappropriate there (just as
> it really should be after any redirect operator, which is why I
> was surprised to see
>
> $ cat < a{1,2,3}
> -bash: a{1,2,3}: ambiguous redirect
>
> rather than something more like
> -bash: a{1,2,3}: No such file or directory
>
> The only example of brace expansion actually "working" in a
> redirect, is something like
>
> $ cat -bash: a1: No such file or directory
>
> but I am unable to think of any situation where doing that is
> better than just "cat expanded, we know that either they have the same value, in which
> case we only need to use one of them in which case we get an "ambiguous redirect error". Crazy.
>
> kre
>
>
>
Re: Document that here strings don't support brace expansion.
While it is probably useful to have every place where expansions occur list
which ones apply, perhaps it would (also) be useful to include the general
principle which controls all of this.
That is, expansions are performed, late (just as commands are about to
be run, after all parsing has finished, etc) in one of two basic contexts.
In one, like in arg lists to commands (also the list after "in" in a for
statement, and probablty more places I have forgotten), any number of words
can be produced. In those, all the expansions typically happen.
In the other, exactly one word is required - like the word in var=${word}
or the word in case ${word} in, or (in bash, and perhaps a couple of other
shells) the word after a here string operator <<<${word} (and one more I
will come to in a minute). [The word after a here-doc redirect operator (<<)
is strange for hysterical (oops: historical) reasons.]
In those, the syntax allows for only one word to be produced, so performing
field splitting, globbing, or brace expansion, all make no sense at all,
and so the basic assumption, unless specified otherwise, is that none of
those will happen. (This is also why var="$@" is just plain stupid to write,
and has unspecified consequences, though people keep doing it).
quote removal always happens (everywhere).
The one other place where just one word is required, and is all that
makes sense, is after a redirect operator - the file name (or fd if the
operator is >& or similar). That is >${word} (etc). For most shells
none of the multi-word generating "expansions" happen there either, as
there has to be exactly one filename to write into (in this example), not
several. However, at least when interactive, some shells perform globbing
at this point, but require that only a single filename match the pattern
(making it an error if more than one do) - just because users are too lazy
to type the entire filename -- or use a func like:
setfn()
{
var=$1; shift
set -- "$@"
case "$#" in
1) test -e "$1" && {
eval "$var='$1'"
return 0
}
printf >&2 'No match\n'
return 1;
;;
esac
printf >&2 'Ambiguous: %d files matched\n', "$#"
return 1
}
Used like
setfn f a*.txt
after which, if it worked, you can use "$f" instead of the
filename you are too lazy to type, which is shorter than a
pattern guaranteed to only match one file will often be.
Further it doesn't risk failing later, if something you do
causes the pattern to later match more than one file (say you
create abc-new.txt as a new version of abc.txt or save the
old one as abc.was.txt). "$f" doesn't change until you change it.
Bash allows filename expansion after a redirect, at least in
interactive shells (haven't tested it for non-interactive), it
also performs brace expansion there - but that's largely guaranteed
to fail, so I have no idea why - no field splitting though.
Those exceptions to the general rule need to be clearly pointed out,
but explaining every time something requires just one word, that
some expansions don't happen, when in a context where it would be
insane if they did, seems overdone.
<<< requires a single word, as so all other redirect operators.
Brace expansion, which has as its whole purpose, generating multiple
words (as does field splitting, though it doesn't always succeed)
(globbing merely can) should be obviously inappropriate there (just as
it really should be after any redirect operator, which is why I
was surprised to see
$ cat < a{1,2,3}
-bash: a{1,2,3}: ambiguous redirect
rather than something more like
-bash: a{1,2,3}: No such file or directory
The only example of brace expansion actually "working" in a
redirect, is something like
$ cat
