Re: string substitution broken since 5.2
2022年11月4日(金) 8:48 Greg Wooledge :
> On Thu, Nov 03, 2022 at 05:20:52PM -0400, Greg Wooledge wrote:
> > On Fri, Nov 04, 2022 at 04:09:10AM +0900, Koichi Murase wrote:
> > > When one wants to also support Bash 4.2, one possible workaround is to
> > > assign the result to a variable (without quoting the right-hand side
> > > of the assignment):
> > >
> > > $ bash-4.2 -c 'string=\"hi\"; string=${string//\"/\}; echo
> > > "$string"'
> > > $ bash-4.2 -c 'str="a string"; rep="a"; str=${str//a/"$rep"}; echo
> > > "$str"'
> > >
> > > These work as expected in all Bash versions I can test (2.0..5.2).
> >
> > That's amazingly clever. You're forcing the semantics of double-quoting
> > the right-hand side (implicitly, due to the simple assignment), without
> > putting actual quotes in it.
>
> Hmm, no, you *are* including quotes, so that's not why it works.
> I actually have no idea why it works.
Ah, sorry for confusing you. By "without quoting the right-hand side",
I meant that we strip the *outer* quoting of the entire right-hand
side. More explicitly,
string="${string//\"/\}"
still does not work in Bash 4.2 or with `shopt -s compat42', but
string=${string//\"/\}
works. I think you already know it, but this is because, in Bash 4.2,
quoting inside the parameter expansion (i.e., \& in this case) is
affected by the context of the parameter expansion itself (i.e.,
${string ...}). When ${string ...} is placed inside the outer double
quotes "${string ...}", the inner quoting is also treated as if it is
inside the double quotes, i.e. \& is treated as literal \&. When
${string ...} is not placed inside the double quotes, the inner
quoting is processed as if it is directly in the right-hand sides of
assignments, so \& becomes &.
Re: string substitution broken since 5.2
On Thu, Nov 03, 2022 at 05:20:52PM -0400, Greg Wooledge wrote:
> On Fri, Nov 04, 2022 at 04:09:10AM +0900, Koichi Murase wrote:
> > When one wants to also support Bash 4.2, one possible workaround is to
> > assign the result to a variable (without quoting the right-hand side
> > of the assignment):
> >
> > $ bash-4.2 -c 'string=\"hi\"; string=${string//\"/\}; echo "$string"'
> > $ bash-4.2 -c 'str="a string"; rep="a"; str=${str//a/"$rep"}; echo "$str"'
> >
> > These work as expected in all Bash versions I can test (2.0..5.2).
>
> That's amazingly clever. You're forcing the semantics of double-quoting
> the right-hand side (implicitly, due to the simple assignment), without
> putting actual quotes in it.
Hmm, no, you *are* including quotes, so that's not why it works.
I actually have no idea why it works. But that's OK. I'm just happy
that it *does* work.
So far, that makes two cases where a temporary variable is the only
known workaround for incompatible changes introduced in bash 5.2:
parameter expansion with search-and-replace, and associative array
element access within an arithmetic context.
Re: string substitution broken since 5.2
On Fri, Nov 04, 2022 at 04:09:10AM +0900, Koichi Murase wrote:
> When one wants to also support Bash 4.2, one possible workaround is to
> assign the result to a variable (without quoting the right-hand side
> of the assignment):
>
> $ bash-4.2 -c 'string=\"hi\"; string=${string//\"/\}; echo "$string"'
> $ bash-4.2 -c 'str="a string"; rep="a"; str=${str//a/"$rep"}; echo "$str"'
>
> These work as expected in all Bash versions I can test (2.0..5.2).
That's amazingly clever. You're forcing the semantics of double-quoting
the right-hand side (implicitly, due to the simple assignment), without
putting actual quotes in it.
Re: bash "extglob" needs to upgrade at least like zsh "kshglob"
On 10/29/22 1:50 AM, Martin D Kealey wrote: PS: While we're about it, can we please fix the expansion of « a/*/b/c/d/* » so it only calls readdir on a/ and each dir matching a/*/b/c/d/ and NOT call readdir on dirs matching a/*/, a/*/b/, or a/*/b/c/. It already does this. Given $ find . -print . ./a ./a/z ./a/z/b ./a/z/b/c ./a/x ./a/x/b ./a/x/b/c ./a/x/b/c/d ./a/x/b/c/d/1 ./a/x/b/c/d/4 ./a/x/b/c/d/3 ./a/x/b/c/d/2 ./a/x/b/c/d/5 ./a/y ./a/y/b Running `echo a/*/b/c/d/*' calls opendir(2) twice: $ ./bash ./x1 opendir: a/ opendir: a/x/b/c/d a/x/b/c/d/1 a/x/b/c/d/2 a/x/b/c/d/3 a/x/b/c/d/4 a/x/b/c/d/5 -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: string substitution broken since 5.2
2022年11月4日(金) 3:46 Greg Wooledge :
> On Fri, Nov 04, 2022 at 03:31:44AM +0900, Koichi Murase wrote:
> > [...]
> >
> > Another option is to quote `&' in the replacement if you want to make
> > it work regardless of the 5.2 feature, patsub_replacement:
> >
> > string="${string//\"/\}"
>
> Sadly, bash 5.2 breaks backward compatibility in ways that are highly
> inconvenient. There's essentially no way you can write this that will
> work with bash 4.2, bash 4.3-5.1, and bash 5.2.
>
> [,,,]
>
> In another 10 years or so, this may not matter so much, as most instances
> of bash 4.2 and earlier will have been upgraded or retired. But for now,
> it's a pretty significant issue.
When one wants to also support Bash 4.2, one possible workaround is to
assign the result to a variable (without quoting the right-hand side
of the assignment):
$ bash-4.2 -c 'string=\"hi\"; string=${string//\"/\}; echo "$string"'
$ bash-4.2 -c 'str="a string"; rep="a"; str=${str//a/"$rep"}; echo "$str"'
These work as expected in all Bash versions I can test (2.0..5.2). I
used this approach in one of my fixes related to patsub_replacement:
https://github.com/akinomyoga/ble.sh/commit/a75bb25a3255fd87ba73f0f1a5ebedc23fcc71b4
--
Koichi
Re: string substitution broken since 5.2
On 11/3/22 1:32 PM, thierryb--- via Bug reports for the GNU Bourne Again
SHell wrote:
Bash Version: 5.2
Patch Level: 2
Release Status: release
Description:
String substitution code running for years is broken in 5.2.
Repeat-By:
string = 'xdotool type "sudo apt update"'
string="${string//\"/}"
printf '%s' "$string"
previously outputs:
xdotool type sudo apt update
but now outputs:
xdotool type "quot;sudo apt update"quot;
From NEWS:
x. New shell option: patsub_replacement. When enabled, a `&' in the replacement
string of the pattern substitution expansion is replaced by the portion of
the string that matched the pattern. Backslash will escape the `&' and
insert a literal `&'.
The option is enabled by default. If you want to restore the previous
behavior, add `shopt -u patsub_replacement'.
--
``The lyf so short, the craft so long to lerne.'' - Chaucer
``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
Re: string substitution broken since 5.2
On Fri, Nov 04, 2022 at 03:31:44AM +0900, Koichi Murase wrote:
> 2022年11月4日(金) 2:50 Andreas Schwab :
> > On Nov 03 2022, thierryb--- via Bug reports for the GNU Bourne Again SHell
> > wrote:
> > > Description:
> > > String substitution code running for years is broken in 5.2.
> > >
> > > Repeat-By:
> > > string = 'xdotool type "sudo apt update"'
> > > string="${string//\"/}"
> > > printf '%s' "$string"
> > >
> > > previously outputs:
> > > xdotool type sudo apt update
> > >
> > > but now outputs:
> > > xdotool type "quot;sudo apt update"quot;
> >
> > shopt -u patsub_replacement
>
> Another option is to quote `&' in the replacement if you want to make
> it work regardless of the 5.2 feature, patsub_replacement:
>
> string="${string//\"/\}"
Sadly, bash 5.2 breaks backward compatibility in ways that are highly
inconvenient. There's essentially no way you can write this that will
work with bash 4.2, bash 4.3-5.1, and bash 5.2.
unicorn:~$ bash-4.2 -c 'string=\"hi\"; echo "${string//\"/\}"'
\hi\
unicorn:~$ bash-4.3 -c 'string=\"hi\"; echo "${string//\"/\}"'
hi
unicorn:~$ bash-5.2 -c 'string=\"hi\"; echo "${string//\"/\}"'
hi
This also affects quoting/non-quoting of the replacement, if it's in a
variable. If you DON'T quote, it works in everything up to 5.1, but
not later:
unicorn:~$ bash-4.2 -c 'str="a string"; rep="a"; echo "${str//a/$rep}"'
a string
unicorn:~$ bash-4.3 -c 'str="a string"; rep="a"; echo "${str//a/$rep}"'
a string
unicorn:~$ bash-5.2 -c 'str="a string"; rep="a"; echo "${str//a/$rep}"'
aab string
If you DO quote, it works in everything from 4.3 up, but not earlier:
unicorn:~$ bash-4.2 -c 'str="a string"; rep="a"; echo "${str//a/"$rep"}"'
"a" string
unicorn:~$ bash-4.3 -c 'str="a string"; rep="a"; echo "${str//a/"$rep"}"'
a string
unicorn:~$ bash-5.2 -c 'str="a string"; rep="a"; echo "${str//a/"$rep"}"'
a string
In another 10 years or so, this may not matter so much, as most instances
of bash 4.2 and earlier will have been upgraded or retired. But for now,
it's a pretty significant issue.
Re: string substitution broken since 5.2
Le 03/11/2022 à 18:49, Andreas Schwab a écrit : shopt -u patsub_replacement OK thanks, I read the 5.2 release notes. I just regret that this option is enabled by default, it will break many scripts running since years. -- Thierry Bothorel OpenPGP_0x81BB850F60BA2BC2.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature
Re: string substitution broken since 5.2
2022年11月4日(金) 2:50 Andreas Schwab :
> On Nov 03 2022, thierryb--- via Bug reports for the GNU Bourne Again SHell
> wrote:
> > Description:
> > String substitution code running for years is broken in 5.2.
> >
> > Repeat-By:
> > string = 'xdotool type "sudo apt update"'
> > string="${string//\"/}"
> > printf '%s' "$string"
> >
> > previously outputs:
> > xdotool type sudo apt update
> >
> > but now outputs:
> > xdotool type "quot;sudo apt update"quot;
>
> shopt -u patsub_replacement
Another option is to quote `&' in the replacement if you want to make
it work regardless of the 5.2 feature, patsub_replacement:
string="${string//\"/\}"
Re: string substitution broken since 5.2
On Nov 03 2022, thierryb--- via Bug reports for the GNU Bourne Again SHell
wrote:
> Description:
> String substitution code running for years is broken in 5.2.
>
> Repeat-By:
> string = 'xdotool type "sudo apt update"'
> string="${string//\"/}"
> printf '%s' "$string"
>
> previously outputs:
> xdotool type sudo apt update
>
> but now outputs:
> xdotool type "quot;sudo apt update"quot;
shopt -u patsub_replacement
--
Andreas Schwab, sch...@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
string substitution broken since 5.2
Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2 -fstack-protector-strong -Wformat
-Werror=format-security -Wall
uname output: Linux thierrybo-desk 6.0.0-2-amd64 #1 SMP PREEMPT_DYNAMIC Debian
6.0.5-1 (2022-10-28) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu
Bash Version: 5.2
Patch Level: 2
Release Status: release
Description:
String substitution code running for years is broken in 5.2.
Repeat-By:
string = 'xdotool type "sudo apt update"'
string="${string//\"/}"
printf '%s' "$string"
previously outputs:
xdotool type sudo apt update
but now outputs:
xdotool type "quot;sudo apt update"quot;
