Re: manual page missing ${parameter-replacement}
On Mon, Nov 21, 2022, at 2:20 AM, Ulrich Windl wrote:
> I think the bash manual page lacks some important detail of parameter
> substitution, namely:
> "Bash tests for a parameter that is unset or null. Omitting the colon
> results in a test only for a parameter that is unset."
>
> (found in
> https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html)
You omitted all of the relevant context. The full paragraph is:
When not performing substring expansion, using the form [sic]
described below (e.g., ':-'), Bash tests for a parameter
that is unset or null. Omitting the colon results in a
test only for a parameter that is unset. Put another way,
if the colon is included, the operator tests for both
_parameter_'s existence and that its value is not null; if
the colon is omitted, the operator tests only for existence.
The manual then goes on to describe the ${parameter:-word},
${parameter:=word}, ${parameter:?word}, and ${parameter:+word}
forms. These are what your excerpt is talking about.
So what do you think is missing, exactly?
--
vq
manual page missing ${parameter-replacement}
I think the bash manual page lacks some important detail of parameter substitution, namely: "Bash tests for a parameter that is unset or null. Omitting the colon results in a test only for a parameter that is unset." (found in https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html)
Re: bash memory leak when querying associative array for nonexisting element
2022年11月20日(日) 23:07 Ralf Oehler :
> Bash Version: 5.2
> Patch Level: 2
> Release Status: release
>
> Description:
> [Detailed description of the problem, suggestion, or
> complaint.]
> I want to read an associative array. Like so: y="${aa[$i]}"
> If aa contains the element queried, then everything works as
> expected
> If aa does not contain such an element, the result is the
> empty string, which is expected, but the query permanently
> increases the memory consumption of the executing bash,
> which is not expected. The program below demonstrates this
> behaviour. When run, the bash process increases its memory
> indefinitely.
I also noticed a memory leak early this month in testing my new
extglob implementation and was planning to submit a patch later. I now
checked this report and confirmed that this is actually caused by the
same part of the code. Here, I attach my patch
[r0037.parameter_brace_expand_word.memleak.patch], though I haven't
yet carefully tested it because I originally planned to submit it
sometime later.
--
Koichi
From 4183e204ea0f954de42dea6ba2ec0f47a061f7f3 Mon Sep 17 00:00:00 2001
From: Koichi Murase
Date: Thu, 3 Nov 2022 14:01:41 +0900
Subject: [PATCH 1/2] fix memleak in parameter_brace_expand_word
---
subst.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/subst.c b/subst.c
index 7ec8f8e9..a018b1b1 100644
--- a/subst.c
+++ b/subst.c
@@ -7507,8 +7507,6 @@ expand_arrayref:
? quote_string (temp)
: quote_escapes (temp);
rflags |= W_ARRAYIND;
- if (estatep)
- *estatep = es; /* structure copy */
}
/* Note that array[*] and array[@] expanded to a quoted null string by
returning the W_HASQUOTEDNULL flag to the caller in addition to TEMP.
*/
@@ -7517,7 +7515,9 @@ expand_arrayref:
else if (es.subtype == 2 && temp && QUOTED_NULL (temp) && (quoted &
(Q_DOUBLE_QUOTES|Q_HERE_DOCUMENT)))
rflags |= W_HASQUOTEDNULL;
- if (estatep == 0)
+ if (etastep)
+ *estatep = es; /* structure copy */
+ else
flush_eltstate ();
}
#endif
--
2.37.2
bash memory leak when querying associative array for nonexisting element
Configuration Information [Automatically generated, do not change]:
Machine: aarch64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables
-fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type
-flto=auto -g -D_GNU_SOURCE -DRECYCLES_PIDS -Wall -g -Wuninitialized -Wextra
-Wno-switch-enum -Wno-unused-variable -Wno-unused-parameter -Wno-parentheses
-ftree-loop-linear -pipe -DBNC382214=0 -DIMPORT_FUNCTIONS_DEF=0
uname output: Linux gate3 6.0.3-1-default #1 SMP PREEMPT_DYNAMIC Sat Oct 22
07:43:26 UTC 2022 (f00a35f) aarch64 aarch64 aarch64 GNU/Linux
Machine Type: aarch64-suse-linux-gnu
Bash Version: 5.2
Patch Level: 2
Release Status: release
Description:
[Detailed description of the problem, suggestion, or complaint.]
I want to read an associative array. Like so: y="${aa[$i]}"
If aa contains the element queried, then everything works as expected
If aa does not contain such an element, the result is the empty string,
which is expected, but the query permanently increases the memory consumption
of the executing bash, which is not expected. The program below demonstrates
this behaviour. When run, the bash process increases its memory indefinitely.
I did not find a fix.
The only workaround I could think of is to somehow avoid accessing
nonexistant keys, but this would induce an ugly programmatic overhead and
destroy the beauty of the associative arrays.
I think, the ability to write
if [ -z ${aa[$i]} ]; then
aa[$i]="new value"
fi
is something that really should be possible without causing an
ever-increasing memory footprint.
What is happening here? Unneccessarily increasing hash tables?
Repeat-By:
[Describe the sequence of events that causes the problem
to occur.]
### program begin ##
#!/bin/bash
# associative array
declare -A aa
# fill array with something. This makes the memory leak more pronounced.
for i in $(seq 1 1); do
aa[$i]="v$i"
done
# loop to show the memory leak
while true; do
# print the amount of memory this program currently uses
grep VmSize /proc/$$/status >/tmp/x
read x Mem x no memory
leak
key="xyz" # query nonexisting elements -> memory
leak
value="${aa[$key]}" # this is the evil line that
leaks memory
done
done
# program end #
pgpnhezgulpLI.pgp
Description: OpenPGP digital signature
Re: bash "extglob" needs to upgrade at least like zsh "kshglob"
2022年11月18日(金) 2:11 Chet Ramey : > "If a pattern ends with an unescaped , it is unspecified whether > the pattern does not match anything or the pattern is treated as invalid." > > Bash uses the former interpretation. If "the pattern is treated as invalid" > means trying to literally match the open bracket and going on from there, > your interpretation is valid as well. The standard doesn't use that > language in other places it specifies to treat the bracket as an ordinary > character to be matched literally, however. There seem to be still remaining issues. It is fine for me if Bash chooses the former, ``the pattern does not match anything'' with a backslash followed by NUL, but the following cases (see the attached [reduced3.sh]) with a backslash followed by a slash should still be fixed: #1: pat=a[b\/c] str=a[b/c] no/yes #2: pat=a[b\/c] str=ab no/no #3: pat=a[b\/c] str=ac yes/no [...] Where the fourth column shows the result of the current devel 407d9afc with FNM_PATHNAME (xxx) and the result I expect (yyy). "yes" means the pattern matches the string, and "no" means the pattern does not match. * I expect "yes" for #1 because the bracket expression contains a slash before its closing right bracket `]' and thus the beginning `[' should be matched literally. However, the actual behavior is "no". * I expect "no" for both #2 and #3 because the beginning bracket `[' should be matched literally. Even when an escaped slash would be allowed in the bracket expression so that [b\/c] forms a complete bracket expression, the results of #2 and #3 being "no" and "yes", respectively, are inconsistent. This difference is caused because the slash after the backslash is only checked after a matching character is found (lib/glob/sm_loop.c:703). The same check should be applied also before a matching character is found (lib/glob/sm_loop.c:573). I attach a patch for this [r0037.brackmatch6.remaining-slash.patch]. -- There is another related inconsistency. I just modified my new extglob engine to follow Bash's choice described above, but then the behavior became different from that of the actual implementation of Bash of the current devel. > "If a pattern ends with an unescaped , it is unspecified whether > the pattern does not match anything or the pattern is treated as invalid." > > Bash uses the former interpretation. The corresponding sentence in the POSIX standard describes the unescaped backslashes in the general context of the pattern instead of that in the bracket expression, so I applied this to the new extglob engine. However, ``the former interpretation'' that Bash adopts turned out to be only applied to the unescaped backslashes *inside a bracket expression*. This is the remaining part of the output of the attached [example3.sh] with the current devel 407d9afc: [...] #4: pat=a\ str=a\ yes/??? So the pattern terminated with unescaped backslash actually matches a string, where the backslash is treated as a literally-matching backslash. a. Is this difference between outside and inside of the bracket expressions intensional? I.e., the former interpretation "the pattern does not match anything" seems to only apply to the inside of bracket expressions. b. If this is the behavior for the unescaped backslashes outside the bracket expressions, which is intensionally different from those in the bracket expressions, would it be possible to change the treatment of the unescaped backslashes inside the bracket expression the same as that of outside so the bracket `[' matches literally (as expected in cases #28..#31 of my previous reply [1])? The attached [r0037.brackmatch7.unescaped-backslash-option-b.patch] is the corresponding patch. [1] https://lists.gnu.org/archive/html/bug-bash/2022-11/msg00070.html c. If the behavior of the unescaped backslash of the outside should also be modified to follow the former interpretation "the pattern does not match anything", another patch is [r0037.brackmatch7.unescaped-backslash-option-c.patch]. However, the current behavior outside the bracket expression seems to be explicitly required by the tests on tests/glob2.sub:32 and tests/glob2.sub:41. I prefer option b, which keeps the behavior required by tests/glob2.sub and also consistent between the inside and the outside of bracket expressions. It is also consistent with the behavior for the string end inside bracket expressions. -- Koichi From 828b93de72263785d93f86a285d919fdc5be156d Mon Sep 17 00:00:00 2001 From: Koichi Murase Date: Sun, 20 Nov 2022 16:09:09 +0900 Subject: [PATCH 1/2] fix(BRACKMATCH): fix remaining slash check --- lib/glob/sm_loop.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/glob/sm_loop.c b/lib/glob/sm_loop.c index fa350daa..151d10bd 100644 ---
