Re: [bash 4] 'test -v 1' is never true

[Alex Colomar]

Hi Martin,

On 11/27/22 17:53, Martin D Kealey wrote:
The expansion `$[var+replacement}` was part of the Bourne shell when I 
used it in 1985 (before POSIX was first published),


I didn't know that; interesting.  Still looks ugly/convoluted to me to 
use that feature here.  Might be useful if I didn't had another tool...


whereas `test A -gt 
B` is rather newer,


Being POSIX, it's old enough for my taste :)

and `test -v` is only a Bash extension (and 
similarly wasn't added until later).


Yeah, I started using that for consistency with other -v tests.  In 
fact, the first time I used it instead of $#, I wasn't sure if it would 
work.


I think $# wins for me.

Thanks!

Alex



The man and info pages have retained the rather terse descriptions of 
those oldest expansions, and they are buried in the sheer volume of the 
current documentation.



On Sun, 27 Nov 2022 at 23:00, Alejandro Colomar > wrote:


Hi Alexey,

On 11/27/22 12:41, Alexey wrote:
 > On 2022-11-26 21:45, Alejandro Colomar wrote:
 >> That was my gut; thanks!  I'll workaround with $#.
 >
 > I could suggest you to use for clarity another construction:
 > [[ ${1+isset} ]] || echo "not set"
 >
 > Here "isset" is just for readability. You could place any other
string literal
 > there.

I actually find that very confusing.  What feature is it using?  I
couldn't find
it with `info bash | less` then `/\+`.

      test $# -ge 1

has the benefit that it's just a sed(1) away from my old

      test -v 1

And it only needs the following to be understood:

      man test | grep -A1 ' -ge '
      man sh | grep '   # '

The old version (test -v 1) required:

      help test | grep ' -v '
      man sh | sed -n '/^   Positional/,/^$/p'

Both are easy to find.

 >
 > Regards,
 > Alexey.

Cheers,

Alex

-- 
>




--




OpenPGP_signature
Description: OpenPGP digital signature

Re: Bad leaks file fd to child processes

[Greg Wooledge]

On Mon, Nov 28, 2022 at 05:00:28PM -0500, Dale R. Worley wrote:
> Alexey via Bug reports for the GNU Bourne Again SHell 
> > Same behavior was in bash 4.4 (as well as now in bash 5.2):
> >
> > # echo $BASH_VERSION
> > 4.4.0(1)-release
> > # exec 66 > # pvs
> > File descriptor 66 (/etc/hosts) leaked on pvs invocation. Parent PID 
> > 1057606: ./bash
> >
> > But we use the fact tat bash doesn't close FD for example to preliminary 
> > open log file for utility that we will `exec' later.
> > Unfortunately bash doesn't provide any fcntl() mechanism to control FD 
> > flags.
> > It'll be great to have  ability to manage CLOEXEC or any other FD flags 
> > from the script.
> 
> Interesting!  I had misunderstood the complaint, which is that the fd
> leaks *from* bash *to* the program being executed (pvs in the example).
> 
> One aspect is that is the specified behavior of bash.

Given the snippet above, if it were in a script, one would assume that
passing the open FD 66 to the child process was the script author's
intent.  This is useful in a few niche cases.  A similar example that
comes to mind is DJB's tcpclient
 which runs a program with
file descriptors 6 and 7 pre-opened.

The fact that pvs *complains* about this instead of just ignoring it makes
it fairly unique.  I don't know why the authors of pvs chose to do this.
Perhaps they were working around some known or suspected bug in some other
program that was commonly used as pvs's parent.

Re: Bad leaks file fd to child processes

[Dale R. Worley]

Alexey via Bug reports for the GNU Bourne Again SHell 
> Same behavior was in bash 4.4 (as well as now in bash 5.2):
>
> # echo $BASH_VERSION
> 4.4.0(1)-release
> # exec 66 # pvs
> File descriptor 66 (/etc/hosts) leaked on pvs invocation. Parent PID 
> 1057606: ./bash
>
> But we use the fact tat bash doesn't close FD for example to preliminary 
> open log file for utility that we will `exec' later.
> Unfortunately bash doesn't provide any fcntl() mechanism to control FD 
> flags.
> It'll be great to have  ability to manage CLOEXEC or any other FD flags 
> from the script.

Interesting!  I had misunderstood the complaint, which is that the fd
leaks *from* bash *to* the program being executed (pvs in the example).

One aspect is that is the specified behavior of bash.

It's fairly straightforward to prevent a particular fd from leaking to a
single command by explicitly closing it in a redirection on the command:

# echo $BASH_VERSION
5.1.0(1)-release
# exec 66

Re: Localised variables become exposed to environment when FUNCNEST reached

[Chet Ramey]

On 11/27/22 10:55 AM, felix wrote:

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -g -O2
uname output: Linux medium 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 
(2022-10-21) x86_64 GNU/Linux
Machine Type: x86_64-pc-linux-gnu

Bash Version: 5.2
Patch Level: 12
Release Status: release

Description:
Localised variable in a function could alter environ when FUNCNEST is 
reached


Thanks for the report. FUNCNEST is intended more as a fatal error
mechanism than a way for scripts to do flow control.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: degraded error message in case of hash-bang interpreter error

[Chet Ramey]

On 11/28/22 8:13 AM, H.Janssen wrote:


Bash Version: 5.2
Patch Level: 9
Release Status: release

Description:
    If the interpreter after #! is wrong, a non-informative message is printed


Thanks for the report. This has been fixed in the devel git branch for a
while now.

--
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

degraded error message in case of hash-bang interpreter error

[H.Janssen]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS: -O2 -flto=auto -ffat-lto-objects -fexceptions -g 
-grecord-gcc-switches -pipe -Wall -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS 
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong 
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -m64 -mtune=generic 
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
uname output: Linux lap2.home 6.0.9-300.fc37.x86_64 #1 SMP 
PREEMPT_DYNAMIC Wed Nov 16 17:36:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Machine Type: x86_64-redhat-linux-gnu

Bash Version: 5.2
Patch Level: 9
Release Status: release

Description:
   If the interpreter after #! is wrong, a non-informative message is 
printed


Repeat-By:
    "unix2dos bashscript" starting with #!/bin/bash
    ./bashscript
     bash: ./bashscript: cannot execute: required file not found

 Before version 5.2:
    ./bashscript
    bash: ./bashscript: /bin/bash^M: bad interpreter: No such file or 
directory


Fix:
    Probably some code has to be moved in execute_cmd.c

Thanks a lot!


--
H.Janssen
Lekerwaard 36
1824HC Alkmaar
06-58971047

Re: bash "extglob" needs to upgrade at least like zsh "kshglob"

[Koichi Murase]

2022年11月28日(月) 19:51 Koichi Murase :
> The fix for PATSCAN (C) is included in the second patch
> [r0037.patscan1.parse_subbracket.patch.txt], which solves "Repeat-By 1
> and 4" by using PARSE_SUBBRACKET of the first patch. [...]

Sorry, there was an oversight in the second patch
[r0037.patscan1.parse_subbracket.patch.txt]. We need to also update
the function signatures of glob_patscan{,_wc} (expanded from PATSCAN)
in other files, gmisc.c and glob.c. I attach another patch
[r0037.patscan3.extern.patch.txt] for additional fixes to the second
patch, where I passed 0 as FLAGS so the original behavior is
preserved.

--
Koichi
From ddc72aeb2cd6962e85bdc0819419db3d8cb6576d Mon Sep 17 00:00:00 2001
From: Koichi Murase 
Date: Mon, 28 Nov 2022 21:42:14 +0900
Subject: [PATCH 1/2] fix(lib/glob/glob): update uses of glob_patscan

---
 lib/glob/glob.c  | 12 ++--
 lib/glob/gmisc.c |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/lib/glob/glob.c b/lib/glob/glob.c
index b66af85c..686d0f6b 100644
--- a/lib/glob/glob.c
+++ b/lib/glob/glob.c
@@ -127,8 +127,8 @@ static int glob_testdir PARAMS((char *, int));
 static char **glob_dir_to_array PARAMS((char *, char **, int));
 
 /* Make sure these names continue to agree with what's in smatch.c */
-extern char *glob_patscan PARAMS((char *, char *, int));
-extern wchar_t *glob_patscan_wc PARAMS((wchar_t *, wchar_t *, int));
+extern char *glob_patscan PARAMS((char *, char *, int, int));
+extern wchar_t *glob_patscan_wc PARAMS((wchar_t *, wchar_t *, wint_t, int));
 
 /* And this from gmisc.c/gm_loop.c */
 extern int wextglob_pattern_p PARAMS((wchar_t *));
@@ -207,7 +207,7 @@ extglob_skipname (pat, dname, flags)
   wild = *pat == '*' || *pat == '?';
   pp = pat + 2;
   se = pp + strlen (pp);   /* end of pattern string */
-  pe = glob_patscan (pp, se, 0);   /* end of extglob pattern */
+  pe = glob_patscan (pp, se, 0, 0);/* end of extglob pattern */
 
   /* if pe == 0, this is an invalid extglob pattern */
   if (pe == 0)
@@ -234,7 +234,7 @@ extglob_skipname (pat, dname, flags)
   nullpat = pe >= (pat + 2) && pe[-2] == '(' && pe[-1] == ')';
 
   /* check every subpattern */
-  while (t = glob_patscan (pp, pe, '|'))
+  while (t = glob_patscan (pp, pe, '|', 0))
 {
   /* If T == PE and *T == 0 (&& PE[-1] == RPAREN), we have hit the end
 of a pattern with no trailing characters. */
@@ -358,7 +358,7 @@ wextglob_skipname (pat, dname, flags)
   wild = *pat == L'*' || *pat == L'?';
   pp = pat + 2;
   se = pp + wcslen (pp);
-  pe = glob_patscan_wc (pp, se, 0);
+  pe = glob_patscan_wc (pp, se, 0, 0);
 
   /* if pe == 0, this is an invalid extglob pattern */
   if (pe == 0)
@@ -382,7 +382,7 @@ wextglob_skipname (pat, dname, flags)
   nullpat = pe >= (pat + 2) && pe[-2] == L'(' && pe[-1] == L')';
 
   /* check every subpattern */
-  while (t = glob_patscan_wc (pp, pe, '|'))
+  while (t = glob_patscan_wc (pp, pe, '|', 0))
 {
   n = t[-1];   /* ( */
   if (wextglob_pattern_p (pp) && n == L')')/* nested 
extglob? */
diff --git a/lib/glob/gmisc.c b/lib/glob/gmisc.c
index f3d74cea..24fdf746 100644
--- a/lib/glob/gmisc.c
+++ b/lib/glob/gmisc.c
@@ -38,7 +38,7 @@
 #include "glob.h"
 
 /* Make sure these names continue to agree with what's in smatch.c */
-extern char *glob_patscan PARAMS((char *, char *, int));
+extern char *glob_patscan PARAMS((char *, char *, int, int));
 
 /* Compile `gm_loop.c' for single-byte characters. */
 #define CHAR   char
@@ -92,7 +92,7 @@ glob_dirscan (pat, dirsep)
{
  if (se == 0)
se = p + strlen (p) - 1;
- pe = glob_patscan (p + 2, se, 0);
+ pe = glob_patscan (p + 2, se, 0, 0);
  if (pe == 0)
continue;
  else if (*pe == 0)
-- 
2.37.2

Re: bash "extglob" needs to upgrade at least like zsh "kshglob"

[Koichi Murase]

2022年11月23日(水) 5:24 Chet Ramey :
> I attached the latest patch against bash-5.2.9.


> commit 3c9dd4565792bc53de3a94ec38a65a1989f3fe2f (upstream/devel)
>
> associative array elements; last set of changes to globbing
> bracket expressions; fix for timing subshell commands

Thank you for the discussion and for applying the changes.  Besides, I
am sorry that I still have a discussion on the behavior of BRACKMATCH,
so it was not the last set of changes.  After the above fix, I moved
to check the behavior related to PATSCAN, where I found inconsistent
results related to the difference between BRACKMATCH and PATSCAN in
parsing the bracket expressions.  I checked also other parts of the
codebase and found additional inconsistencies.


Description
---

First, let me introduce the symbols (A)..(D) to later reference the
implementations of the bracket expression in the codebase.  There are
four independent codes that implement rules for extracting the bracket
expression in the current codebase:

- (A) The main loop of BRACKMATCH: This handles sub-expressions of a
  bracket expression when a matching sub-expression is not found.

- (B) The section of the `matched' label in BRACKMATCH: This handles
  sub-expressions of the bracket expression after a matching
  sub-expression is found.

- (C) PATSCAN: This skips bracket expressions to determine the end of
  the extglob constructs of the form @(...), ?(...), +(...), etc.

- (D) MATCHLEN (lib/glob/gm_loop.c): This function handles bracket
  expressions to count the number of characters that a fixed-length
  pattern can match.

Actually, each of the four implements a distinct rule, which does not
match any of the other three.  These implementations need to be
adjusted to support an identical and consistent rule.


Repeat-By
-

The differences between (A)..(D) cause various strange behaviors.

1. Strange behavior caused by an inconsistency between (A/B) and (C)

  This is what I was first faced with.  The following shows the result
  of [example4.sh] with the current devel, where column 4
  `{yes,no}/{yes,no}' shows `(result)/(what I expect)':

  --- PATSCAN vs BRACKMATCH ---
  #1: pat=@([[.].])A]) str=]no/yes
  #2: pat=@([[.].])A]) str===]A])   no/no
  #3: pat=@([[.].])A]) str=AA]) yes/no
  #4: pat=@([[=]=])A]) str=]no/no
  #5: pat=@([[=]=])A]) str===]A])   no/yes
  #6: pat=@([[=]=])A]) str=AA]) yes/no

  Obvious strange behaviors can be found in cases #3 and #6, where `A'
  matches twice even if there is only one `A' and no repetition such
  as `*()' or `+()' in the pattern.  This is because PATSCAN (C)
  considers the bracket expression to be `[[.].]' while BRACKMATCH
  (A/B) considers the bracket expression to be `[[.].])A]'.  First,
  PATSCAN extracts `@([[.].])', but BRACKMATCH next matches the first
  `A' in the input string using a pattern character `A' outside the
  construct `@()'.  Finally, the remaining part `A])' in the pattern
  is matched literally.

2. Inconsistency between (A) and (B):

  To fix the above item for (A/B) vs (C), I checked the detailed
  behaviors of both and found this.  The parsing of [.ch.], [=a=], and
  [:var:] are not totally consistent before and after a matching
  sub-expression is found.  The following is the second section of the
  result of [example4.sh]:

  --- BRACKMATCH: after match vs before match ---
  #7: pat=[[=]=]ab]str=ayes/no
  #8: pat=[[.[=.]ab]   str=ayes/yes
  #9: pat=[[.[==].]ab] str=ayes/yes

  #10: pat=[a[=]=]b]str=ano/no
  #11: pat=[a[.[=.]b]   str=ano/yes
  #12: pat=[a[.[==].]b] str=ano/yes

  #13: pat=[a[=]=]b]str=byes/no
  #14: pat=[a[=]=]b]str=a=]b]yes/yes
  #15: pat=[a[.[=.]b]   str=byes/yes
  #16: pat=[a[.[=.]b]   str=ab]  yes/no
  #17: pat=[a[.[==].]b] str=byes/yes
  #18: pat=[a[.[==].]b] str=ab]  yes/no

  Cases #7..#9 succeeds, which means that `[=]=]', `[.[=.]', and
  `[.[==].]' form an equivalence class and collating symbols in
  BRACKMATCH (A).  However, cases #10..#12 (which are the bracket
  expressions of the same sub-expression with different ordering)
  fail, which means that `[=]=]', `[.[=.]', and `[.[==].]'  do not
  form an equivalence class or a collating symbol in BRACKMATCH (B).

  Also, cases #13 vs #14, #15 vs #16, and #17 vs #18 demonstrate that
  the same pattern consisting of bracket expressions and normal
  characters can match different numbers of characters.  This means
  that the boundary of the bracket expression can change depending on
  the input string.

  Actually, these patterns are undefined by the standard because an
  equivalence class shall 

Re: Bad leaks file fd to child processes

[Alexey via Bug reports for the GNU Bourne Again SHell]

On 2022-11-28 14:20, Koichi Murase wrote:

2022年11月28日(月) 19:06 Alexey via Bug reports for the GNU Bourne Again
SHell :
But we use the fact tat bash doesn't close FD for example to 
preliminary

open log file for utility that we will `exec' later.
Unfortunately bash doesn't provide any fcntl() mechanism to control FD
flags.
It'll be great to have  ability to manage CLOEXEC or any other FD 
flags

from the script.


There is a loadable builtin `fdflags' to do that. If the loadable
builtin is installed (e.g. at /usr/lib/bash/fdflags), we can do
something like

enable -f /usr/lib/bash/fdflags fdflags
fdflags -s +cloexec 

--
Koichi


Many thanks for the example.
But, I still have a lot of prod with bash 4.4 (I guess I could build 
fdflags for 4.4 myself)


Regards,
Alexey

Re: Bad leaks file fd to child processes

[Koichi Murase]

2022年11月28日(月) 19:06 Alexey via Bug reports for the GNU Bourne Again
SHell :
> But we use the fact tat bash doesn't close FD for example to preliminary
> open log file for utility that we will `exec' later.
> Unfortunately bash doesn't provide any fcntl() mechanism to control FD
> flags.
> It'll be great to have  ability to manage CLOEXEC or any other FD flags
> from the script.

There is a loadable builtin `fdflags' to do that. If the loadable
builtin is installed (e.g. at /usr/lib/bash/fdflags), we can do
something like

enable -f /usr/lib/bash/fdflags fdflags
fdflags -s +cloexec 

--
Koichi

Re: Bad leaks file fd to child processes

[Alexey via Bug reports for the GNU Bourne Again SHell]

On 2022-11-26 14:05, "凋叶棕" via Bug reports for the GNU Bourne Again 
SHell wrote:

Hello,

I find that the file descriptor leaks when I execute the command pvs
in bash 5.2, The abnormal scenario is similar to the bug which
reported by
http://lists.gnu.org/archive/html/bug-bash/2017-01/msg00026.html

When I execute pvs in the terminal opened through xshell，it’s ok

PV VG  Fmt  Attr PSizePFree
/dev/sda2  euleros lvm2 a--  <126.00g0

But when I execute pvs in the terminal opened through vscode（use
Remote-SHH to connect linux）, the File descriptor was leaked

File descriptor 19 (/dev/ptmx) leaked on pvs invocation. Parent PID
3789: /usr/bin/bash
File descriptor 20 (/dev/ptmx) leaked on pvs invocation. Parent PID
3789: /usr/bin/bash
File descriptor 21 (/dev/ptmx) leaked on pvs invocation. Parent PID
3789: /usr/bin/bash
File descriptor 99
(/root/.vscode-server/bin/899d46d82c4c95423fb7e10e68eba52050e30ba3/vscode-remote-lock.root.899d46d82c4c95423fb7e10e68eba52050e30ba3)
leaked on pvs invocation. Parent PID 3789: /usr/bin/bash
  PV VG  Fmt  Attr PSizePFree
  /dev/sda2  euleros lvm2 a--  <126.00g0

I found that the leaked file descriptor was created in subst.c

/* Pipe the output of executing STRING into the current shell. */
if (pipe(fildes) < 0)
{
sys_error("%s", _("cannot make pipe for command substitution"));
goto error_exit;
}

add it leaked in

if (--bufn <= 0)
{
SET_CLOSE_ON_EXEC(fd);
bufn = zread(fd, buf, sizeof(buf));
if (bufn <= 0)
break;
bufp = buf;
}

I try to add SET_CLOSE_ON_EXEC (fd) before zread, but it doesn't seem 
to work.


Same behavior was in bash 4.4 (as well as now in bash 5.2):

# echo $BASH_VERSION
4.4.0(1)-release
# exec 66File descriptor 66 (/etc/hosts) leaked on pvs invocation. Parent PID 
1057606: ./bash


But we use the fact tat bash doesn't close FD for example to preliminary 
open log file for utility that we will `exec' later.
Unfortunately bash doesn't provide any fcntl() mechanism to control FD 
flags.
It'll be great to have  ability to manage CLOEXEC or any other FD flags 
from the script.


Regards,
Alexey.