A question about bash change for efficiency speedups in multibyte locales

[yangyajing]

Hello,  I recently did lsb-test-core on ubuntu 16.04. There is a test
that can be successful in bash 4.1 but fails in bash 4.3.

The test comment is :  bash tp10.sh.

The content of tp10.sh is :   echo [[:xdigit:]] [[:hyphen:]].

And this test sets a multibyte locale that defines a charclass
containing ''hyphen''.

When I run the tp10.sh script with bash, I find that ''hyphen'' can be
recognized in bash4.1 but not in bash4.3.

By comparing the source code, I found that compared to bash4.1, there
are 2 more lines of code in bash4.3, which in the xstrmatch() function
in bash/lib/glob/smatch.c.
The two lines of code are as follows:


When I checked the change log at
https://tiswww.case.edu/php/chet/bash/CHANGES, I guess this change may
be for efficiency speedups.

So I want to confirm with you whether my guess is correct, and whether
this change ignores the fact that pattern does not have a multibyte
character but is defined in the charclass in multibyte locales.

Looking forward to your reply.

File completion scans unrelated directory

[Bart Schaefer]

Configuration Information [Automatically generated, do not change]:
Machine: x86_64
OS: linux-gnu
Compiler: gcc
Compilation CFLAGS:  -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64'
-DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu'
-DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale'
-DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H   -I.  -I. -I./include
-I./lib   -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=4
-grecord-gcc-switches   -m64 -mtune=generic
uname output: Linux g001-ftp.us-west-2.elasticbeanstalk.com
4.4.19-29.55.amzn1.x86_64 #1 SMP Mon Aug 29 23:29:40 UTC 2016 x86_64
x86_64 x86_64 GNU/Linux
Machine Type: x86_64-redhat-linux-gnu

Bash Version: 4.2
Patch Level: 46
Release Status: release

Description:
For reasons I hope I don't need to go into, I have a directory with
275k files in it on an NFS volume.  This is mounted under /shared/.

Everything else on the host is on the local disk.

When I attempt to file-complete a path in /tmp, e.g.

# ls /tmp/b

bash goes off running getdents() on everything in /shared/:

strace: Process 1553 attached
getdents(3, /* 514 entries */, 32768)   = 32752
getdents(3, /* 513 entries */, 32768)   = 32712
getdents(3, /* 512 entries */, 32768)   = 32744
getdents(3, /* 512 entries */, 32768)   = 32712
getdents(3, /* 510 entries */, 32768)   = 32712
getdents(3, /* 515 entries */, 32768)   = 32768
getdents(3, /* 514 entries */, 32768)   = 32720
(etc. for many minutes)

Why is completion scanning /shared/ when the file I care about is in /tmp/ ?

There is no way to interrupt bash while this is going on, so my shell
session is effectively jammed.

Repeat-By:
See description.

Re: command_not_found_handle strange behaviour

[Eduardo A . Bustamante López]

On Wed, Mar 28, 2018 at 06:42:21PM +0300, Кириллов Дима wrote:
[...]
> Bash Version: 4.4
> Patch Level: 19
> Release Status: release
> 
> Description:
>   I can't understand why read works without subshell before it, 
> but with subshell before read it stops
> 
> command_not_found_handle()
> {
> (true) # subshell call
> read line
> echo "$line"
> }
> 
>   bash-4.4$ foo
>   
>   [1]+  Stopped foo
> 
> And without subshell call it works fine:

I can reproduce this.


A few observations:

(1) `read' is stopped because it's in a different process group (i.e. it's a
background process), and background process that attempt to read from the
controlling terminal receive a SIGTTIN from the kernel.

(2) The subshell is a red herring. You can replace it with anything that causes
bash to fork a new process, e.g. /bin/true

(3) This behaviour was apparently introduced in bash-20151113 snapshot
(f9b024c839a3bbb9c6c2a98a16b1cf362010340a), although due to the nature of that
change, I suspect this issue has been present for far more time, and it was just
"uncovered" by this commit.

(4) This only affects interactive shells.

(5) The command_not_found_handle executes in a child process, i.e. the process
that was intended to be used by the not found program. Apparently it also
executes in its own process group, so it means that it should consistently
behave as a background process. 

Re: Strange behaviour when sending SIGTERM to bash

[Chet Ramey]

On 3/28/18 12:52 PM, Martin Schulte wrote:
> Hello,
> 
> I'm running bash under Debian Stretch:
> 
> $ echo $BASH_VERSION 
> 4.4.12(1)-release
> $ trap
> $ echo $$
> 1100
> $ kill 1100
> $ 
> 
> => Looks fine for me
> 
> Then I do a
> 
> kill 1100
> 
> from another session and the bash above terminates.
> 
> => Looks wrong for me

It's readline. The difference between your two cases is that in the second
one, readline gets the signal, and it interrupts the read. Readline handles
the signal, calls the bash signal handler (which does nothing but set a
flag and return), and returns a read error. You really have to return the
error so the signal gets handled properly -- all the signal handler is
allowed to do is to set a flag, and that's all the readline and bash signal
handlers do.

The idea is that terminal emulators (at least in 2011) send SIGTERM or
SIGHUP when you use the close button, and unless readline returns an error,
you get processes looping on a stream of read(2) returning EINTR or EIO.

There is a related discussion beginning at

http://lists.gnu.org/archive/html/bug-bash/2011-03/msg00054.html

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: proper bash configure behavior when /dev/stdin is present, /dev/{,self/}fd are not?

[Chet Ramey]

On 3/28/18 5:59 PM, Brian Carnes wrote:
> When configuring bash on a self-hosted QNX 6.6 or QNX7 system:
> 
> This is an OS that has /dev/stdin and friends, but does not have /dev/fd.
> 
> When blind cross-compiling bash for QNX, one would get a guess answer of:
>   bash_cv_dev_fd=standard
>   bash_cv_dev_stdin=present
> 
> When doing the safer self-hosted native configure, we get several
> significant improvements, but also:
>   bash_cv_dev_fd=absent
>   bash_cv_dev_stdin=absent
> 
> When "the truth" for QNX should really be (we think):
>   bash_cv_dev_fd=absent
>   bash_cv_dev_stdin=present
> 
> The odd configure output is due to the test having the presence of
> /dev/stdin guarded by testing for the existence of /dev/fd or /dev/self/fd:
> 
>   if *test -d /dev/fd* && (exec test -r /dev/stdin < /dev/null) ; then
>bash_cv_dev_stdin=present
>  elif *test -d /proc/self/fd* && (exec test -r /dev/stdin < /dev/null) ;
> then
>bash_cv_dev_stdin=present
>  else
>bash_cv_dev_stdin=absent
>  fi

This test dates from 1999. It's quite possible that the assumptions valid
at that time are no longer so, like every implementation of /dev/stdin
making it a symlink to /dev/fd/0.


> At first glance it appears that:
>   lib/sh/eaccess.c
>   redir.c

To emulate the filenames with consistent semantics on systems that can't
be guaranteed to provide it.

> So, can those in the know weigh in on which interpretation below is correct:

None of these is relevant unless you want /dev/stdin treated as different
from file descriptor 0 in some way. If your semantics for /dev/stdin differ
(e.g., if a stat on /dev/stdin returns something different from an fstat on
file descriptor 0), then the distinction matters. Is there something
special that happens, say, when you open /dev/stdin that is different from
dup'ing file descriptor 0?

Unless you have special semantics, I don't think it matters.

> 
> (a) The DEV_STDIN code block isn't relevant unless you're in the presence
> of DEV_FD, so it's harmless and appropriate to have the current configure
> output.
> 
> (b) The dependency is there for good reason, and we're missing what will
> break in the presence of DEV_FD=no + DEV_STDIN=yes

I recall an old system (FreeBSD maybe?) that made /dev/fd a loadable
file system, shipped with /dev/stdin a symlink to /dev/fd/0, and could
get itself in a state that testing /dev/stdin would pause until someone
(auto-?) mounted /dev/fd. It's been 19 years.

> (d) The DEV_FD -> DEV_STDIN dependency should be removed from the test
> completely, as a patch for all environments?

I'd be ok with this one as long as it doesn't cause problems. Can anyone
think of a system in common use today where it would?

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Extra parenthesis in documentation

[Jorge]

Under 6.3.3 Interactive Shell Behavior

In the absence of any traps, |SIGINT| is caught and handled *((*see
Signals ).