possible compgen bug
HI ! problem: compgen assigns first option on receiving an empty string as input (i.e. by read -t 5 VAR timing out) systems and OS: X86 (Duron and Sempron) Debian etch: [EMAIL PROTECTED]:~/shell$ uname -a Linux backup 2.6.18-hpt374 #4 SMP Thu Mar 20 00:07:28 CET 2008 i686 GNU/Linux [EMAIL PROTECTED]:~/shell$ bash --version GNU bash, version 3.1.17(1)-release (i486-pc-linux-gnu) Copyright (C) 2005 Free Software Foundation, Inc. Slackware 10.0: [EMAIL PROTECTED]:~/fipsi# uname -a Linux rtl14 2.4.26 #6 Mon Jun 14 19:07:27 PDT 2004 i686 unknown unknown GNU/Linux [EMAIL PROTECTED]:~/fipsi# bash --version GNU bash, version 2.05b.0(1)-release (i486-slackware-linux-gnu) Copyright (C) 2002 Free Software Foundation, Inc. both systems show the same (supposedly wrong) behavior of the compgen builtin function. corect behavior: when using read with a timeout and then evaluating a variable I get the following expected behavior: #! /bin/bash read -p timeout 5 -t 5 CMD if [ $CMDx == exitx ] ; then echo exiting $CMD elif [ $CMDx == quitx ] ; then echo quite not a valid command calling exit with -1 exit -1 else echo invalid command - exiting with -2 exit -2 fi which gives me the expected response when it times out due to no input [EMAIL PROTECTED]:~/shell$ ./1.sh timeout 5 invalid command - exiting with -2 [EMAIL PROTECTED]:~/shell$ probably errornous behavior: when using compgen to allow shortcuts (in the below example for quit and exit I get a strange behavior when no input is entered and write times out after 5 seconds: #! /bin/bash read -p timeout 5 -t 5 FOO CMD=( $(compgen -W quit exit -- $FOO) ) if [ $CMDx == exitx ] ; then echo exiting $CMD elif [ $CMDx == quitx ] ; then echo quit not a valid command calling exit with -1 exit -1 else echo invalid command - exiting with -2 exit -2 fi [EMAIL PROTECTED]:~/shell$ ./2.sh timeout 5 quit not a valid command calling exit with -1 [EMAIL PROTECTED]:~/shell$ it seems to expand CMD to quit - not the behavior I expected I assume that this is a bug in compgen as it indentifies an empty string with the first word in the options passed - I would have expected it to be a null-string (just as if one would type x, which has no match, thus the null-string is assigned to CMD) thx! hofrat
load small history file after append breaks further appending
I've been implementing something in my shell where I'm using multiple history files, and I switch between them. This changes the shell's identity, to basically swap its history file with another. This is being used to have per-project histories. The procedure I am using to switch is something like this: history -a # flush anything left history -c # start anew HISTFILE=newhist# prepare to load this one history -r # and do it (identities changed) however I have also tried this: history -a history -cr newhist HISTFILE=newhist At this point we should be able to enter new commands, and when we decide to, history -a (append them to the `newhist' file). However, bash fails to do this in the case that the new history file is either very small, or new, i.e. zero bytes. I believe I have tracked down the reason this is happening: `bash' keeps the notion of how many lines it has inserted into the history since the history file was read (see `bashhist.c:history_lines_this_session'). This amount is zeroed after a successful append. If this amount ever becomes greater than or equal to the size of the new history that has been read (which is what is returned from the global `history_offset' in `history.c:where_history()'), bash will never allow the append to succeed again (see the check on whether history should be appended or not, which can be found in `bashhist.c:maybe_append_history()'). BUG: This situation arises if there are a few commands in between the successful append to the old history, and the import of the new history, IFF the new history is smaller than said number. BUG: This happens also for NEW histories. Even with a history -cr (resulting in `history_lines_this_session' value of `1') instead of history -c; history -r (`history_lines_this_session' value of `2'), the value is still greater than zero, which is the size of a new history file just read, so it will NEVER work again and all subsequent history will NOT be written with a history -a, which silently fails to append data when it should. Note that on the append which occurs at shell exit, the data WILL be appended, for two reasons: (1) the test in `maybe_append_history()': history_lines_this_session where_history() differs from that in `maybe_save_shell_history()': history_lines_this_session = where_history() ^^ (2) the history is appended unconditionally anyways if `force_append_history' is set (which corresponds to `shopt -s appendhist'), whereas no such condition is checked for in `maybe_append_history()'. I am wondering, why not zero `history_lines_this_session' upon a successful history -c ? As it is, clearing the history does not change the number at all and I can keep entering commands and clearing them all day; `history_lines_this_session' continues to rise. What reason is there to keep this variable with a positive value after all history settings are cleared? There's nothing to write. The following demonstrates the problem: $ id -un # starting as self smcdermott $ sudo -u test rm -f ~test/.bash_history # give test account $ sudo -u test touch ~test/.bash_history # empty history $ sudo su - test # become test user $ wc -l $HISTFILE# was indeed empty 0 /home/test/.bash_history $ shopt -s histappend# force append on exit $ logout $ $ sudo su - test # now go back and load $ wc -l $HISTFILE# 2 commands appended 2 /home/test/.bash_history $ cp $HISTFILE $HISTFILE.new # which we stow $ history -a # note append works $ wc -l $HISTFILE 5 /home/test/.bash_history $ history -cr $HISTFILE.new # clear, load stowed 2 $ HISTFILE=$HISTFILE.new # switch to stow file $ test 1 # add a few commands $ test 2 # that should be $ test 3 # appended $ history -a # like so $ wc -l $HISTFILE# but they're not 2 /home/test/.bash_history.new Now note when I add ONE extra command, otherwise unchanged: --- $ id -un smcdermott $ sudo -u test rm -f ~test/.bash_history $ sudo -u test touch ~test/.bash_history $ sudo su - test $ wc -l $HISTFILE 0 /home/test/.bash_history $ shopt -s histappend $ test 1 # --- here makes 3 to save $ logout $ sudo su - test $ wc -l $HISTFILE 3 /home/test/.bash_history $ cp $HISTFILE $HISTFILE.new $ history -a $ wc -l $HISTFILE 6 /home/test/.bash_history $ history -cr $HISTFILE.new $ HISTFILE=$HISTFILE.new $ test 1 $
builtin echo buffers failed writes
This is correct (external echo): $ /bin/echo hello world /dev/null 10 /bin/echo: write error: Bad file descriptor $ /bin/echo goodbye world goodbye world $ This is not (builtin echo): $ echo hello world /dev/null 10 bash: echo: write error: Bad file descriptor $ echo goodbye world hello world goodbye world $ The second echo printed the data that failed to get printed by the first one! I am using the bash preinstalled on Ubuntu Gutsy. I have not checked whether it is fixed in later versions: $ bash --version GNU bash, version 3.2.25(1)-release (i486-pc-linux-gnu) Copyright (C) 2005 Free Software Foundation, Inc. $ Russ Cox
Re: set -e's bug
Hiroshi Fujishima wrote: Configuration Information [Automatically generated, do not change]: Machine: i386 OS: freebsd7.0 Compiler: cc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='i386' -DCONF_OSTYPE='freebsd7.0' -DCONF_MACHTYPE='i386-portbld-freebsd7.0' -DCONF_VENDOR='portbld' -DLOCALEDIR='/usr/local/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -I/usr/local/include -O2 -fno-strict-aliasing -pipe uname output: FreeBSD yamato.tonteki.org 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb 24 19:59:52 UTC 2008 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 Machine Type: i386-portbld-freebsd7.0 Bash Version: 3.2 Patch Level: 39 Release Status: release Description: Executing the following script. It exits at `until command false'. set -e until command false; do break done echo ok Thanks for the report. This has been fixed for the next version. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer Live Strong. No day but today. Chet Ramey, ITS, CWRU[EMAIL PROTECTED]http://cnswww.cns.cwru.edu/~chet/
Re: Memory leak when catting(/sedding/...) large binary files with backticks
[EMAIL PROTECTED] wrote: Configuration Information [Automatically generated, do not change]: Machine: x86_64 OS: linux-gnu Compiler: gcc Compilation CFLAGS: -DPROGRAM='bash' -DCONF_HOSTTYPE='x86_64' -DCONF_OSTYPE='linux-gnu' -DCONF_MACHTYPE='x86_64-redhat-linux-gnu' -DCONF_VENDOR='redhat' -DLOCALEDIR='/usr/share/locale' -DPACKAGE='bash' -DSHELL -DHAVE_CONFIG_H -I. -I. -I./include -I./lib -D_GNU_SOURCE -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic uname output: Linux pmpc983.npm.ac.uk 2.6.24.4-64.fc8 #1 SMP Sat Mar 29 09:15:49 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux Machine Type: x86_64-redhat-linux-gnu Bash Version: 3.2 Patch Level: 33 Release Status: release Description: Using echo `cat ...` on a large binary file causes lots of memory to be used (fine), but if you ctrl-c while it's running it doesn't die properly and doesn't return used memory when finished. Originally found by screwing up a sed command (can also reproduce bug using sed rather than cat) while trying to rename a group of files. Repeat-By: Every time 1. Find large binary data file for test (mine is ~3.2GB) 2. echo `cat filename` 3. Ctrl-C previous command while running (doesn't terminate) 4. When step 2 eventually returns it does not release memory I'm not sure what you mean by `doesn't return used memory', but if you mean a process's size as reported by ps or similar, that does not indicate a memory leak. A memory leak is memory that has been allocated by a program to which it retains no handles. malloc acts as a cache between an application and the kernel. Memory obtained from the kernel using malloc may, under some circumstances, be returned to the kernel upon free, but this may not always be possible. Memory that is not returned to the kernel by freeing pages or using sbrk with a negative argument is retained and used to satisfy future requests. I ran your test using valgrind to check for memory leaks (but with only a 330 MB file), and it reported no leaks after ^C. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer Live Strong. No day but today. Chet Ramey, ITS, CWRU[EMAIL PROTECTED]http://cnswww.cns.cwru.edu/~chet/
Re: builtin echo buffers failed writes
Russ Cox wrote: This is correct (external echo): $ /bin/echo hello world /dev/null 10 /bin/echo: write error: Bad file descriptor $ /bin/echo goodbye world goodbye world $ This is not (builtin echo): $ echo hello world /dev/null 10 bash: echo: write error: Bad file descriptor $ echo goodbye world hello world goodbye world $ The second echo printed the data that failed to get printed by the first one! I am using the bash preinstalled on Ubuntu Gutsy. I have not checked whether it is fixed in later versions: This is actually dependent on your stdio implementation. Linux versions exhibit the above behavior, others (BSD, for example), do not. Future versions of bash will use fpurge(3) to make sure that stdio output buffers are emptied when changing the underlying file descriptor. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer Live Strong. No day but today. Chet Ramey, ITS, CWRU[EMAIL PROTECTED]http://cnswww.cns.cwru.edu/~chet/
Re: export -p may output invalid variables
Stephane Chazelas wrote: Bash Version: 3.2 Patch Level: 33 Release Status: release Description: ~$ env -i '1=a' sh -c 'export -p' export 1=a export OLDPWD export PWD=/home/chazelas export SHLVL=1 ~$ env -i '1=a' bash -c 'export -p' declare -x 1=a declare -x OLDPWD declare -x PWD=/home/chazelas declare -x SHLVL=1 ~$ env -i '1=a' sh -c 'export -p' | sh sh: line 1: export: `1=a': not a valid identifier ~$ env -i '1=a' bash -c 'export -p' | bash bash: line 1: declare: `1=a': not a valid identifier Thanks for the report. The right fix is to ignore words in the environment that are not valid shell assignment statements. Bash already does this for words without an `=', for instance. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer Live Strong. No day but today. Chet Ramey, ITS, CWRU[EMAIL PROTECTED]http://cnswww.cns.cwru.edu/~chet/
