[RFC] Logically composable variant of errexit
Hi all,
the errexit option can be very useful in simple scripts. This option
is being ignored in many contexts like lists and conditionals though.
I understand that this is by design and that errexit cannot be
fixed to behave more reasonably. Still, this makes bash a lot less
useful than it could be; working around this limitation is painful,
ugly, and leads to fragile code.
So, since we cannot fix errexit, can we maybe introduce another
option like errfail that behaves like errexit for simple commands,
but is also logically composable? Let me show what I mean with the
following pseudo-code:
set -o errfail
fail() {
false
echo 'oops!' 2 # not reached
}
! fail
fail || :
if fail; then
:
fi
set -- `fail` # script fails here
echo 'oops!' 2 # not reached
Having such an option would certainly make bash a lot more useful to me.
What do you guys think, could such an option be implemented with
reasonable effort?
Thanks,
Andreas
Re: umask --help
On 2 Sep 2014 16:48, Chet Ramey chet.ra...@case.edu wrote: On 9/2/14, 5:37 AM, Notes Jonny wrote: Hi Guys Any update on implementing this? I haven't done anything with this yet. If and when it happens, it will show up in the devel git branch on savannah. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRU c...@case.edu http://cnswww.cns.cwru.edu/~chet/ Thank you for your reply. I understand that requests can't be implemented immediately. Is there anything I could do to get this implemented? Eg. I would offer to pay £100 for an engineer to complete the updates. Regards Jonny
Re: [RFC] Logically composable variant of errexit
Andreas Grünbacher wrote:
Hi all,
the errexit option can be very useful in simple scripts. This option
is being ignored in many contexts like lists and conditionals though.
I understand that this is by design and that errexit cannot be
fixed to behave more reasonably. Still, this makes bash a lot less
useful than it could be; working around this limitation is painful,
ugly, and leads to fragile code.
So, since we cannot fix errexit, can we maybe introduce another
option like errfail that behaves like errexit for simple commands,
but is also logically composable? Let me show what I mean with the
following pseudo-code:
I think you should explicitely state where and you would like it not to
do it. I find unlikely to develop it, but at least they could be
documented, with its workarounds.
On some cases with non-POSIX features, maybe the behavior could be
changed for some non-POSIX (eg. the change of arighmetic expansion on
bash-4.1)
For reference, the current doc says:
if the
failed command is part of the command list immediately following
an `until' or `while' keyword, part of the test following the `if'
or `elif' reserved words, part of a command executed in a `' or
`||' list except the command following the final `' or `||', any
command in a pipeline but the last, or if the command's return
status is being inverted using `!'
set -o errfail
fail() {
false
echo 'oops!' 2 # not reached
}
! fail
fail || :
if fail; then
:
fi
set -- `fail` # script fails here
echo 'oops!' 2 # not reached
This seems to request a change on two cases:
- inherit the errexit flag into functions
- make shell expansions fail the command including them
For the first one, I don't see a spec reason for that, other than it
has been done like that, requiring an explicit set -o errfail. Maybe
the set options should be inherited for set -E
While we discuss it, I find odd that this code doesn't run the trap
inside a function but does outside:
fail() {
trap echo trap executed ERR
false
}
As for the second option, I also missed such a ability a while back, but
was able to get the expected behavior with the workaround:
FAIL=$(fail)
set -- $FAIL
Having such an option would certainly make bash a lot more useful to me.
What do you guys think, could such an option be implemented with
reasonable effort?
I don't think it would require special effort, but IMHO adding a
_slightly different errexit_ wouldn't be appropiate. Changing errexit
semantic, has a big potential for broken scripts. And having readily
available workarounds…
Re: Testing for Shellshock ... combinatorics and latest(Shellshock) Bash Vulnerability...(attn: Chet Ramey)
Wow, that's a very long email. While it's possible to fuzzy test bash, the problem is, first, you have to find a way to generate strings that maximize the chance of being a genuine command or a command that triggers a bug. This is very expensive... Second, once you generate a command, how will your test program know if it found a bug? It's easy when bash segfaults, but in the case of shellshock, it wasn't a crash. It was rather that bash was executing code where it shouldn't. Not even humans were able to tell that for more than 20 years. I'm sure Stephane wasn't the first one to notice that, but he was the first one to realize the huge problem that this represented. Also, you have to find a way to isolate your machine/OS from the testing, because bash has access to the file system, and other things, that could destroy the operating system in the case of a command gone wrong. (I suggest looking at 'shbot' for this particular issue). Other than that, I guess, good luck. I also played with this idea for some time in my mind, but, didn't bother implementing it, because of the second point, mainly. I guess you could feed your command generator a large corpus of scripts, and use a Markov model to generate sequences of commands, that could be better than the combinatorics.
Re: Random loss of bash history
Darshit Shah wrote: I'm running Bash 4.3.30(1)-release on Arch Linux. Recently after I rebooted by system I realized that all my Bash History had been erased. The ~/.bash_history file existed, but it was completely empty. The situation occurred just now once again. I boot up my system and realize that I have no bash history at all. The issue does not occur at every boot, it's random and I am unable to correlate the two instances in which this has happened to me. I haven't any idea as to how I should debug this problem to provide more information. Could you please start out by helping me there? What type of file system is your ~/.bash_history located upon? This sounds like a deja vu from the ext4 file system. Do you observe any other zero sized files after a reboot? Bob
Re: Testing for Shellshock ... combinatorics and latest(Shellshock) Bash Vulnerability...(attn: Chet Ramey)
On 10/09/2014 08:46 PM, Rick Karcich (rkarcich) wrote: Hello Chet, Re: testing for Shellshock... would like your feedback... specifically, regarding the possibility of human-directed combinatorial testing to find this Bash vulnerability... Sounds like how Michal Zalewski found the related CVE-2014-6278 http://lcamtuf.blogspot.ie/2014/10/bash-bug-how-we-finally-cracked.html Pádraig.
Re: bash uses tmp files for inter-process communication instead of pipes?
Greg Wooledge wrote: there is a file named /etc/udev/rules.d/70-persistent-net.rules Perhaps you should get in touch with a SuSE mailing list and see how other SuSE users have dealt with this. You might be reinventing the wheel. --- SuSE has had the same file. At first we were told port name customization was going away. That didn't go well, so they decided to continue to support the udev rules for this next release. I.e. -- it was going to be removed this release, but but that decision got reversed this time. I figure it's just a matter of time, like other things they wanted to get rid of. Seems the systemd people may put in a feature to deal with this, but they really would rather people use hardware-busmapped based names like p5p1 p5p2 p3p1 p3p2, that would changed if you changed from one slot to another (those are what I get for eth[0145]). So... think I got in to the current system in some early incarnation of it. It left my old interfaces the same, but new interfaces were added according to the new scheme with no easy way to get old behavior back. After manually renaming them, they stayed, but again... for how long? I figured I should have a script that would set/reset them to consistent values so I would have to deal with variable new release policies. Even using basic color and status scripts from their process is problematic now, as it has checks to no longer call scripts -- but systemd, if it's configs tell it to -- i.e. a status include file was upgraded to forward some requests to systemd, while others were replaced by systemd commands. Nice status.
Re: Bash 4.3 with patch 29 causing segfault during ./configure when trying to upgrade to patch 30
On 10/8/14, 8:17 PM, jon wrote: dmesg. [307688.764489] configure[25847]: segfault at 9558104 ip 080e2246 sp bfd478f0 error 4 in bash[8048000+148000] [307689.436739] configure[25966]: segfault at 95580b4 ip 080e2246 sp bfd478f0 error 4 in bash[8048000+148000] [307689.467279] configure[25983]: segfault at 9558104 ip 080e2246 sp bfd478f0 error 4 in bash[8048000+148000] ./configure I can't reproduce this on Mac OS X or RHEL 5 using `bash ./configure' (so I know what version of bash is running the script). Since configure starts with `#! /bin/sh', it's going to run whatever version of bash you have installed as /bin/sh. If you don't get the same results if you run `bash ./configure', open a bug report with your vendor against /bin/sh. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: umask --help
On 10/9/14, 3:05 PM, Notes Jonny wrote: If and when it happens, it will show up in the devel git branch on savannah. Chet Thank you for your reply. I understand that requests can't be implemented immediately. Is there anything I could do to get this implemented? Eg. I would offer to pay £100 for an engineer to complete the updates. Did you check the devel git branch on savannah? :-) The changes went in in early September. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Bash 4.3 with patch 29 causing segfault during ./configure when trying to upgrade to patch 30
On Thu, 2014-10-09 at 19:56 -0400, Chet Ramey wrote:
On 10/8/14, 8:17 PM, jon wrote:
dmesg.
[307688.764489] configure[25847]: segfault at 9558104 ip 080e2246 sp
bfd478f0 error 4 in bash[8048000+148000]
[307689.436739] configure[25966]: segfault at 95580b4 ip 080e2246 sp
bfd478f0 error 4 in bash[8048000+148000]
[307689.467279] configure[25983]: segfault at 9558104 ip 080e2246 sp
bfd478f0 error 4 in bash[8048000+148000]
./configure
I can't reproduce this on Mac OS X or RHEL 5 using `bash ./configure' (so
I know what version of bash is running the script). Since configure starts
with `#! /bin/sh', it's going to run whatever version of bash you have
installed as /bin/sh.
If you don't get the same results if you run `bash ./configure', open a
bug report with your vendor against /bin/sh.
I am the vendor, this is custom linux distro for DVR machines that I
maintain.
It gets even stranger.
I checked /bin/sh
# ls -l /bin/sh
lrwxrwxrwx1 root root4 Oct 27 2012 /bin/sh - bash
# which bash
/bin/bash
# /bin/bash --version
GNU bash, version 4.3.29(3)-release (i686-pc-linux-gnu)
I modified configure to :
echo $BASH_VERSION
I assume this is the version of the currently active shell not the
parent ?
# ./configure
4.3.29(3)-release
checking build system type... i686-pc-linux-gnu
SNIP
checking for presence of necessary job control definitions...
Segmentation fault
Ok - so far so bad - then I try
# /bin/bash ./configure
4.3.29(3)-release
checking build system type... i686-pc-linux-gnu
SNIP
config.status: executing default-1 commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
config.status: executing default commands
#
Same shell version, but now all works, very odd?
This is all inside the same ssh session, as far as I can tell the system
has only one shell binary in the search path. Note currently a
statically linked version - faulty lib maybe ?
# file /bin/bash
/bin/bash: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), for
GNU/Linux 2.2.5, statically linked, not stripped
I also tried
bash -c './configure' that segfaults.
I attached strace to the bash session running ./configure - it forks but
I re-attached to the child to get this, not sure that helps though ...
Thanks,
Jon
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
fork() = 8686
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x806d3b0, [], SA_RESTORER, 0x80f08f8}, {0x8085d20, [],
SA_RESTORER, 0x80f08f8}, 8) = 0
wait4(-1, [{WIFEXITED(s) WEXITSTATUS(s) == 0}], 0, NULL) = 8686
rt_sigaction(SIGINT, {0x8085d20, [], SA_RESTORER, 0x80f08f8}, {0x806d3b0, [],
SA_RESTORER, 0x80f08f8}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
wait4(-1, 0xbf977288, WNOHANG, NULL)= -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
stat64(/bin/rm, {st_mode=S_IFREG|0755, st_size=26780, ...}) = 0
rt_sigprocmask(SIG_BLOCK, [INT CHLD], [], 8) = 0
fork() = 8689
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0
rt_sigaction(SIGINT, {0x806d3b0, [], SA_RESTORER, 0x80f08f8}, {0x8085d20, [],
SA_RESTORER, 0x80f08f8}, 8) = 0
wait4(-1, [{WIFEXITED(s) WEXITSTATUS(s) == 0}], 0, NULL) = 8689
rt_sigaction(SIGINT, {0x8085d20, [], SA_RESTORER, 0x80f08f8}, {0x806d3b0, [],
SA_RESTORER, 0x80f08f8}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
--- SIGCHLD (Child exited) @ 0 (0) ---
wait4(-1, 0xbf98, WNOHANG, NULL)= -1 ECHILD (No child processes)
sigreturn() = ? (mask now [])
rt_sigprocmask(SIG_BLOCK, NULL, [], 8) = 0
open(., O_RDONLY|O_NONBLOCK|O_LARGEFILE|O_DIRECTORY) = 3
fstat64(3, {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
getdents64(3, /* 128 entries */, 4096) = 4088
getdents64(3, /* 5 entries */, 4096)= 160
getdents64(3, /* 0 entries */, 4096)= 0
brk(0) = 0x8b21000
brk(0x8b1f000) = 0x8b1f000
close(3)= 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
sigreturn() = ? (mask now [])
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 9848 detached
Re: Bash 4.3 with patch 29 causing segfault during ./configure when trying to upgrade to patch 30
On 10/9/14, 9:06 PM, jon wrote: On Thu, 2014-10-09 at 19:56 -0400, Chet Ramey wrote: On 10/8/14, 8:17 PM, jon wrote: dmesg. [307688.764489] configure[25847]: segfault at 9558104 ip 080e2246 sp bfd478f0 error 4 in bash[8048000+148000] [307689.436739] configure[25966]: segfault at 95580b4 ip 080e2246 sp bfd478f0 error 4 in bash[8048000+148000] [307689.467279] configure[25983]: segfault at 9558104 ip 080e2246 sp bfd478f0 error 4 in bash[8048000+148000] ./configure I can't reproduce this on Mac OS X or RHEL 5 using `bash ./configure' (so I know what version of bash is running the script). Since configure starts with `#! /bin/sh', it's going to run whatever version of bash you have installed as /bin/sh. If you don't get the same results if you run `bash ./configure', open a bug report with your vendor against /bin/sh. I am the vendor, this is custom linux distro for DVR machines that I maintain. It gets even stranger. Hmm...the leading !# /bin/sh makes ./configure turn into /bin/sh ./configure after the kernel gets done with it. Do you get the seg fault when running that? I also tried bash -c './configure' that segfaults. Sure, that does the same thing. I attached strace to the bash session running ./configure - it forks but I re-attached to the child to get this, not sure that helps though ... What would really help is a stack traceback from gdb on the core dump from the seg fault. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [RFC] Logically composable variant of errexit
On 10/9/14, 8:20 AM, Andreas Grünbacher wrote: Hi all, the errexit option can be very useful in simple scripts. This option is being ignored in many contexts like lists and conditionals though. I understand that this is by design and that errexit cannot be fixed to behave more reasonably. Still, this makes bash a lot less useful than it could be; working around this limitation is painful, ugly, and leads to fragile code. So, since we cannot fix errexit, can we maybe introduce another option like errfail that behaves like errexit for simple commands, but is also logically composable? Let me show what I mean with the following pseudo-code: What does logically composable mean in this context? Maybe it would be better if you described how you would like the behavior to differ from errexit. Like Angel, I see that you want errfail to be `inherited' by functions, like errexit; inhibited in the same set of exceptions; but that command substitution failures will cause the shell to exit. Is that accurate? Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Testing for Shellshock ... combinatorics and latest(Shellshock) Bash Vulnerability...(attn: Chet Ramey)
On 10/9/14, 6:06 PM, Pádraig Brady wrote: On 10/09/2014 08:46 PM, Rick Karcich (rkarcich) wrote: Hello Chet, Re: testing for Shellshock... would like your feedback... specifically, regarding the possibility of human-directed combinatorial testing to find this Bash vulnerability... Sounds like how Michal Zalewski found the related CVE-2014-6278 http://lcamtuf.blogspot.ie/2014/10/bash-bug-how-we-finally-cracked.html That's a promising approach. I asked Michal to continue running the fuzzer against the patched, but he did not respond to that yet. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: [Help-bash] Testing for Shellshock ... combinatorics and latest(Shellshock) Bash Vulnerability...(attn: Chet Ramey)
On 10/9/14, 4:50 PM, Eduardo A. Bustamante López wrote: Second, once you generate a command, how will your test program know if it found a bug? It's easy when bash segfaults, but in the case of shellshock, it wasn't a crash. This is the problem. It's hard to tell whether bash reporting a syntax error is a true syntax error, or a bug. The odds are considerably in favor of the former, which make it hard to weed out the false positives. You can use an environment variable in the form of an exported function to test the parser, or run bash with the -n option, to minimize potential damage to your system. The fact that Michal's fuzzer found bugs that had been in bash for years and never reported is a testament to the value of the approach. It just takes a lot of work to wade through the results. -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Re: Testing for Shellshock ... combinatorics and latest(Shellshock) Bash Vulnerability...(attn: Chet Ramey)
On 10/9/14, 3:46 PM, Rick Karcich (rkarcich) wrote: Hello Chet, Re: testing for Shellshock... would like your feedback... specifically, regarding the possibility of human-directed combinatorial testing to find this Bash vulnerability... Given the knowledge about Shellshock that's been developed, I'm wanting to define more of the attack surface for Shellshock - and put this into an input model for combinatorial testing... It's not clear that this problem would have been appropriate for automated testing. It was code that did what was intended: the problem is that it was invoked in (remotely-triggered) contexts that were unintended. The triggering condition was the appropriately-formatted environment variable. The issue that made it dangerous was that there were circumstances under which remote users could specify values that were turned into environment variables by applications that ended up invoking bash. In a local environment, it was a non-issue: a clever way to execute commands bash would have let you execute in a more straightforward way. If your attack surface is the shell parser, then you can just run bash -nc 'randomly-generated-string' all day long and see what happens. Chet -- ``The lyf so short, the craft so long to lerne.'' - Chaucer ``Ars longa, vita brevis'' - Hippocrates Chet Ramey, ITS, CWRUc...@case.eduhttp://cnswww.cns.cwru.edu/~chet/
Cannot build bash-4.2 with Patch 53
If I compile from bash-4.2 from source, cumulatively applying patches through 52, things work fine. If I start from scratch and apply through 53, it errors out: gcc -L./builtins -L./lib/readline -L./lib/readline -L./lib/glob -L./lib/tilde -L./lib/malloc -L./lib/sh -rdynamic -g -O2 -o bash shell.o eval.o y.tab.o general.o make_cmd.o print_cmd.o dispose_cmd.o execute_cmd.o variables.o copy_cmd.o error.o expr.o flags.o jobs.o subst.o hashcmd.o hashlib.o mailcheck.o trap.o input.o unwind_prot.o pathexp.o sig.o test.o version.o alias.o array.o arrayfunc.o assoc.o braces.o bracecomp.o bashhist.o bashline.o list.o stringlib.o locale.o findcmd.o redir.o pcomplete.o pcomplib.o syntax.o xmalloc.o -lbuiltins -lglob -lsh -lreadline -lhistory -ltermcap -ltilde -lmalloc-ldl ./builtins/libbuiltins.a(evalstring.o): In function `parse_and_execute': /home/ttrimmer/depot/ext/bash/patch/src/builtins/evalstring.c:274: undefined reference to `parser_remaining_input' collect2: ld returned 1 exit status make: *** [bash] Error 1 I can see parser_remaining_input patched in parse.y and builtins/evalstring.c. However, it will not compile.
Re: Cannot build bash-4.2 with Patch 53
On Oct 9, 2014, at 9:34 PM, TODD TRIMMER todd.trim...@gmail.com wrote: If I compile from bash-4.2 from source, cumulatively applying patches through 52, things work fine. If I start from scratch and apply through 53, it errors out: gcc -L.. . . ./builtins/libbuiltins.a(evalstring.o): In function `parse_and_execute': /home/ttrimmer/depot/ext/bash/patch/src/builtins/evalstring.c:274: undefined reference to `parser_remaining_input' collect2: ld returned 1 exit status make: *** [bash] Error 1 I can see parser_remaining_input patched in parse.y and builtins/evalstring.c. However, it will not compile. Sounds like y.tab.[ch] never got (re)built from parse.y. Try renaming them to -old and give the 'make' command again. If you don't have yacc or bison, that'll fail. Get and install bison and try again.
