Re: expand_prompt_string segmentation faults

2018-09-17 Thread Eduardo A . Bustamante López 
On Sun, Sep 16, 2018 at 07:24:13PM -0400, Chet Ramey wrote:
> On 9/16/18 5:15 PM, Eduardo A. Bustamante López wrote:
> > On Sun, Sep 16, 2018 at 02:39:07PM -0400, Chet Ramey wrote:
> >> On 9/16/18 4:12 AM, Eduardo A. Bustamante López wrote:
> >>> Here's another interesting crash:
> >>>
> >>> $ base64 < 53
> >>> MBgFEBAQMBgFEBAfEA==
> >>>
> >>> $ gdb -batch -ex=r -ex=bt --args ./bash --noprofile --norc -c 'PATH=; set 
> >>> -o emacs; IFS= read -re' < 53
> >>
> >> I can't reproduce this on Mac OS X or Fedora 28.
> 
> OK, let me look around some more with your debugging hints. If I find it,
> it will be in the next devel branch push.

Hah! Thanks for the fix to rl_do_undo. I appreciate the comments, it makes it
easier to understand what's going on.

I ran this latest push through the set of crashes I found. They're all fixed.

Thanks again!

Re: expand_prompt_string segmentation faults

2018-09-16 Thread Chet Ramey 
On 9/16/18 5:15 PM, Eduardo A. Bustamante López wrote:
> On Sun, Sep 16, 2018 at 02:39:07PM -0400, Chet Ramey wrote:
>> On 9/16/18 4:12 AM, Eduardo A. Bustamante López wrote:
>>> Here's another interesting crash:
>>>
>>> $ base64 < 53
>>> MBgFEBAQMBgFEBAfEA==
>>>
>>> $ gdb -batch -ex=r -ex=bt --args ./bash --noprofile --norc -c 'PATH=; set 
>>> -o emacs; IFS= read -re' < 53
>>
>> I can't reproduce this on Mac OS X or Fedora 28.

OK, let me look around some more with your debugging hints. If I find it,
it will be in the next devel branch push.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: expand_prompt_string segmentation faults

2018-09-16 Thread Eduardo A . Bustamante López 
On Sun, Sep 16, 2018 at 02:39:07PM -0400, Chet Ramey wrote:
> On 9/16/18 4:12 AM, Eduardo A. Bustamante López wrote:
> > Here's another interesting crash:
> > 
> > $ base64 < 53
> > MBgFEBAQMBgFEBAfEA==
> > 
> > $ gdb -batch -ex=r -ex=bt --args ./bash --noprofile --norc -c 'PATH=; set 
> > -o emacs; IFS= read -re' < 53
> 
> I can't reproduce this on Mac OS X or Fedora 28.

Hm, I can reliably reproduce it with or without bash-malloc.

$ base64 -d <<< 'MBgFEBAQMBgFEBAfEA==' > 53

$ cat -v 53
0^X^E^P^P^P0^X^E^P^P^_^P

$ CC=clang CFLAGS='-O0 -ggdb' ./configure  && make -j4
(...)

$ ./bash --noprofile --norc -c 'PATH=; set -o emacs; IFS= read -re < 53' 
0
./bash: vim: No such file or directory
0rl_maybe_unsave_line: rl_undo_list=0x236c848
rl_maybe_unsave_line: rl_undo_list->next=(nil)
rl_maybe_unsave_line: rl_undo_list=0x236c848
rl_maybe_unsave_line: rl_undo_list->next=(nil)
0
./bash: vim: No such file or directory
rl_do_undo: rl_undo_list = (nil)
rl_do_undo: xfree(release = 0x236c848)
rl_maybe_unsave_line: rl_undo_list=0x2360188
rl_maybe_unsave_line: rl_undo_list->next=0x236c848
rl_do_undo: rl_undo_list = 0x236c848
rl_do_undo: xfree(release = 0x2360188)
rl_do_undo: rl_undo_list = 0x23601c0
rl_do_undo: xfree(release = 0x236c848)

malloc: unknown:0: assertion botched
malloc: 0x236c848: allocated: last allocated from unknown:0
free: called with already freed block argument
Aborting...Aborted


My interpretation of the issue is that the `^_' triggers an `undo', which then
causes bash to release that entry (i.e. `0x236c848' in the example above). Then
the `^P' somehow restores that free'd rl_undo_list entry and bash tries to free
it again during readline_internal_teardown.


Under GDB:

dualbus@ubuntu:~/src/gnu/bash$ gdb --args ./bash --noprofile --norc -c 'PATH=; 
set -o emacs; IFS= read -re < 53' 
GNU gdb (Ubuntu 8.2-0ubuntu1) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./bash...done.
(gdb) b rl_maybe_unsave_line 
Breakpoint 1 at 0x526128: file misc.c, line 354.
(gdb) b rl_do_undo 
Breakpoint 2 at 0x51e6a8: file undo.c, line 175.
(gdb) r
Starting program: /home/dualbus/src/gnu/bash/bash --noprofile --norc -c 
PATH=\;\ set\ -o\ emacs\;\ IFS=\ read\ -re\ \<\ 53
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0
[Detaching after fork from child process 20660]
/home/dualbus/src/gnu/bash/bash: vim: No such file or directory
0
Breakpoint 1, rl_maybe_unsave_line () at misc.c:354
354   if (_rl_saved_line_for_history)
(gdb) bt
#0  rl_maybe_unsave_line () at misc.c:354
#1  0x005266ea in rl_get_previous_history (count=1, key=16) at 
misc.c:618
#2  0x004fb350 in _rl_dispatch_subseq (key=16, map=0x575db0 
, got_subseq=0) at readline.c:852
#3  0x004face9 in _rl_dispatch (key=16, map=0x575db0 
) at readline.c:798
#4  0x004fac59 in readline_internal_char () at readline.c:632
#5  0x004fc282 in readline_internal_charloop () at readline.c:659
#6  0x004fa5ae in readline_internal () at readline.c:671
#7  0x004fa470 in readline (prompt=0x5546af "") at readline.c:377
#8  0x004caa46 in edit_line (p=0x5546af "", itext=0x0) at 
./read.def:1104
#9  0x004c94ea in read_builtin (list=0x0) at ./read.def:563
#10 0x0044b599 in execute_builtin (builtin=0x4c8590 , 
words=0x629428, flags=0, subshell=0) at execute_cmd.c:4677
#11 0x0044a96f in execute_builtin_or_function (words=0x629428, 
builtin=0x4c8590 , var=0x0, redirects=0x628d08, 
fds_to_close=0x628dc8, flags=0) at execute_cmd.c:5185
#12 0x004437c9 in execute_simple_command (simple_command=0x628c08, 
pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x628dc8) at execute_cmd.c:4449
#13 0x004412ab in execute_command_internal (command=0x628bc8, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x628dc8) at 
execute_cmd.c:840
#14 0x00445208 in execute_connection (command=0x628d88, asynchronous=0, 
pipe_in=-1, pipe_out=-1, fds_to_close=0x628dc8) at execute_cmd.c:2689
#15 0x00441681 in execute_command_internal (command=0x628d88, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x628dc8) at 
execute_cmd.c:1013
#16 0x004bf557 in parse_and_execute (string=0x628408 "PATH=; set -o 
emacs; IFS= read -re < 53", from_file=0x5353fb "-c",

Re: expand_prompt_string segmentation faults

2018-09-16 Thread Chet Ramey 
On 9/16/18 2:55 AM, Eduardo Bustamante wrote:
> On Sat, Sep 15, 2018 at 8:15 PM Chet Ramey  wrote:
> (...)
>> Thanks for the report. They are the same problem.
> 
> Thanks Chet. Do you have a patch for the issue? I have a few other
> cases but I'm not sure if they're all due to the same cause.

Sure.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/
*** ../bash-20180907/expr.c 2018-08-27 15:31:05.0 -0400
--- expr.c  2018-09-15 22:53:56.0 -0400
***
*** 326,329 
--- 326,332 
int aflags;
  
+   if (lhs == 0 || *lhs == 0)
+ return;   /* XXX */
+ 
  #if defined (ARRAY_VARS)
aflags = (assoc_expand_once && already_expanded) ? ASS_NOEXPAND : 0;
***
*** 1013,1017 
  else
  #endif
!   expr_bind_variable (tokstr, vincdec);
}
free (vincdec);
--- 1016,1021 
  else
  #endif
!   if (tokstr)
! expr_bind_variable (tokstr, vincdec);
}
free (vincdec);

Re: expand_prompt_string segmentation faults

2018-09-16 Thread Chet Ramey 
On 9/16/18 4:12 AM, Eduardo A. Bustamante López wrote:
> Here's another interesting crash:
> 
> $ base64 < 53
> MBgFEBAQMBgFEBAfEA==
> 
> $ gdb -batch -ex=r -ex=bt --args ./bash --noprofile --norc -c 'PATH=; set -o 
> emacs; IFS= read -re' < 53

I can't reproduce this on Mac OS X or Fedora 28.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: expand_prompt_string segmentation faults

2018-09-16 Thread Eduardo A . Bustamante López 
Here's another interesting crash:

$ base64 < 53
MBgFEBAQMBgFEBAfEA==

$ gdb -batch -ex=r -ex=bt --args ./bash --noprofile --norc -c 'PATH=; set -o 
emacs; IFS= read -re' < 53
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0
[Detaching after fork from child process 30991]
/home/dualbus/src/gnu/bash/bash: vim: No such file or directory
0
[Detaching after fork from child process 30992]
/home/dualbus/src/gnu/bash/bash: vim: No such file or directory

malloc: unknown:0: assertion botched
malloc: 0x630808: allocated: last allocated from unknown:0
free: called with already freed block argument
Aborting...
Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x779a7535 in __GI_abort () at abort.c:79
#2  0x004597bb in programming_error (format=0x5557be "free: called with 
already freed block argument") at error.c:175
#3  0x005322c2 in xbotch (mem=0x630808, e=1, s=0x5557be "free: called 
with already freed block argument", file=0x0, line=0) at malloc.c:354
#4  0x0053155c in internal_free (mem=0x630808, file=0x0, line=0, 
flags=0) at malloc.c:931
#5  0x00531d4f in free (mem=0x630808) at malloc.c:1388
#6  0x004b5150 in xfree (string=0x630808) at xmalloc.c:150
#7  0x0051e94c in rl_do_undo () at undo.c:240
#8  0x0051eb23 in rl_revert_line (count=1, key=0) at undo.c:312
#9  0x004fa7f9 in readline_internal_teardown (eof=1) at readline.c:471
#10 0x004fa591 in readline_internal () at readline.c:672
#11 0x004fa440 in readline (prompt=0x55466f "") at readline.c:377
#12 0x004caa16 in edit_line (p=0x55466f "", itext=0x0) at 
./read.def:1104
#13 0x004c94ba in read_builtin (list=0x0) at ./read.def:563
#14 0x0044b599 in execute_builtin (builtin=0x4c8560 , 
words=0x628f88, flags=0, subshell=0) at execute_cmd.c:4677
#15 0x0044a96f in execute_builtin_or_function (words=0x628f88, 
builtin=0x4c8560 , var=0x0, redirects=0x0, fds_to_close=0x628ce8, 
flags=0) at execute_cmd.c:5185
#16 0x004437c9 in execute_simple_command (simple_command=0x628c08, 
pipe_in=-1, pipe_out=-1, async=0, fds_to_close=0x628ce8) at execute_cmd.c:4449
#17 0x004412ab in execute_command_internal (command=0x628bc8, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x628ce8) at 
execute_cmd.c:840
#18 0x00445208 in execute_connection (command=0x628d48, asynchronous=0, 
pipe_in=-1, pipe_out=-1, fds_to_close=0x628ce8) at execute_cmd.c:2689
#19 0x00441681 in execute_command_internal (command=0x628d48, 
asynchronous=0, pipe_in=-1, pipe_out=-1, fds_to_close=0x628ce8) at 
execute_cmd.c:1013
#20 0x004bf527 in parse_and_execute (string=0x628408 "PATH=; set -o 
emacs; IFS= read -re", from_file=0x5353fb "-c", flags=4) at evalstring.c:436
#21 0x00423845 in run_one_command (command=0x7fffea92 "PATH=; set 
-o emacs; IFS= read -re") at shell.c:1416
#22 0x00421920 in main (argc=5, argv=0x7fffe758, 
env=0x7fffe788) at shell.c:735


Bash compiled with:

`CC=clang CFLAGS='-O0 -ggdb' ./configure && make -j4' from `devel' 
(b52e30b8dd5bba5aed7b13040c9d4f7a0c1990cc) on Linux
(Ubuntu 18.04)

Re: expand_prompt_string segmentation faults

2018-09-16 Thread Eduardo Bustamante 
On Sat, Sep 15, 2018 at 8:15 PM Chet Ramey  wrote:
(...)
> Thanks for the report. They are the same problem.

Thanks Chet. Do you have a patch for the issue? I have a few other
cases but I'm not sure if they're all due to the same cause.

Re: expand_prompt_string segmentation faults

2018-09-15 Thread Chet Ramey 
On 9/14/18 3:26 AM, Eduardo A. Bustamante López wrote:
> Found the following two cases by fuzzing with AFL:

Thanks for the report. They are the same problem.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

expand_prompt_string segmentation faults

2018-09-14 Thread Eduardo A . Bustamante López 
Found the following two cases by fuzzing with AFL:

# Case #1: array_expand_index
bash <<'EOF'
x='${p[--b[?]]'; echo ${x@P}
EOF

# Case #1 backtrace
: <<'EOF'
Program received signal SIGSEGV, Segmentation fault.
0x0080e0d3 in __strchr_sse2 ()
#0  0x0080e0d3 in __strchr_sse2 ()
#1  0x006d954b in mbschr (s=0x0, c=91) at mbschr.c:90
#2  0x0058acdf in valid_array_reference (name=0x0, flags=0) at 
arrayfunc.c:899
#3  0x0049c4e9 in bind_int_variable (lhs=0x0, rhs=0xbb5228 "-1", 
flags=0) at variables.c:3371
#4  0x004c632c in expr_bind_variable (lhs=0x0, rhs=) at 
expr.c:333
#5  exp0 () at expr.c:1015
#6  exp1 () at expr.c:983
#7  0x004c54ae in exppower () at expr.c:938
#8  0x004c4cf8 in exp2 () at expr.c:863
#9  0x004c4695 in exp3 () at expr.c:837
#10 expshift () at expr.c:813
#11 0x004c3d95 in exp4 () at expr.c:783
#12 exp5 () at expr.c:761
#13 0x004c3a61 in expband () at expr.c:743
#14 expbxor () at expr.c:724
#15 0x004c3621 in expbor () at expr.c:705
#16 expland () at expr.c:678
#17 0x004c2e01 in explor () at expr.c:650
#18 expcond () at expr.c:603
#19 0x004c1f2b in expassign () at expr.c:488
#20 0x004be48e in expcomma () at expr.c:472
#21 subexpr (expr=0xbc9a48 "--b[?]") at expr.c:450
#22 0x004bdba0 in evalexp (expr=0xbc9a48 "--b[?]", flags=, validp=0x7fffce14) at expr.c:415
#23 0x00589d81 in array_expand_index (var=, s=, len=, flags=) at arrayfunc.c:952
#24 0x0058b7f5 in array_value_internal (s=0xbc9a08 "p[--b[?]]", 
quoted=, flags=1, rtype=0x7fffce9c, indp=) at 
arrayfunc.c:1133
#25 0x0053eed1 in parameter_brace_expand_word (name=0xbc9a08 
"p[--b[?]]", var_is_special=0, quoted=1, pflags=, 
indp=0x7fffcf40) at subst.c:6584
#26 0x00536c7b in parameter_brace_expand (string=, 
quoted=, pflags=, contains_dollar_at=, indexp=, quoted_dollar_atp=) at subst.c:8702
#27 param_expand (string=0xbc5fe8 "${p[--b[?]]", sindex=, 
quoted=, expanded_something=, 
contains_dollar_at=, quoted_dollar_at_p=, 
had_quoted_null_p=0x0, pflags=) at subst.c:9316
#28 0x00510893 in expand_word_internal (word=0x7fffd0b0, 
quoted=, isexp=, contains_dollar_at=, expanded_something=) at subst.c:9887
#29 0x0050f595 in expand_prompt_string (string=0xbc7ec8 "${p[--b[?]]", 
quoted=1, wflags=) at subst.c:3804
#30 0x00420e71 in decode_prompt_string (string=) at 
./parse.y:6065
#31 0x0055059c in string_transform (xc=, v=0xbc7dc8, 
s=0xbc5fc8 "${p[--b[?]]") at subst.c:7468
#32 0x0054a2b5 in parameter_brace_transform (varname=, 
value=, ind=, xform=, rtype=0, 
quoted=, pflags=0, flags=) at subst.c:7616
#33 0x0053bb17 in parameter_brace_expand (string=, 
quoted=, pflags=, contains_dollar_at=, indexp=, quoted_dollar_atp=) at subst.c:8884
#34 param_expand (string=0xbc7e68 "${REPLY@P}", sindex=, 
quoted=, expanded_something=, 
contains_dollar_at=, quoted_dollar_at_p=, 
had_quoted_null_p=, pflags=) at subst.c:9316
#35 0x00510893 in expand_word_internal (word=0xbc7828, 
quoted=, isexp=, contains_dollar_at=, expanded_something=) at subst.c:9887
#36 0x00529560 in shell_expand_word_list (tlist=, 
eflags=0) at subst.c:11233
#37 expand_word_list_internal (list=, eflags=) at 
subst.c:11357
#38 0x0046f341 in execute_simple_command (simple_command=, pipe_in=-1, pipe_out=-1, async=, fds_to_close=) at execute_cmd.c:4278
#39 execute_command_internal (command=, asynchronous=, pipe_in=, pipe_out=, 
fds_to_close=) at execute_cmd.c:840
#40 0x0046b5cb in execute_connection (command=, 
asynchronous=, pipe_in=, pipe_out=, fds_to_close=) at execute_cmd.c:2689
#41 execute_command_internal (command=0xbc5e48, asynchronous=, 
pipe_in=, pipe_out=, fds_to_close=) at execute_cmd.c:1013
#42 0x00605bcc in parse_and_execute (string=, 
from_file=, flags=4) at evalstring.c:436
#43 0x00409a8c in run_one_command (command=) at 
shell.c:1416
#44 0x004063a7 in main (argc=, argv=, 
env=) at shell.c:735
EOF

# Case #2
bash <<'EOF'
x='$[++K[+]]/'; echo ${x@P}
EOF

# Case #2 backtrace
: <<'EOF'
Program received signal SIGSEGV, Segmentation fault.
0x0080e0d3 in __strchr_sse2 ()
#0  0x0080e0d3 in __strchr_sse2 ()
#1  0x006d954b in mbschr (s=0x0, c=91) at mbschr.c:90
#2  0x0058acdf in valid_array_reference (name=0x0, flags=0) at 
arrayfunc.c:899
#3  0x0049c4e9 in bind_int_variable (lhs=0x0, rhs=0xbb5248 "1", 
flags=0) at variables.c:3371
#4  0x004c632c in expr_bind_variable (lhs=0x0, rhs=) at 
expr.c:333
#5  exp0 () at expr.c:1015
#6  exp1 () at expr.c:983
#7  0x004c54ae in exppower () at expr.c:938
#8  0x004c4cf8 in exp2 () at expr.c:863
#9  0x004c4695 in exp3 () at expr.c:837
#10 expshift () at expr.c:813
#11 0x004c3d95 in exp4 () at expr.c:783
#12 exp5 () at expr.c:761
#13 0x004c3a61 in expband () at expr.c:743
#14 expbxor () at