Re: What is the purpose of parser-built?
On 2/6/19 5:05 PM, Peng Yu wrote: > On Wed, Feb 6, 2019 at 4:49 PM Eric Blake wrote: >> >> On 2/6/19 4:18 PM, Peng Yu wrote: >>> Hi, >>> >>> I deleted the file parser-built, and bash still compiles and an empty >>> parser-built file will be generated upon compilation. What is the >>> purpose of this file? Should it be deleted? Thanks. > >> parser-built is a witness that $(YACC) was run, even if the timestamps >> did not change (because the generated file did not change compared to >> last time). It exists in the file system so as to let make compute >> timestamp dependencies where we know the parser is up-to-date, even >> though the actual file we depend on has a timestamp that does NOT change >> (because we intentionally don't override it when there is no >> difference), all in order to minimize the time spent rebuilding the >> project when making a tweak to parse.y (for example, being able to tell >> the difference between a minor edit that changes a comment but not the >> generated parser, vs. a major edit that requires rebuilding other files >> to pick up the changes implied by the changed parser). > > I don't get the point. Should parser-built be deleted? If bash can be > compiled without it, why put it in the source tar.gz file? Deleting it makes rebuild times longer, AND means that your build will depend on having a decent $(YACC) installed. The file is placed in the tar.gz so that downstream users can unpack bash (even if it corrupts timestamps), and still be able to './configure && make' without having bison installed; where only the upstream maintainer has to have bison. If you deleted the file and things still built, then you happen to have a good-enough bison installed, OR you got lucky and didn't make any changes to parser.y that would have resulted in changes where the fallback rule of copying the pre-built y.tab.h that shipped with the tarball disagrees with an actual y.tab.h that you would get if you did have bison installed. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
Re: Where is yacc_EOF defined?
On 2/6/19 4:33 PM, Peng Yu wrote: > Hi, > > yacc_EOF is mentioned in parse.y in something like this > > %left '&' ';' '\n' yacc_EOF > | error yacc_EOF > > But I don't find where it is defined similarly to other tokens like BAR_AND. Then you aren't very familiar with yacc. Per 'man yacc': The following declares name to be a token: %token [] name [number] [name [number]]... If tag is present, the C type for all tokens on this line shall be declared to be the type referenced by tag. If a positive integer, num‐ ber, follows a name, that value shall be assigned to the token. The following declares name to be a token, and assigns precedence to it: %left [] name [number] [name [number]]... %right [] name [number] [name [number]]... So, the %left line that you quoted above IS what defined yacc_EOF to be a token. If you've never used yacc/bison before, trying to learn how they work by using bash as your starting point is a rather heavy crash-course. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
Re: What is the purpose of parser-built?
On Wed, Feb 6, 2019 at 4:49 PM Eric Blake wrote: > > On 2/6/19 4:18 PM, Peng Yu wrote: > > Hi, > > > > I deleted the file parser-built, and bash still compiles and an empty > > parser-built file will be generated upon compilation. What is the > > purpose of this file? Should it be deleted? Thanks. > parser-built is a witness that $(YACC) was run, even if the timestamps > did not change (because the generated file did not change compared to > last time). It exists in the file system so as to let make compute > timestamp dependencies where we know the parser is up-to-date, even > though the actual file we depend on has a timestamp that does NOT change > (because we intentionally don't override it when there is no > difference), all in order to minimize the time spent rebuilding the > project when making a tweak to parse.y (for example, being able to tell > the difference between a minor edit that changes a comment but not the > generated parser, vs. a major edit that requires rebuilding other files > to pick up the changes implied by the changed parser). I don't get the point. Should parser-built be deleted? If bash can be compiled without it, why put it in the source tar.gz file? -- Regards, Peng
Re: What is the purpose of parser-built?
On 2/6/19 4:18 PM, Peng Yu wrote:
> Hi,
>
> I deleted the file parser-built, and bash still compiles and an empty
> parser-built file will be generated upon compilation. What is the
> purpose of this file? Should it be deleted? Thanks.
Look at Makefile.in:
GRAM_H = parser-built
y.tab.o: y.tab.h y.tab.c ${GRAM_H} command.h ${BASHINCDIR}/stdc.h input.h
${GRAM_H}: y.tab.h
@-if test -f y.tab.h ; then \
cmp -s $@ y.tab.h 2>/dev/null || cp -p y.tab.h $@; \
fi
y.tab.c: parse.y
# -if test -f y.tab.h; then mv -f y.tab.h old-y.tab.h; fi
$(YACC) -d $(srcdir)/parse.y
touch parser-built
# -if cmp -s old-y.tab.h y.tab.h; then mv old-y.tab.h y.tab.h;
else cp -p y.tab.h ${GRAM_H}; fi
parser-built is a witness that $(YACC) was run, even if the timestamps
did not change (because the generated file did not change compared to
last time). It exists in the file system so as to let make compute
timestamp dependencies where we know the parser is up-to-date, even
though the actual file we depend on has a timestamp that does NOT change
(because we intentionally don't override it when there is no
difference), all in order to minimize the time spent rebuilding the
project when making a tweak to parse.y (for example, being able to tell
the difference between a minor edit that changes a comment but not the
generated parser, vs. a major edit that requires rebuilding other files
to pick up the changes implied by the changed parser).
You really ought to learn how to use things like 'git grep parser-built'
to try and discover where this file is mentioned, prior to asking your
questions. Had you asked "I see that Makefile creates parser-built
during the y.tab.c: parse.y production, but didn't understand why", that
comes across much nicer than your generic question which reads more like
"I see an empty file and don't know why, but couldn't be bothered to
even explain what things I've tried in order to take a guess at why, but
would rather just have you do all the work for me".
http://www.catb.org/esr/faqs/smart-questions.html is a great read; it
will teach you how to come across as someone worth chatting to, rather
than an infinite time sink, if you would use the advice in that file to
ask smarter questions.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
signature.asc
Description: OpenPGP digital signature
Where is yacc_EOF defined?
Hi, yacc_EOF is mentioned in parse.y in something like this %left '&' ';' '\n' yacc_EOF | error yacc_EOF But I don't find where it is defined similarly to other tokens like BAR_AND. %token GREATER_BAR BAR_AND Where is yacc_EOF defined? (y.tab.c and y.tab.h are files generated by bison. so yacc_EOF must be defined before bison is called?) -- Regards, Peng
What is the purpose of parser-built?
Hi, I deleted the file parser-built, and bash still compiles and an empty parser-built file will be generated upon compilation. What is the purpose of this file? Should it be deleted? Thanks. -- Regards, Peng
AddressSanitizer: heap-use-after-free on (...) in rl_do_undo ../../../bash-devel/lib/readline/undo.c:188
I found another issue in rl_do_undo, but I haven't been successful in figuring
out how it happens.
I've been working with the `devel' branch, commit
`8a9718cfc93958b34e205d0507c3bbf64cba6db5'
Here's how I built the binaries I use below:
debian@debian-fuzz:~/tmp$ cat ~/build.sh
#!/bin/bash
mkdir build-devel{,-asan,-gdb}
(cd build-devel && CFLAGS='' CC='afl-clang-fast'
../bash-devel/configure --silent --without-bash-malloc && make
-sj$(nproc))
(cd build-devel-asan && CFLAGS='-O0 -ggdb -fno-omit-frame-pointer
-fsanitize=address ' ../bash-devel/configure --silent --without-bash-malloc &&
make -sj$(nproc))
(cd build-devel-gdb && CFLAGS='-O0 -ggdb -fno-omit-frame-pointer '
../bash-devel/configure --silent --without-bash-malloc && make
-sj$(nproc))
This is the input to `read -e':
debian@debian-fuzz:~/tmp$ base64 < o
MBgFEBAQDhUwEBgoHx8wEDAYRRQbEDAYBTAYRQ4=
debian@debian-fuzz:~/tmp$ cat -A o
0^X^E^P^P^P^N^U0^P^X(^_^_0^P0^XE^T^[^P0^X^E0^XE^N
My poor attempt of a trace under GDB:
debian@debian-fuzz:~/tmp$ gdb --batch --command=trace_rl_undo.gdb --args
~/build-devel-gdb/bash --noprofile --norc -c 'EDITOR=: PATH= read -e < o' >
out.txt 2>&1
(no output)
---
dualbus@system76-pc:~/src/dualbus/bash-fuzzing/tmp/tmp$ cat out.txt
0
0
/tmp/bash-fc.9lfkNE: line 1: 0: No such file or directory
000
000
/tmp/bash-fc.HkVuNJ: line 1: 000: No such file or directory
readline: maximum macro execution nesting level exceeded
munmap_chunk(): invalid pointer
---
dualbus@system76-pc:~/src/dualbus/bash-fuzzing/tmp/tmp$ cat gdb.txt
Temporary breakpoint 1 at 0x2eea9: file ../bash-devel/shell.c, line 392.
Temporary breakpoint 1, main (argc=5, argv=0x7fffdfd8, env=0x7fffe008)
at ../bash-devel/shell.c:392
392 code = setjmp_nosigs (top_level);
Breakpoint 2 at 0x5565004c: file ../../../bash-devel/lib/readline/undo.c,
line 177.
[Detaching after fork from child process 588]
Breakpoint 2, rl_do_undo () at ../../../bash-devel/lib/readline/undo.c:177
(... snip ...)
Breakpoint 2, rl_do_undo () at ../../../bash-devel/lib/readline/undo.c:177
177 start = end = waiting_for_begin = 0;
$625 = "~~~ print_rl_undo_list>"
$626 = 0
$627 = (UNDO_LIST *) 0x556f9e50
$628 = (struct undo_list *) 0x556fa6f0
$629 = 0x0
$630 = 1
$631 = (struct undo_list *) 0x556fa6f0
$632 = (struct undo_list *) 0x0
$633 = 0x0
$634 = "~~~ print_the_history>"
$635 = 0
$636 = (HIST_ENTRY *) 0x556fa720
$637 = 0x556debd0 "000"
$638 = 0
$639 = (UNDO_LIST *) 0x556fa6f0
$640 = (struct undo_list *) 0x0
$641 = 0x0
$642 = 1
$643 = (HIST_ENTRY *) 0x556fa7b0
$644 = 0x556f89a0 "0"
$645 = 0
$646 = (UNDO_LIST *) 0x556fa330
$647 = (struct undo_list *) 0x556fa3d0
$648 = 0x0
$649 = 1
$650 = (struct undo_list *) 0x556fa3d0
$651 = (struct undo_list *) 0x556f9f00
$652 = 0x556de940 "\360\237oUUU"
$653 = 2
$654 = (struct undo_list *) 0x556f9f00
$655 = (struct undo_list *) 0x0
$656 = 0x556fa750 " "
$657 = 2
$658 = (HIST_ENTRY *) 0x556fa2c0
$659 = 0x556f85f0 ""
$660 = 0
$661 = (UNDO_LIST *) 0x556f90e0
$662 = (struct undo_list *) 0x0
$663 = 0x0
$664 = 3
$665 = (HIST_ENTRY *) 0x556f9130
$666 = 0x556fa6d0 ""
$667 = 0
$668 = (UNDO_LIST *) 0x556f90b0
$669 = (struct undo_list *) 0x0
$670 = 0x0
#0 rl_do_undo () at ../../../bash-devel/lib/readline/undo.c:177
#1 0x556504fd in rl_undo_command (count=1, key=31) at
../../../bash-devel/lib/readline/undo.c:358
#2 0x55633b75 in _rl_dispatch_subseq (key=31, map=0x5569f200
, got_subseq=0) at
../../../bash-devel/lib/readline/readline.c:852
#3 0x556338ec in _rl_dispatch (key=1433265360, map=0x5569f200
) at ../../../bash-devel/lib/readline/readline.c:798
#4 0x5563356f in readline_internal_char () at
../../../bash-devel/lib/readline/readline.c:632
#5 0x556335ca in readline_internal_charloop () at
../../../bash-devel/lib/readline/readline.c:659
#6 0x556335ea in readline_internal () at
../../../bash-devel/lib/readline/readline.c:671
#7 0x55633008 in readline (prompt=0x556767bc "") at
../../../bash-devel/lib/readline/readline.c:377
#8 0x5560c4cc in edit_line (p=0x556767bc "", itext=0x0) at
../../bash-devel/builtins/../../bash-devel/builtins/read.def:1107
#9 0x5560b2b2 in read_builtin (list=0x0) at
../../bash-devel/builtins/../../bash-devel/builtins/read.def:566
#10 0x555a3bc5 in execute_builtin (builtin=0x5560a45d
, words=0x556c1030, flags=0, subshell=0) at
../bash-devel/execute_cmd.c:4709
#11 0x555a4ae9 in execute_builtin_or_function (words=0x556c1030,
builtin=0x5560a45d , var=0x0, redirects=0x556bf720,
fds_to_close=0x556c0cb0, flags=0) at ../bash-devel/execute_cmd.c:5217
#12 0x5
Re: The use of register keyword in bash source code
On Feb 06 2019, Peng Yu wrote: > If it is ignored anyway, why "register" is used in many places in the > code? Thanks. Because compilers were dumb in the old days. Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
Re: The use of register keyword in bash source code
> No, that is what volatile means. The register keyword is just an > optimisation hint, and is mostly ignored by the compiler. If it is ignored anyway, why "register" is used in many places in the code? Thanks. -- Regards, Peng
Re: The use of register keyword in bash source code
On Feb 06 2019, Peng Yu wrote: > I see many variables are declared with the "register" keyword. I know > its purpose is to tell compile always access the corresponding memory > without assuming the previously accessed values are preserved. This is > usually to deal with some external devices. No, that is what volatile means. The register keyword is just an optimisation hint, and is mostly ignored by the compiler. Andreas. -- Andreas Schwab, SUSE Labs, sch...@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different."
The use of register keyword in bash source code
Hi, I see many variables are declared with the "register" keyword. I know its purpose is to tell compile always access the corresponding memory without assuming the previously accessed values are preserved. This is usually to deal with some external devices. But I don't understand why it is useful in bash. Could anybody help explain? Thanks. -- Regards, Peng
