On Thu, May 8, 2025 at 6:28 PM Serhei Makarov <ser...@serhei.io> wrote: > > Missing a few pieces, but worth sharing as an RFC. My idea is to > ensure better test coverage for eu-stack and then > eu-stacktrace+libdwfl_stacktrace by running against a live process > with known content, stopped at a known location, and aggressively > scrubbing output that's known to vary from testrun to testrun. > > This is a very basic preview of how that might look. If the approach > is sound, I hope to make it more sophisticated/reliable. > > Unanswered questions: > - Scrub more data (e.g. libc symvers) from a more known program. > Scrub stack frame numbers to account for a case where extra frames > appear / are missing at the bottom of the stack? > - Something better than sed for the scrubbing?
Sed is fine, we already use it (along with cut) in other test scripts for scrubbing components of output that might differ between runs or across environments. run-dwfl-core-noncontig.sh and run-sysroot.sh are two examples. > - An equivalent eu-stacktrace test will require privileged perf_events > access for profiling data and therefore likely to be skipped by > default. How feasible is it to be enabled on the buildbots, though? > > * tests/run-stack-live-test.sh: New test with wild and fuzzy > testrun_compare variant that scrubs inherently unpredictable parts of > the data. Needs to scrub even more. > * tests/Makefile.am (TESTS): Add run-stack-live-test.sh. > --- > tests/Makefile.am | 3 +- > tests/run-stack-live-test.sh | 64 ++++++++++++++++++++++++++++++++++++ > 2 files changed, 66 insertions(+), 1 deletion(-) > create mode 100755 tests/run-stack-live-test.sh > > diff --git a/tests/Makefile.am b/tests/Makefile.am > index 00ba754d..ecd514c7 100644 > --- a/tests/Makefile.am > +++ b/tests/Makefile.am > @@ -191,7 +191,8 @@ TESTS = run-arextract.sh run-arsymtest.sh run-ar.sh > newfile test-nlist \ > run-backtrace-core-s390x.sh run-backtrace-core-s390.sh \ > run-backtrace-core-aarch64.sh run-backtrace-core-sparc.sh \ > run-backtrace-demangle.sh run-stack-d-test.sh run-stack-i-test.sh \ > - run-stack-demangled-test.sh run-readelf-zx.sh run-readelf-zp.sh \ > + run-stack-demangled-test.sh run-stack-live-test.sh \ > + run-readelf-zx.sh run-readelf-zp.sh \ > run-readelf-arm-flags.sh \ > run-readelf-addr.sh run-readelf-str.sh \ > run-readelf-multi-noline.sh \ > diff --git a/tests/run-stack-live-test.sh b/tests/run-stack-live-test.sh > new file mode 100755 > index 00000000..808421bb > --- /dev/null > +++ b/tests/run-stack-live-test.sh > @@ -0,0 +1,64 @@ > +#! /bin/sh > +# Copyright (C) 2025 Red Hat, Inc. > +# This file is part of elfutils. > +# > +# This file is free software; you can redistribute it and/or modify > +# it under the terms of the GNU General Public License as published by > +# the Free Software Foundation; either version 3 of the License, or > +# (at your option) any later version. > +# > +# elfutils is distributed in the hope that it will be useful, but > +# WITHOUT ANY WARRANTY; without even the implied warranty of > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +# GNU General Public License for more details. > +# > +# You should have received a copy of the GNU General Public License > +# along with this program. If not, see <http://www.gnu.org/licenses/>. > + > +. $srcdir/test-subr.sh > + > +# Depending on whether we are running make check or make installcheck > +# the actual binary name under test might be different. It is used in > +# the error message, which we also try to match. > +if test "$elfutils_testrun" = "installed"; then > +STACKCMD=${bindir}/`program_transform stack` > +else > +STACKCMD=${abs_top_builddir}/src/stack > +fi > + > +# TODO(REVIEW): Can we make the data-scrubbing generic enough > +# (across multiple eu-stack/eu-stacktrace test cases) to move > +# to test_subr.sh? If it's not quite generic enough for test-subr.sh we can always include this in a stacktrace-subr.sh file, for instance. > +# > +# TODO(REVIEW): Better shell-isms for comparing file and regex? > +# \(\s\e\d\)\+\i\s\a\d\d\i\c\t\e\d\\t\o\b\a\ck\s\l\a\s\h\e\s I think the use of sed here is fine. If you wanted to improve readability you could perform the scrubbing across multiple sed commands each with a comment explaining the step. > +testrun_compare_fuzzy() > +{ > + outfile="${1##*/}.out" > + testrun_out $outfile "$@" > + sed -i 's/\(PID\|TID\|#[0-9]\+\)\( \+\)\(\(0x\)\?[0-9a-f]\+\)/\1\2nn/g' > $outfile > + diff -u $outfile - > +} > + > +# TODO: Need to scrub more data (e.g. GLIBC_ bits), > +# and use a program whose inner content we control: > +sleep 10 & > +PID=$! > +testrun_compare_fuzzy ${abs_top_builddir}/src/stack -p $PID <<EOF > +PID nn - process > +TID nn: > +#0 nn clock_nanosleep@GLIBC_2.2.5 > +#1 nn __nanosleep > +#2 nn main > +#3 nn __libc_start_call_main > +#4 nn __libc_start_main@@GLIBC_2.34 > +#5 nn _start > +EOF > +# PID 169385 - process > +# TID 169385: > +# #0 0x00007f04a98adbd7 clock_nanosleep@GLIBC_2.2.5 > +# #1 0x00007f04a98b9c47 __nanosleep > +# #2 0x0000561e7fdd9a9f main > +# #3 0x00007f04a97f4088 __libc_start_call_main > +# #4 0x00007f04a97f414b __libc_start_main@@GLIBC_2.34 > +# #5 0x0000561e7fdd9c05 _start > -- > 2.47.0 > FYI here is my results diff from running this test: --- stack.out 2025-05-14 09:34:00.676383411 -0400 +++ - 2025-05-14 09:34:00.677344470 -0400 @@ -1,9 +1,8 @@ PID nn - process TID nn: -#0 nn __internal_syscall_cancel -#1 nn clock_nanosleep@GLIBC_2.2.5 -#2 nn __nanosleep -#3 nn main -#4 nn __libc_start_call_main -#5 nn __libc_start_main@@GLIBC_2.34 -#6 nn _start +#0 nn clock_nanosleep@GLIBC_2.2.5 +#1 nn __nanosleep +#2 nn main +#3 nn __libc_start_call_main +#4 nn __libc_start_main@@GLIBC_2.34 +#5 nn _start FAIL run-stack-live-test.sh (exit status: 1) Aaron
