Wondering if anyone else has encountered/dealt with debugging lldb test failures like the one shown at the end of this email ("AssertionError: 10 != 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" while checking that a breakpoint was reached)
Is there anything that could be done to improve the debuggability of such failures? Logging standard output/error from the lldb process or any other logging it might have? At least for one of these I managed to figure it out by running lldb directly on the binary and finding that the binary couldn't find libc++.so (that's a story for another bug/email thread, looks like maybe lldb libc++ pretty printer tests are using the system installed libc++, not the just-built libc++ (& thus not running if there is no system installed libc++)). But my current failure like this seems a bit more inscrutible and I'm still looking into it. Looks like dotest.py has some sense of logging (it has a --log-success option which says preserves the logs even on failure - though the output of dotest.py, at least for me, has no mention of logs, log files, or where they might be located, and looking at the source points to some sort of ".log" files... ah, found some) So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc - should dotest print something about the paths to those files when it exits with failure, maybe? ******************** FAIL: lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py (23 of 23) ******************** TEST 'lldb-api :: functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py' FAILED ******************** Script: -- /usr/bin/python3 /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env OBJCOPY=/usr/bin/objcopy --env LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib --arch x86_64 --build-dir /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex --lldb-module-cache-dir /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb --compiler /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/clang --dsymutil /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/dsymutil --llvm-tools-dir /usr/local/google/home/blaikie/dev/llvm/build/release/./bin --lldb-libs-dir /usr/local/google/home/blaikie/dev/llvm/build/release/./lib /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set -p TestDataFormatterLibcxxSet.py -- Exit Code: 1 Command Output (stdout): -- lldb version 14.0.0 (g...@github.com:llvm/llvm-project.git revision 6176fda3f992b5086302b3826aa0636135cc4cc0) clang revision 6176fda3f992b5086302b3826aa0636135cc4cc0 llvm revision 6176fda3f992b5086302b3826aa0636135cc4cc0 Skipping the following test categories: ['dsym', 'gmodules', 'debugserver', 'objc'] -- Command Output (stderr): -- UNSUPPORTED: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_ref_and_ptr_dsym (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does not fall in any category of interest for this run) FAIL: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_ref_and_ptr_dwarf (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) FAIL: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_ref_and_ptr_dwo (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) UNSUPPORTED: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_ref_and_ptr_gmodules (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does not fall in any category of interest for this run) UNSUPPORTED: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_with_run_command_dsym (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does not fall in any category of interest for this run) FAIL: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_with_run_command_dwarf (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) FAIL: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_with_run_command_dwo (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) UNSUPPORTED: LLDB (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) :: test_with_run_command_gmodules (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does not fall in any category of interest for this run) ====================================================================== FAIL: test_ref_and_ptr_dwarf (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) Test that the data formatters work on ref and ptr. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1823, in test_method return attrvalue(self) File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", line 128, in test_ref_and_ptr (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 970, in run_to_source_breakpoint return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 892, in run_to_breakpoint_do_run test.assertEqual(process.GetState(), lldb.eStateStopped) AssertionError: 10 != 5 Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang ====================================================================== FAIL: test_ref_and_ptr_dwo (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) Test that the data formatters work on ref and ptr. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1823, in test_method return attrvalue(self) File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", line 128, in test_ref_and_ptr (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 970, in run_to_source_breakpoint return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 892, in run_to_breakpoint_do_run test.assertEqual(process.GetState(), lldb.eStateStopped) AssertionError: 10 != 5 Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang ====================================================================== FAIL: test_with_run_command_dwarf (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) Test that that file and class static variables display correctly. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1823, in test_method return attrvalue(self) File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", line 49, in test_with_run_command (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 970, in run_to_source_breakpoint return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 892, in run_to_breakpoint_do_run test.assertEqual(process.GetState(), lldb.eStateStopped) AssertionError: 10 != 5 Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang ====================================================================== FAIL: test_with_run_command_dwo (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) Test that that file and class static variables display correctly. ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", line 1823, in test_method return attrvalue(self) File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", line 49, in test_with_run_command (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 970, in run_to_source_breakpoint return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, File "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", line 892, in run_to_breakpoint_do_run test.assertEqual(process.GetState(), lldb.eStateStopped) AssertionError: 10 != 5 Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang ---------------------------------------------------------------------- Ran 8 tests in 3.035s RESULT: FAILED (0 passes, 4 failures, 0 errors, 4 skipped, 0 expected failures, 0 unexpected successes)
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev