I just saw in your review comment that this is using LLVM_ENABLE_RUNTIMES and not LLVM_ENABLE_PROJECTS for libcxx, so the failure just comes from us setting the wrong RPATH due to the different runtimes library directory (at least from what I can see). Would it be possible to put libcxx in LLVM_ENABLE_PROJECTS for now? I think this shouldn't be too hard to fix (famous last words?).
Am Mo., 18. Okt. 2021 um 22:30 Uhr schrieb David Blaikie <dblai...@gmail.com>: > > On Mon, Oct 18, 2021 at 9:45 AM Raphael Isemann <teempe...@gmail.com> wrote: >> >> I think https://reviews.llvm.org/D111978 , >> https://reviews.llvm.org/D111981 and the other patches Pavel & me put >> up today should improve this situation IIUC. > > > Thanks Raphael - really appreciate you & looking into this! > > With https://reviews.llvm.org/D111981 I still seem to not have that cxx > dependency (building/running the test, then building libcxx, then running the > test again goes from unsupported -> failing) - didn't seem to work for me? > > The diagnostic improvement sounds good to me (pity about whatever limitations > it has re: remote or windows execution gathering the stdout). I guess > gathering the logs in the remote execution case has the same problem (that > the log is made on the remote machine/non-trivial to retrieve?)? > > & yeah, have you got any patches/ideas about how to make the libcxx tests use > the just-built libcxx library (using LD_LIBRARY_PATH, rpath, etc)? For now, > even with both these fixes I'll just be seeing (consistent, which is nice) > failures, rather than being able to run these tests successfully. I'll either > have to get used to ignoring certain failures, or disable the tests by not > building libcxx in that build tree, which would also be unfortunate. (or > maybe there's some other workarounds?) Any idea how this works for other > folks? > > - Dave > >> - Raphael >> >> Am Mo., 18. Okt. 2021 um 05:54 Uhr schrieb David Blaikie via lldb-dev >> <lldb-dev@lists.llvm.org>: >> > >> > Wondering if anyone else has encountered/dealt with debugging lldb test >> > failures like the one shown at the end of this email ("AssertionError: 10 >> > != 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" while >> > checking that a breakpoint was reached) >> > >> > Is there anything that could be done to improve the debuggability of such >> > failures? Logging standard output/error from the lldb process or any other >> > logging it might have? At least for one of these I managed to figure it >> > out by running lldb directly on the binary and finding that the binary >> > couldn't find libc++.so (that's a story for another bug/email thread, >> > looks like maybe lldb libc++ pretty printer tests are using the system >> > installed libc++, not the just-built libc++ (& thus not running if there >> > is no system installed libc++)). But my current failure like this seems a >> > bit more inscrutible and I'm still looking into it. >> > >> > Looks like dotest.py has some sense of logging (it has a --log-success >> > option which says preserves the logs even on failure - though the output >> > of dotest.py, at least for me, has no mention of logs, log files, or where >> > they might be located, and looking at the source points to some sort of >> > ".log" files... ah, found some) >> > >> > So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc - >> > should dotest print something about the paths to those files when it exits >> > with failure, maybe? >> > >> > ******************** >> > >> > FAIL: lldb-api :: >> > functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py >> > (23 of 23) >> > >> > ******************** TEST 'lldb-api :: >> > functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py' >> > FAILED ******************** >> > >> > Script: >> > >> > -- >> > >> > /usr/bin/python3 >> > /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u >> > CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env >> > OBJCOPY=/usr/bin/objcopy --env >> > LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib >> > --arch x86_64 --build-dir >> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex >> > --lldb-module-cache-dir >> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api >> > --clang-module-cache-dir >> > /usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api >> > --executable >> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb >> > --compiler >> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/clang >> > --dsymutil >> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin/dsymutil >> > --llvm-tools-dir >> > /usr/local/google/home/blaikie/dev/llvm/build/release/./bin >> > --lldb-libs-dir >> > /usr/local/google/home/blaikie/dev/llvm/build/release/./lib >> > /usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set >> > -p TestDataFormatterLibcxxSet.py >> > >> > -- >> > >> > Exit Code: 1 >> > >> > >> > Command Output (stdout): >> > >> > -- >> > >> > lldb version 14.0.0 (g...@github.com:llvm/llvm-project.git revision >> > 6176fda3f992b5086302b3826aa0636135cc4cc0) >> > >> > clang revision 6176fda3f992b5086302b3826aa0636135cc4cc0 >> > >> > llvm revision 6176fda3f992b5086302b3826aa0636135cc4cc0 >> > >> > Skipping the following test categories: ['dsym', 'gmodules', >> > 'debugserver', 'objc'] >> > >> > >> > -- >> > >> > Command Output (stderr): >> > >> > -- >> > >> > UNSUPPORTED: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_ref_and_ptr_dsym >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case >> > does not fall in any category of interest for this run) >> > >> > FAIL: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_ref_and_ptr_dwarf >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > FAIL: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_ref_and_ptr_dwo >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > UNSUPPORTED: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_ref_and_ptr_gmodules >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case >> > does not fall in any category of interest for this run) >> > >> > UNSUPPORTED: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_with_run_command_dsym >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case >> > does not fall in any category of interest for this run) >> > >> > FAIL: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_with_run_command_dwarf >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > FAIL: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_with_run_command_dwo >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > UNSUPPORTED: LLDB >> > (/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) >> > :: test_with_run_command_gmodules >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case >> > does not fall in any category of interest for this run) >> > >> > ====================================================================== >> > >> > FAIL: test_ref_and_ptr_dwarf >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > Test that the data formatters work on ref and ptr. >> > >> > ---------------------------------------------------------------------- >> > >> > Traceback (most recent call last): >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", >> > line 1823, in test_method >> > >> > return attrvalue(self) >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", >> > line 128, in test_ref_and_ptr >> > >> > (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 970, in run_to_source_breakpoint >> > >> > return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 892, in run_to_breakpoint_do_run >> > >> > test.assertEqual(process.GetState(), lldb.eStateStopped) >> > >> > AssertionError: 10 != 5 >> > >> > Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang >> > >> > ====================================================================== >> > >> > FAIL: test_ref_and_ptr_dwo >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > Test that the data formatters work on ref and ptr. >> > >> > ---------------------------------------------------------------------- >> > >> > Traceback (most recent call last): >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", >> > line 1823, in test_method >> > >> > return attrvalue(self) >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", >> > line 128, in test_ref_and_ptr >> > >> > (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 970, in run_to_source_breakpoint >> > >> > return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 892, in run_to_breakpoint_do_run >> > >> > test.assertEqual(process.GetState(), lldb.eStateStopped) >> > >> > AssertionError: 10 != 5 >> > >> > Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang >> > >> > ====================================================================== >> > >> > FAIL: test_with_run_command_dwarf >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > Test that that file and class static variables display correctly. >> > >> > ---------------------------------------------------------------------- >> > >> > Traceback (most recent call last): >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", >> > line 1823, in test_method >> > >> > return attrvalue(self) >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", >> > line 49, in test_with_run_command >> > >> > (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 970, in run_to_source_breakpoint >> > >> > return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 892, in run_to_breakpoint_do_run >> > >> > test.assertEqual(process.GetState(), lldb.eStateStopped) >> > >> > AssertionError: 10 != 5 >> > >> > Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang >> > >> > ====================================================================== >> > >> > FAIL: test_with_run_command_dwo >> > (TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) >> > >> > Test that that file and class static variables display correctly. >> > >> > ---------------------------------------------------------------------- >> > >> > Traceback (most recent call last): >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py", >> > line 1823, in test_method >> > >> > return attrvalue(self) >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py", >> > line 49, in test_with_run_command >> > >> > (self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint( >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 970, in run_to_source_breakpoint >> > >> > return run_to_breakpoint_do_run(test, target, breakpoint, launch_info, >> > >> > File >> > "/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py", >> > line 892, in run_to_breakpoint_do_run >> > >> > test.assertEqual(process.GetState(), lldb.eStateStopped) >> > >> > AssertionError: 10 != 5 >> > >> > Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang >> > >> > ---------------------------------------------------------------------- >> > >> > Ran 8 tests in 3.035s >> > >> > >> > RESULT: FAILED (0 passes, 4 failures, 0 errors, 4 skipped, 0 expected >> > failures, 0 unexpected successes) >> > >> > >> > _______________________________________________ >> > lldb-dev mailing list >> > lldb-dev@lists.llvm.org >> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
