In addition to flaky tests, I think some of these are just decorated too broadly (e.g. it's marked expectedFailureLinux, but fails only on i386 with gcc). I occasionally enable tests that I see are passing consistently, but I am currently more worried about tests failing unexpectedly than succeeding.
The 30 minutes for running the test seems very long, something must have gone wrong there. If you do a "ps" after 5 minutes, which processes do you still see running? What about after 15? What are the specs of the machine you are running this on? What is the exact command line you launching the tests with? I wouldn't be too worried about the timeouts, these are the two of our longest-running tests, so I think they are just getting killed for running too slow. We need to figure out what is causing the whole suite to run so slowly. (Unless you see them constantly timing out at the exact same test, in which case it could be interesting.) cheers, pl On 4 February 2016 at 04:11, Todd Fiala via lldb-dev <lldb-dev@lists.llvm.org> wrote: >> I don't recall exactly what it was off the top of my head, but I wonder >> if Zachary needs that? > > That is the lldb_enable_attach() call that I make in the beginning of the > inferior test driver, defined in > packages/Python/lldbsuite/test/make/test_common.h. This is already called, > so shouldn't be the issue. > > On Wed, Feb 3, 2016 at 8:07 PM, Todd Fiala <todd.fi...@gmail.com> wrote: >> >> You also need to pass "hello, world" as a launch arg (in quotes). That is >> what will make it get echoed back. >> >> On Wed, Feb 3, 2016 at 8:05 PM, Todd Fiala <todd.fi...@gmail.com> wrote: >>> >>> Hey Zachary, >>> >>> For the test listed above, it is failing trying to match output from the >>> inferior process being debugged by lldb-server. First, it is trying to get >>> a hello, world string to be printed. Then, it is expecting the process to >>> exit without failure. >>> >>> If you go in that directory and make/run the a.out program, it should >>> print hello world and exit with an exit value of 0. You may find that it >>> doesn't print, perhaps? Or maybe your terminal is set differently, so that >>> the text isn't matching as expected? (I would expect to have heard others >>> with this issue). >>> >>> Pavel just added some gdb remote logging that is easier to access than >>> the way I had it rigged up before. If you end up getting stuck, if you get >>> the output log from either the lldb-server side, that would probably help >>> figure out what's getting stuck. But I wouldn't bother with that if you can >>> rule out that something with the a.out is going wrong first. >>> >>> -Todd >>> >>> On Wed, Feb 3, 2016 at 3:16 PM, Siva Chandra <sivachan...@google.com> >>> wrote: >>>> >>>> Yes, there is something like that but I am unable to recollect. >>>> However, I do not think Zach's problem is that. He is able to get all >>>> but 2 of the tests passing. >>>> >>>> Zach, is it possible for you to run with clang-3.5? >>>> >>>> On Wed, Feb 3, 2016 at 3:05 PM, Todd Fiala <todd.fi...@gmail.com> wrote: >>>> > (Security around ptrace). >>>> > >>>> > On Wed, Feb 3, 2016 at 3:04 PM, Todd Fiala <todd.fi...@gmail.com> >>>> > wrote: >>>> >> >>>> >> Hmm I wonder if your lldb-server is able to attach to processes? >>>> >> Siva, we >>>> >> used to have some kind of kernel flag or something that would allow >>>> >> attaching to a process that was launched by something else. I don't >>>> >> recall >>>> >> exactly what it was off the top of my head, but I wonder if Zachary >>>> >> needs >>>> >> that? >>>> >> >>>> >> -Todd >>>> >> >>>> >> On Wed, Feb 3, 2016 at 3:02 PM, Zachary Turner via lldb-dev >>>> >> <lldb-dev@lists.llvm.org> wrote: >>>> >>> >>>> >>> In my logs I'm seeing this: >>>> >>> >>>> >>> UNSUPPORTED: LLDB >>>> >>> >>>> >>> (/usr/local/google_ssd/src/llvm/build/ninja_release/bin/clang-3.9-x86_64) >>>> >>> :: >>>> >>> test_inferior_print_exit_debugserver_dwo >>>> >>> (TestLldbGdbServer.LldbGdbServerTestCase) (debugserver tests) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google/home/zturner/ssd/src/llvm/tools/lldb/test/dotest.py", >>>> >>> line 7, in <module> >>>> >>> lldbsuite.test.run_suite() >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/dotest.py", >>>> >>> line 1089, in run_suite >>>> >>> resultclass=test_result.LLDBTestResult).run(configuration.suite) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/runner.py", >>>> >>> line 162, in run >>>> >>> test(result) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/suite.py", >>>> >>> line 65, in __call__ >>>> >>> return self.run(*args, **kwds) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/suite.py", >>>> >>> line 85, in run >>>> >>> self._wrapped_run(result) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/suite.py", >>>> >>> line 115, in _wrapped_run >>>> >>> test._wrapped_run(result, debug) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/suite.py", >>>> >>> line 117, in _wrapped_run >>>> >>> test(result) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/case.py", >>>> >>> line 433, in __call__ >>>> >>> return self.run(*args, **kwds) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/case.py", >>>> >>> line 361, in run >>>> >>> success = self.runMethod(testMethod, result) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/third_party/Python/module/unittest2/unittest2/case.py", >>>> >>> line 391, in runMethod >>>> >>> testMethod() >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/lldbtest.py", >>>> >>> line 1900, in dwarf_test_method >>>> >>> return attrvalue(self) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/decorators.py", >>>> >>> line 112, in wrapper >>>> >>> func(*args, **kwargs) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py", >>>> >>> line 250, in test_inferior_print_exit_llgs >>>> >>> self.inferior_print_exit() >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/tools/lldb-server/TestLldbGdbServer.py", >>>> >>> line 237, in inferior_print_exit >>>> >>> context = self.expect_gdbremote_sequence() >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/tools/lldb-server/gdbremote_testcase.py", >>>> >>> line 549, in expect_gdbremote_sequence >>>> >>> return expect_lldb_gdbserver_replay(self, self.sock, >>>> >>> self.test_sequence, timeout_seconds, self.logger) >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/tools/lldb-server/lldbgdbserverutils.py", >>>> >>> line 252, in expect_lldb_gdbserver_replay >>>> >>> context["O_content"] = pump.get_accumulated_output() >>>> >>> File >>>> >>> >>>> >>> "/usr/local/google_ssd/src/llvm/tools/lldb/packages/Python/lldbsuite/test/tools/lldb-server/socket_packet_pump.py", >>>> >>> line 81, in __exit__ >>>> >>> traceback.print_stack() >>>> >>> lldb-server exiting... >>>> >>> >>>> >>> Could this be related to the timeout I'm seeing? Has anyone seen >>>> >>> this >>>> >>> before? It doesn't appear flaky, happens every time. >>>> >>> >>>> >>> On Wed, Feb 3, 2016 at 2:57 PM Siva Chandra <sivachan...@google.com> >>>> >>> wrote: >>>> >>>> >>>> >>>> Our bot is running on Ubuntu 14.04 and is green: >>>> >>>> http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake >>>> >>>> >>>> >>>> One thing though, the bot does not run the testsuite with >>>> >>>> clang-3.6. >>>> >>>> About the unexpected successes, they are very likely tests which >>>> >>>> were >>>> >>>> found to be flaky and marked as expectedFailure (or something >>>> >>>> similar) >>>> >>>> to keep the bot green. Even the bot logs show these unexpected >>>> >>>> successes. >>>> >>>> >>>> >>>> On Wed, Feb 3, 2016 at 2:50 PM, Zachary Turner via lldb-dev >>>> >>>> <lldb-dev@lists.llvm.org> wrote: >>>> >>>> > >>>> >>>> > On Linux I get the following test results: >>>> >>>> > >>>> >>>> > UNEXPECTED SUCCESS: test_and_run_command_dwarf >>>> >>>> > (lang/c/const_variables/TestConstVariables.py) >>>> >>>> > UNEXPECTED SUCCESS: test_and_run_command_dwo >>>> >>>> > (lang/c/const_variables/TestConstVariables.py) >>>> >>>> > UNEXPECTED SUCCESS: test_command_script_immediate_output_dwarf >>>> >>>> > >>>> >>>> > >>>> >>>> > (functionalities/command_script_immediate_output/TestCommandScriptImmediateOutput.py) >>>> >>>> > UNEXPECTED SUCCESS: test_command_script_immediate_output_dwo >>>> >>>> > >>>> >>>> > >>>> >>>> > (functionalities/command_script_immediate_output/TestCommandScriptImmediateOutput.py) >>>> >>>> > UNEXPECTED SUCCESS: test_fd_leak_basic_dwarf >>>> >>>> > (functionalities/avoids-fd-leak/TestFdLeak.py) >>>> >>>> > UNEXPECTED SUCCESS: test_fd_leak_basic_dwo >>>> >>>> > (functionalities/avoids-fd-leak/TestFdLeak.py) >>>> >>>> > UNEXPECTED SUCCESS: test_fd_leak_log_dwarf >>>> >>>> > (functionalities/avoids-fd-leak/TestFdLeak.py) >>>> >>>> > UNEXPECTED SUCCESS: test_fd_leak_log_dwo >>>> >>>> > (functionalities/avoids-fd-leak/TestFdLeak.py) >>>> >>>> > UNEXPECTED SUCCESS: test_fd_leak_multitarget_dwarf >>>> >>>> > (functionalities/avoids-fd-leak/TestFdLeak.py) >>>> >>>> > UNEXPECTED SUCCESS: test_fd_leak_multitarget_dwo >>>> >>>> > (functionalities/avoids-fd-leak/TestFdLeak.py) >>>> >>>> > UNEXPECTED SUCCESS: test_file_scope_lookup_with_run_command_dwarf >>>> >>>> > (lang/cpp/namespace/TestNamespaceLookup.py) >>>> >>>> > UNEXPECTED SUCCESS: test_file_scope_lookup_with_run_command_dwo >>>> >>>> > (lang/cpp/namespace/TestNamespaceLookup.py) >>>> >>>> > UNEXPECTED SUCCESS: test_lldbmi_gdb_set_target_async_off >>>> >>>> > (tools/lldb-mi/TestMiGdbSetShow.py) >>>> >>>> > UNEXPECTED SUCCESS: test_lldbmi_process_output >>>> >>>> > (tools/lldb-mi/syntax/TestMiSyntax.py) >>>> >>>> > UNEXPECTED SUCCESS: >>>> >>>> > test_lldbmi_settings_set_target_run_args_after >>>> >>>> > (tools/lldb-mi/interpreter/TestMiInterpreterExec.py) >>>> >>>> > UNEXPECTED SUCCESS: >>>> >>>> > test_lldbmi_settings_set_target_run_args_before >>>> >>>> > (tools/lldb-mi/interpreter/TestMiInterpreterExec.py) >>>> >>>> > UNEXPECTED SUCCESS: test_restart_bug_dwarf >>>> >>>> > (functionalities/signal/raise/TestRaise.py) >>>> >>>> > UNEXPECTED SUCCESS: test_restart_bug_dwo >>>> >>>> > (functionalities/signal/raise/TestRaise.py) >>>> >>>> > UNEXPECTED SUCCESS: >>>> >>>> > test_scope_lookup_before_using_with_run_command_dwo >>>> >>>> > (lang/cpp/namespace/TestNamespaceLookup.py) >>>> >>>> > TIMEOUT: test_qThreadInfo_matches_qC_attach_llgs_dwo >>>> >>>> > (tools/lldb-server/TestLldbGdbServer.py) >>>> >>>> > TIMEOUT: test_watchpoint_delay_watchpoint_one_breakpoint_dwarf >>>> >>>> > >>>> >>>> > (functionalities/thread/concurrent_events/TestConcurrentEvents.py) >>>> >>>> > >>>> >>>> > >>>> >>>> > This is a ton of unexpected successes. Does anyone regularly run >>>> >>>> > the >>>> >>>> > test >>>> >>>> > suite on Linux? Is this normal? I also notice that it takes >>>> >>>> > almost >>>> >>>> > 30 >>>> >>>> > minutes to complete, and I get these timeouts: >>>> >>>> > >>>> >>>> > TIMEOUT: test_qThreadInfo_matches_qC_attach_llgs_dwo >>>> >>>> > (tools/lldb-server/TestLldbGdbServer.py) >>>> >>>> > TIMEOUT: test_watchpoint_delay_watchpoint_one_breakpoint_dwarf >>>> >>>> > >>>> >>>> > (functionalities/thread/concurrent_events/TestConcurrentEvents.py) >>>> >>>> > >>>> >>>> > Are these known issues? I'm using Ubuntu 14.04 and building >>>> >>>> > tests >>>> >>>> > with >>>> >>>> > Clang 3.6 >>>> >>>> > >>>> >>>> > _______________________________________________ >>>> >>>> > lldb-dev mailing list >>>> >>>> > lldb-dev@lists.llvm.org >>>> >>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>> >>>> > >>>> >>> >>>> >>> >>>> >>> _______________________________________________ >>>> >>> lldb-dev mailing list >>>> >>> lldb-dev@lists.llvm.org >>>> >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev >>>> >>> >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> -Todd >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > -Todd >>> >>> >>> >>> >>> -- >>> -Todd >> >> >> >> >> -- >> -Todd > > > > > -- > -Todd > > _______________________________________________ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
