Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev
That's a good point, Tamas. I use (so I claim) the same all upper-case markers for the test result details. Including, not using XPASS but rather UNEXPECTED SUCCESS for unexpected successes. (The former would trigger the lit script IIRC to parse that as a failing-style result). The intent is

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev
Specifically, the markers for issue details are: FAIL ERROR UNEXPECTED SUCCESS TIMEOUT (These are the fourth field in the array entries (lines 275 - 290) of packages/Python/lldbsuite/test/basic_results_formatter.py). -Todd On Wed, Dec 9, 2015 at 8:04 AM, Todd Fiala

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Pavel Labath via lldb-dev
If it's not too much work, I think the extra bit of noise will not be a problem. But I don't think it is really necessary either. I assume the actual flip will be a small change that we can back out easily if we notice troubles... After a sufficient grace period we can remove the old formatter

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev
These went in as: r255130 - turn it on by default r255131 - create known issues. This one is to be reverted if all 3 types show up properly. On Wed, Dec 9, 2015 at 9:41 AM, Todd Fiala wrote: > It is a small change. > > I almost have all the trial tests ready, so I'll

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Tamas Berghammer via lldb-dev
Thank you for making the experiment. It looks reasonable. For the ERROR the buildbot detected it and it will fail the build but it isn't listed in the list of failing tests what should be fixed. After this experiment I think it is fine to change the default output formatter from our side. Tamas

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev
Great, thanks Tamas! I left the default turned on, and just essentially removed the issues by parking them as .py.parked files. That way we can flip them on in the future if we want to verify a testbot's detection of these. I will be going back to the xUnit Results formatter and making sure it

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev
The reports look good at the test level: http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/9294 I'd say the buildbot reflection script missed the ERROR, so that is something maybe Ying can look at (the summary line in the build run), but that is unrelated AFAICT. I'm going

Re: [lldb-dev] BasicResultsFormatter - new test results summary

2015-12-09 Thread Todd Fiala via lldb-dev
Verification tests parked (i.e. disabled) here: r255134 I decided to leave them in the repo so it is faster/easier to do this in the future. -Todd On Wed, Dec 9, 2015 at 10:26 AM, Todd Fiala wrote: > The reports look good at the test level: > > >

[lldb-dev] swig generation and "six" module

2015-12-09 Thread Ted Woodward via lldb-dev
r252764 changes finishSwigPythonLLDB.py to symlink the "six" module in site-packages. six.py is a symlink to /tools/lldb/third_party/Python/module/six/six.py. This assumes I have access to this build's sources when I run lldb, which I don't - our workstations don't have access to the buildbots'

[lldb-dev] Benchmark tests

2015-12-09 Thread Zachary Turner via lldb-dev
Is anyone using the benchmark tests? None of the command line options related to the benchmark tests were claimed as being used by anyone. Which makes me wonder if the tests are even being used

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Enrico Granata via lldb-dev
Historically I would do $ ./dotest.py +b but I am not strongly attached to that workflow - it's just what I learnt the first time I needed to run one Sent from my iPhone > On Dec 9, 2015, at 2:08 PM, Zachary Turner wrote: > > When you do run the benchmark tests, what

Re: [lldb-dev] LLDB/NetBSD test suite results as of 8 Dec 2015

2015-12-09 Thread Kamil Rytarowski via lldb-dev
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 08.12.2015 17:21, Ed Maste wrote: > On 7 December 2015 at 19:28, Kamil Rytarowski via lldb-dev > wrote: >> >> I ran the LLDB test suite of the LLDB-current version on >> NetBSD/amd64 (v. 7.99.21). >> >> The results

Re: [lldb-dev] Benchmark tests

2015-12-09 Thread Jason Molenda via lldb-dev
FWIW, nope, I've never messed with the benchmark tests. > On Dec 9, 2015, at 1:22 PM, Todd Fiala wrote: > > Hey Jason, > > Are you the benchmark user? > > -Todd > > On Wed, Dec 9, 2015 at 12:32 PM, Zachary Turner via lldb-dev > wrote: > Is

Re: [lldb-dev] swig generation and "six" module

2015-12-09 Thread Ted Woodward via lldb-dev
Unfortunately, the LLDB we build needs to run on SLES 11, Ubuntu 10+ and who knows what else. I don’t think I can require they install a six package. I can make our build do a copy instead of a link, if you want to keep it a link. Are the static swig bindings checked in yet, or do we still