Re: [lldb-dev] test results look typical?

[Todd Fiala via lldb-dev]

Great, thanks for filing, Dawn!

On Thu, Sep 17, 2015 at 2:08 PM,  wrote:

> On Mon, Aug 24, 2015 at 05:37:43PM -0700, via lldb-dev wrote:
> > On Mon, Aug 24, 2015 at 03:37:52PM -0700, Todd Fiala via lldb-dev wrote:
> > > On Linux on non-virtualized hardware, I currently see the failures
> below on
> > > Ubuntu 14.04.2 using a setup like this:
> > > [...]
> > >
> > > ninja check-lldb output:
> > >
> > > Ran 394 test suites (15 failed) (3.807107%)
> > > Ran 474 test cases (17 failed) (3.586498%)
> >
> > I don't think you can't trust the reporting of dosep.py's "Ran N test
> > cases", as it fails to count about 500 test cases.
>
> I finally got around to opening a bug for this - see
> https://llvm.org/bugs/show_bug.cgi?id=24869
>



-- 
-Todd
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] test results look typical?

[via lldb-dev]

On Mon, Aug 24, 2015 at 05:37:43PM -0700, via lldb-dev wrote:
> On Mon, Aug 24, 2015 at 03:37:52PM -0700, Todd Fiala via lldb-dev wrote:
> > On Linux on non-virtualized hardware, I currently see the failures below on
> > Ubuntu 14.04.2 using a setup like this:
> > [...]
> > 
> > ninja check-lldb output:
> > 
> > Ran 394 test suites (15 failed) (3.807107%)
> > Ran 474 test cases (17 failed) (3.586498%)
> 
> I don't think you can't trust the reporting of dosep.py's "Ran N test
> cases", as it fails to count about 500 test cases.  

I finally got around to opening a bug for this - see
https://llvm.org/bugs/show_bug.cgi?id=24869
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] test results look typical?

[Todd Fiala via lldb-dev]

On Tue, Aug 25, 2015 at 5:40 AM, Tamas Berghammer tbergham...@google.com
wrote:

 Going back to the original question I think you have more test failures
 then expected. As Chaoren mentioned all TestDataFormatterLibc* tests are
 failing because of a missing dependency,


Thanks, Tamas.  I'm going to be testing again today with libc++ installed.


 but I think the rest of the tests should pass (I wouldn't expect them to
 depend on libc++-dev).


I'll get a better handle on what's failing once I get rid of that first
batch.


 You can see the up to date list of failures on the Linux buildbot here:
 http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake


Ah yes,that'll be good to cross reference.


 The buildbot is running in Google Compute Engine with Linux version:
 Linux buildbot-master-ubuntu-1404 3.16.0-31-generic #43~14.04.1-Ubuntu SMP
 Tue Mar 10 20:13:38 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

 LLDB is compiled by Clang (not sure about witch version but can find out
 if somebody thinks it matters) and the inferiors are compiled with
 clang-3.5, clang-tot, gcc-4.9.2. In all tested configuration there should
 be no failure (all failing tests should be XFAIL-ed).


Ah okay good to know.  In the past IIRC I did get different failures using
clang-built vs. gcc-built lldb on Ubuntu 14.04.  The clang-built lldbs at
the time were harder to debug on Linux for one reason or another (I think
particularly if any optimizations were enabled due to loss of debuginfo,
but there might have been more).  Are you using a clang-built lldb and
debugging it reasonably well on Linux?  If so I'd just assume move over to
using clang so there's one less difference when I'm looking across
platforms.


 For the flaky tests we introduced an expectedFlaky decorator what
 executes the test twice and expects it to pass at least once,


Ah that's a good addition.  We had talked about doing something to watch
tests over time to see when it might be good to promote an XFAIL test that
is consistently passing to a static expect success test.  The flaky flag
sounds handy for those that flap.


 but it haven't been applied to all flaky test yet. The plan with the tests
 passing with unexpected success at the moment is to gather statistics
 about them and based on that mark them as expected flaky or remove the
 expected failure based on the number of failures we seen in the last few
 hundreds runs.


Ah yes that :-)  Love it.

Thanks, Tamas!



 Tamas

 On Tue, Aug 25, 2015 at 2:50 AM via lldb-dev lldb-dev@lists.llvm.org
 wrote:

 On Mon, Aug 24, 2015 at 05:37:43PM -0700, via lldb-dev wrote:
  On Mon, Aug 24, 2015 at 03:37:52PM -0700, Todd Fiala via lldb-dev wrote:
   On Linux on non-virtualized hardware, I currently see the failures
 below on
   Ubuntu 14.04.2 using a setup like this:
   [...]
  
   ninja check-lldb output:

 FYI, ninja check-lldb actually calls dosep.

   Ran 394 test suites (15 failed) (3.807107%)
   Ran 474 test cases (17 failed) (3.586498%)
 
  I don't think you can trust the reporting of dosep.py's Ran N test
  cases, as it fails to count about 500 test cases.  The only way I've
  found to get an accurate count is to add up all the Ns from Ran N tests
  in as follows:
 
  ./dosep.py -s --options -v --executable $BLDDIR/bin/lldb 21 | tee
 test_out.log
  export total=`grep -E ^Ran [0-9]+ tests? in test_out.log | awk
 '{count+=$2} END {print count}'`

 Of course, these commands assume you're running the tests from the
 lldb/test directory.

  (See comments in http://reviews.llvm.org/rL238467.)

 I've pasted (and tweaked) the relavent comments from that review here,
 where I describe a narrowed case showing how dosep fails to count all the
 test cases from one test suite in test/types.  Note that the tests were run
 on OSX, so your counts may vary.

 The final count from:
 Ran N test cases .*
 is wrong, as I'll explain below. I've done a comparison between dosep and
 dotest on a narrowed subset of tests to show how dosep can omit the test
 cases from a test suite in its count.

 Tested on subset of lldb/test with just the following directories/files
 (i.e. all others directories/files were removed):
 test/make
 test/pexpect-2.4
 test/plugins
 test/types
 test/unittest2
 # The .py files kept in test/types are as follows (so
 test/types/TestIntegerTypes.py* was removed):
 test/types/AbstractBase.py
 test/types/HideTestFailures.py
 test/types/TestFloatTypes.py
 test/types/TestFloatTypesExpr.py
 test/types/TestIntegerTypesExpr.py
 test/types/TestRecursiveTypes.py

 Tests were run in the lldb/test directory using the following commands:
 dotest:
 ./dotest.py -v
 dosep:
 ./dosep.py -s --options -v

 Comparing the test case totals, dotest correctly counts 46, but dosep
 counts only 16:
 dotest:
 Ran 46 tests in 75.934s
 dosep:
 Testing: 23 tests, 4 threads ## note: this number changes
 randonmly
 Ran 6 tests in 

Re: [lldb-dev] test results look typical?

[Todd Fiala via lldb-dev]

So specifying CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -GNinja ...

did the trick for getting rid of the libc++ issues.  I think I may try to
see if we can get those tests to make a run-time check to see if the
inferior is linked against libc++, and if not, to skip it.  We can have
lldb do it by looking at the image list.  Sound reasonable?  That seems
more fool-proof than guessing based on the compiler.

An alternative I considered and probably also might be valid to do anyway
for cases where we look at the compiler binary is to fully resolve symbolic
links before making decisions based on the binary.

Thoughts?

Separately, with the tests correctly seeing gcc now, I am down to the
following errors:

Ran 394 test suites (5 failed) (1.269036%)
Ran 451 test cases (5 failed) (1.108647%)
Failing Tests (5)
FAIL: LLDB (suite) :: TestExitDuringStep.py (Linux lldb 3.19.0-26-generic
#28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
FAIL: LLDB (suite) :: TestNumThreads.py (Linux lldb 3.19.0-26-generic
#28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
FAIL: LLDB (suite) :: TestThreadExit.py (Linux lldb 3.19.0-26-generic
#28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)

Unexpected Successes (10)
UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)
UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py (Linux
lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
x86_64 x86_64)


Some of those failures look like old friends that were failing a year ago.
Can anybody tell me anything about those failures on Linux?  Are they being
looked at?  Any hunches at to what is wrong?

Thanks!

-Todd

On Tue, Aug 25, 2015 at 10:04 AM, Todd Fiala todd.fi...@gmail.com wrote:

 Okay.

 So the culprit then is that I'm using:
 cmake -GNinja ../llvm

 with one extra flag for build type.  And cmake is then just choosing
 /usr/bin/cc.

 We could improve this by having the compiler symbolic links fully resolved:
 /usr/bin/cc - /etc/alternatives/cc - /usr/bin/gcc, which would have then
 caught that it doesn't support libc++.

 Couldn't we use gcc with libc++?  (i.e. is it sufficient to assume we
 don't have libc++ if we're using gcc?)  I have never tried that combo but I
 don't know that it is impossible.  (After all, I just added libc++-dev to
 the system, which presumably I can link against).

 On Tue, Aug 25, 2015 at 9:48 AM, Tamas Berghammer tbergham...@google.com
 wrote:

 In theory the test should be skipped when you are using gcc (cc is an
 alias for it) but we detect the type of the compiler based on the
 executable name and in case of cc we don't recognize that it is a gcc, so
 we don't skip the test.

 On Tue, Aug 25, 2015 at 5:45 PM Chaoren Lin via lldb-dev 
 lldb-dev@lists.llvm.org wrote:

 You're using CC=/usr/bin/cc. It needs to be clang for USE_LIBCPP to do
 anything. :/

 On Tue, Aug 25, 2015 at 9:20 AM, Todd Fiala todd.fi...@gmail.com
 wrote:

 Here are a couple of the failures that came up (the log output from the
 full dosep.py run).

 Let me know if that is not sufficient!

 On Tue, Aug 25, 2015 at 9:14 AM, Pavel Labath lab...@google.com
 wrote:

 There's no need to do anything fancy (yet :) ). For initial diagnosis
 the output of `./dotest.py $your_usual_options -p SomeLibcxxTest.py
 -t` should suffice.

 pl

 On 25 August 2015 at 16:45, Todd Fiala todd.fi...@gmail.com wrote:
 

Re: [lldb-dev] test results look typical?

[via lldb-dev]

On Tue, Aug 25, 2015 at 04:39:14PM -0700, Todd Fiala wrote:
 I may dig into that if nobody beats me to it.  I did the original
 multiprocessing work on dosep ~1.5 years ago and it may be doing something
 goofy.  

Cool!  It would be awesome if you could have a look - I've been meaning to dig
further but just haven't had the time.

 So far the results have been remarkably stable on the counts for me
 over the last 2 days.

They are always the same.  Try the narrowed case I described with only
the tests from test/types - you'll get the same total each time, because
the same test suite is skipped each time.

Thanks!
-Dawn
___
lldb-dev mailing list
lldb-dev@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

Re: [lldb-dev] test results look typical?

[Chaoren Lin via lldb-dev]

Sorry, kernel bug is probably the wrong word. It's a problem specific to
WMware.

On Tue, Aug 25, 2015 at 4:25 PM, Chaoren Lin chaor...@google.com wrote:

 Are you running VMware by any chance? TestStepOverWatchpoint fails on
 VMware because of a kernel bug.

 On Tue, Aug 25, 2015 at 4:17 PM, Todd Fiala todd.fi...@gmail.com wrote:

 One more data point:

 Building/testing on Ubuntu 14.04.3 built with clang-3.6 and the ld.gold
 linker yielded the following test results, bringing me down to a single
 failure (and was 1.6x faster than a Debug build with gcc-4.9 and ld.bfd, 12
 GB RAM and 6 cores allocated):

 Failing Tests (1)
 FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)

 Unexpected Successes (12)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestConstVariables.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestWatchedVarHitWhenInScope.py
 (Linux lldb 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17
 UTC 2015 x86_64 x86_64)


 I'm not yet sure if that's stable, but it's what I'm seeing on my VM.

 -Todd

 On Tue, Aug 25, 2015 at 1:56 PM, Todd Fiala todd.fi...@gmail.com wrote:

 So specifying CC=/usr/bin/gcc CXX=/usr/bin/g++ cmake -GNinja ...

 did the trick for getting rid of the libc++ issues.  I think I may try
 to see if we can get those tests to make a run-time check to see if the
 inferior is linked against libc++, and if not, to skip it.  We can have
 lldb do it by looking at the image list.  Sound reasonable?  That seems
 more fool-proof than guessing based on the compiler.

 An alternative I considered and probably also might be valid to do
 anyway for cases where we look at the compiler binary is to fully resolve
 symbolic links before making decisions based on the binary.

 Thoughts?

 Separately, with the tests correctly seeing gcc now, I am down to the
 following errors:

 Ran 394 test suites (5 failed) (1.269036%)
 Ran 451 test cases (5 failed) (1.108647%)
 Failing Tests (5)
 FAIL: LLDB (suite) :: TestExitDuringStep.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestNumThreads.py (Linux lldb 3.19.0-26-generic
 #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestStepOverWatchpoint.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestThreadExit.py (Linux lldb 3.19.0-26-generic
 #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)

 Unexpected Successes (10)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux lldb
 3.19.0-26-generic #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015
 

Re: [lldb-dev] test results look typical?

[Chaoren Lin via lldb-dev]

  Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
 Pre-14.04 I gave up on it.


We're still using libstdc++ for lldb itself. libc++ is used to compile
inferiors for the TestDataFormatterLibcc* tests. I don't actually know if
libc++ works with lldb. Sorry to get your hopes up. :(

On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala todd.fi...@gmail.com wrote:



 On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin chaor...@google.com wrote:

 The TestDataFormatterLibcc* tests require libc++-dev:

 $ sudo apt-get install libc++-dev


 Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
 Pre-14.04 I gave up on it.

 Will cmake automatically choose libc++ if it is present?  Or do I need to
 pass something to cmake to use libc++?

 Thanks, Chaoren!

 -Todd


 On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev 
 lldb-dev@lists.llvm.org wrote:


 On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner ztur...@google.com
 wrote:

 Can't comment on the failures for Linux, but I don't think we have a
 good handle on the unexpected successes.  I only added that information to
 the output about a week ago, before that unexpected successes were actually
 going unnoticed.


 Okay, thanks Zachary.   A while back we had some flapping tests that
 would oscillate between unexpected success and failure on Linux.  Some of
 those might still be in that state but maybe (!) are fixed.

 Anyone on the Linux end who happens to know if the fails in particular
 look normal, that'd be good to know.

 Thanks!



 It's likely that someone could just go in there and remove the XFAIL
 from those tests.

 On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev 
 lldb-dev@lists.llvm.org wrote:

 Hi all,

 I'm just trying to get a handle on current lldb test failures across
 different platforms.

 On Linux on non-virtualized hardware, I currently see the failures
 below on Ubuntu 14.04.2 using a setup like this:
 * stock linker (ld.bfd),
 * g++ 4.9.2
 * cmake
 * ninja
 * libstdc++

 ninja check-lldb output:

 Ran 394 test suites (15 failed) (3.807107%)
 Ran 474 test cases (17 failed) (3.586498%)
 Failing Tests (15)
 FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)

 Unexpected Successes (10)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 

Re: [lldb-dev] test results look typical?

[Todd Fiala via lldb-dev]

Ah drats!  Okay.  Baby steps :-D

On Mon, Aug 24, 2015 at 4:29 PM, Chaoren Lin chaor...@google.com wrote:

  Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
 Pre-14.04 I gave up on it.


 We're still using libstdc++ for lldb itself. libc++ is used to compile
 inferiors for the TestDataFormatterLibcc* tests. I don't actually know if
 libc++ works with lldb. Sorry to get your hopes up. :(

 On Mon, Aug 24, 2015 at 4:11 PM, Todd Fiala todd.fi...@gmail.com wrote:



 On Mon, Aug 24, 2015 at 4:01 PM, Chaoren Lin chaor...@google.com wrote:

 The TestDataFormatterLibcc* tests require libc++-dev:

 $ sudo apt-get install libc++-dev


 Ah okay, so we are working with libc++ on Ubuntu, that's good to hear.
 Pre-14.04 I gave up on it.

 Will cmake automatically choose libc++ if it is present?  Or do I need to
 pass something to cmake to use libc++?

 Thanks, Chaoren!

 -Todd


 On Mon, Aug 24, 2015 at 3:42 PM, Todd Fiala via lldb-dev 
 lldb-dev@lists.llvm.org wrote:


 On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner ztur...@google.com
 wrote:

 Can't comment on the failures for Linux, but I don't think we have a
 good handle on the unexpected successes.  I only added that information to
 the output about a week ago, before that unexpected successes were 
 actually
 going unnoticed.


 Okay, thanks Zachary.   A while back we had some flapping tests that
 would oscillate between unexpected success and failure on Linux.  Some of
 those might still be in that state but maybe (!) are fixed.

 Anyone on the Linux end who happens to know if the fails in particular
 look normal, that'd be good to know.

 Thanks!



 It's likely that someone could just go in there and remove the XFAIL
 from those tests.

 On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev 
 lldb-dev@lists.llvm.org wrote:

 Hi all,

 I'm just trying to get a handle on current lldb test failures across
 different platforms.

 On Linux on non-virtualized hardware, I currently see the failures
 below on Ubuntu 14.04.2 using a setup like this:
 * stock linker (ld.bfd),
 * g++ 4.9.2
 * cmake
 * ninja
 * libstdc++

 ninja check-lldb output:

 Ran 394 test suites (15 failed) (3.807107%)
 Ran 474 test cases (17 failed) (3.586498%)
 Failing Tests (15)
 FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)

 Unexpected Successes (10)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 
 x86_64)
 UNEXPECTED 

Re: [lldb-dev] test results look typical?

[Zachary Turner via lldb-dev]

Can't comment on the failures for Linux, but I don't think we have a good
handle on the unexpected successes.  I only added that information to the
output about a week ago, before that unexpected successes were actually
going unnoticed.

It's likely that someone could just go in there and remove the XFAIL from
those tests.

On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev 
lldb-dev@lists.llvm.org wrote:

 Hi all,

 I'm just trying to get a handle on current lldb test failures across
 different platforms.

 On Linux on non-virtualized hardware, I currently see the failures below
 on Ubuntu 14.04.2 using a setup like this:
 * stock linker (ld.bfd),
 * g++ 4.9.2
 * cmake
 * ninja
 * libstdc++

 ninja check-lldb output:

 Ran 394 test suites (15 failed) (3.807107%)
 Ran 474 test cases (17 failed) (3.586498%)
 Failing Tests (15)
 FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)

 Unexpected Successes (10)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 ninja: build stopped: subcommand failed.



 On a similar setup, although bumped up to Ubuntu 14.04.3 and now on a
 VMWare VM, everything else the same, I see a similar report from 'ninja
 check-lldb':

 Ran 394 test suites (17 failed) (4.314721%)
 Ran 474 test cases (19 failed) (4.008439%)
 Failing Tests (17)
 FAIL: LLDB (suite) :: TestAttachResume.py (Linux lldb 3.19.0-26-generic
 #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestCPPThis.py (Linux lldb 3.19.0-26-generic
 #28~14.04.1-Ubuntu SMP Wed Aug 12 14:09:17 UTC 2015 x86_64 x86_64)
 

Re: [lldb-dev] test results look typical?

[Todd Fiala via lldb-dev]

On Mon, Aug 24, 2015 at 3:39 PM, Zachary Turner ztur...@google.com wrote:

 Can't comment on the failures for Linux, but I don't think we have a good
 handle on the unexpected successes.  I only added that information to the
 output about a week ago, before that unexpected successes were actually
 going unnoticed.


Okay, thanks Zachary.   A while back we had some flapping tests that would
oscillate between unexpected success and failure on Linux.  Some of those
might still be in that state but maybe (!) are fixed.

Anyone on the Linux end who happens to know if the fails in particular look
normal, that'd be good to know.

Thanks!



 It's likely that someone could just go in there and remove the XFAIL from
 those tests.

 On Mon, Aug 24, 2015 at 3:37 PM Todd Fiala via lldb-dev 
 lldb-dev@lists.llvm.org wrote:

 Hi all,

 I'm just trying to get a handle on current lldb test failures across
 different platforms.

 On Linux on non-virtualized hardware, I currently see the failures below
 on Ubuntu 14.04.2 using a setup like this:
 * stock linker (ld.bfd),
 * g++ 4.9.2
 * cmake
 * ninja
 * libstdc++

 ninja check-lldb output:

 Ran 394 test suites (15 failed) (3.807107%)
 Ran 474 test cases (17 failed) (3.586498%)
 Failing Tests (15)
 FAIL: LLDB (suite) :: TestCPPThis.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccIterator.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibccMultiMap.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxMultiSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxSet.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterLibcxxString.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterSkipSummary.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestDataFormatterUnordered.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestMiGdbSetShowPrint.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestRegisterVariables.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestStaticVariables.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestStepNoDebug.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestTypedefArray.py (Linux rad 3.13.0-57-generic
 #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 FAIL: LLDB (suite) :: TestVectorTypesFormatting.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)

 Unexpected Successes (10)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestBatchMode.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestEvents.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestExitDuringStep.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestFdLeak.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestInferiorAssert.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiGdbSetShow.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiInterpreterExec.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestMiSyntax.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestRaise.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 UNEXPECTED SUCCESS: LLDB (suite) :: TestStubSetSID.py (Linux rad
 3.13.0-57-generic #95-Ubuntu SMP Fri Jun 19 09:28:15 UTC 2015 x86_64 x86_64)
 ninja: build stopped: subcommand failed.



 On a similar setup, although bumped up to Ubuntu 14.04.3 and now on a
 VMWare VM, everything else the same, I see a similar report from