Re: [lldb-dev] Testing through api vs. commands

2015-09-11 Thread Jim Ingham via lldb-dev
> On Sep 11, 2015, at 12:49 PM, Zachary Turner wrote: > > Sounds good. I think perhaps one reason a lot of tests are written using the > commands is because not all functionality available through commands is > available through the SB API. Adrian is working on creating some tests for > cor

Re: [lldb-dev] Testing through api vs. commands

2015-09-11 Thread Zachary Turner via lldb-dev
Sounds good. I think perhaps one reason a lot of tests are written using the commands is because not all functionality available through commands is available through the SB API. Adrian is working on creating some tests for core dump debugging on Windows right now, and there's no way to access th

Re: [lldb-dev] Testing through api vs. commands

2015-09-11 Thread Jim Ingham via lldb-dev
> On Sep 11, 2015, at 11:47 AM, Zachary Turner wrote: > > We'll probably rewrite tests that we find are failing specifically as a > result of issues like this, but I agree it's not worth re-writing everything > else except on an as-needed basis. > > To make the distinction explicit and enforc

Re: [lldb-dev] Testing through api vs. commands

2015-09-11 Thread Jim Ingham via lldb-dev
> On Sep 11, 2015, at 11:47 AM, Zachary Turner wrote: > > We'll probably rewrite tests that we find are failing specifically as a > result of issues like this, but I agree it's not worth re-writing everything > else except on an as-needed basis. > > To make the distinction explicit and enforc

Re: [lldb-dev] Testing through api vs. commands

2015-09-11 Thread Zachary Turner via lldb-dev
We'll probably rewrite tests that we find are failing specifically as a result of issues like this, but I agree it's not worth re-writing everything else except on an as-needed basis. To make the distinction explicit and enforce it kind of at an organnizational level, would it be worth creating fo

Re: [lldb-dev] Testing through api vs. commands

2015-09-11 Thread Jim Ingham via lldb-dev
I have held from the beginning that the only tests that should be written using HandleCommand are those that explicitly test command behavior, and if it is possible to write a test using the SB API you should always do it that way for the very reasons you cite. Not everybody agreed with me at f

[lldb-dev] Testing through api vs. commands

2015-09-11 Thread Zachary Turner via lldb-dev
The past few weeks I've spent a lot of time xfailing the rest of the failing tests on windows so we can enable tests to run on the bots. One thing I ran into more frequently than I would have liked is that the tests were failing not because the functionality is broken, but because the substrings b