> On Sep 19, 2016, at 3:19 PM, Zachary Turner <ztur...@google.com> wrote:
> Obviously I defer to you on whether testing via the SB API is better than 
> what GDB does or used to do.  But these are not the only two systems in the 
> world.  Having used both LLDB and LLVM's test suite extensively, I still 
> remain unconvinced that LLDB's testing situation could not be improved.  Do 
> we improve it by doing what GDB did?  Obviously not.  Can we improve it 
> further by doing something completely different than what GDB did *and* what 
> LLDB currently does?  I remain convinced that we can.

I think you misread my message.  I said:

Having a Python interface so that developers can program the debugger is very 

As evidence, see all the work the gdb folks did to add one (you have to know a 
little bit about how gdb works to see how hard this would be, but you can 
imagine a debugger written in C primarily to dump text to a terminal, think of 
how you would wrap that in Python and you'll get a sense.)

Now forget about gdb, that was only an example of how much some folks thought a 
scripting interface was worth.

Instead, focus on this good thing, the scripting interface.  We should test 
this thing that we think is valuable.

What's a good way to make sure we test that?



> On Mon, Sep 19, 2016 at 3:07 PM Jim Ingham <jing...@apple.com> wrote:
> > On Sep 19, 2016, at 2:11 PM, Zachary Turner via lldb-dev 
> > <lldb-dev@lists.llvm.org> wrote:
> >
> >
> >
> > On Mon, Sep 19, 2016 at 2:02 PM Greg Clayton <gclay...@apple.com> wrote:
> >
> > >               • Separate testing tools
> > >                       • One question that remains open is how to 
> > > represent the complicated needs of a debugger in lit tests.  Part a) 
> > > above covers the trivial cases, but what about the difficult cases?  In 
> > > https://reviews.llvm.org/D24591 a number of ideas were discussed.  We 
> > > started getting to this idea towards the end, about a separate tool which 
> > > has an interface independent of the command line interface and which can 
> > > be used to test.  lldb-mi was mentioned.  While I have serious concerns 
> > > about lldb-mi due to its poorly written and tested codebase, I do agree 
> > > in principle with the methodology.  In fact, this is the entire 
> > > philosophy behind lit as used with LLVM, clang, lld, etc.
> >
> > We have a public API... Why are we going to go and spend _any_ time on 
> > doing anything else? I just don't get it. What a waste of time. We have a 
> > public API. Lets use it. Not simple enough for people? Tough. Testing a 
> > debugger should be done using the public API except when we are actually 
> > trying to test the LLDB command line in pexpect like tests. Those and only 
> > those are fine to covert over to using LIT and use text scraping. But 95% 
> > of our testing should be done using the API that our IDEs use. Using MI? 
> > Please no.
> >
> > FWIW, I agree with you about mi.
> >
> > That said, the public api is problematic.  Here you've got an API which, 
> > once you do something to, is set in stone forever.  And yet in order to 
> > test something, you have to add it to the API.  So now, in order to test 
> > something, you have to make a permanent change to a public API that can 
> > never be undone.
> >
> > It's also a public facing API, when a lot of the stuff  we need to be able 
> > to look at and inspect is not really suitable for public consumption.
> >
> > If something is a public API that can never be changed once it's added, 
> > then there should be an extremely strict review process in order to add 
> > something to it.  It should be VERY HARD to modify (it's currently not, 
> > which is a debate for another day, but anyway...).  And yet that's 
> > fundamentally incompatible with having tests be easy to write.  When it's 
> > hard to add the thing you need to add in order to write the test, then it's 
> > hard to write the test.
> >
> > Furthermore, with a system such as the one I proposed, you could even run 
> > tests with LLDB_DISABLE_PYTHON.  IDK about you, but that would be *amazing* 
> > from my point of view.  Even though I spent months making Python 3 work, I 
> > would be happy to burn it all with fire and go back to just Python 2 if we 
> > had a viable testing strategy.
> It would not be amazing.  One of the primary benefits of lldb IS the Python 
> API.  If you don't think that's wonderful in a debugger, go look at what 
> effort it took to make that happen in gdb once lldb started providing it and 
> you'll see that it was really highly motivated.  So switching our testing 
> away from one of our very strong points would be a very bad choice IMHO.
> On the "should I add it to the SB API to test it or not question."  Most of 
> the time, when I find I need to add an API in order write a test for 
> something, the thing I'm trying to get my hands on is something that other 
> people might be interested in as well.  If we go to the mode of adding 
> affordances only for ourselves when writing for testing, we remove one of the 
> times you should be asking yourself "should I add this to the SB API so other 
> people can use it."  I'm fine with having an lldb-testing module with API's 
> based on the same methodology as the SB API's.  If you think something you 
> are adding is too esoteric to be part of the SB API put it there.  But by 
> default put it into the SB API's.
> Jim
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev@lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev

lldb-dev mailing list

Reply via email to