I thought about this more overnight and I'm more convinced that lit and lldb-mi 
make a great pair.  lldb-mi is a programmatic text format that isn't subject to 
the whims of command-line UI design over the years; well tests written in terms 
of MI will be resilient and stable.  lldb-mi MUCH more closely matches a 
non-lldb developer's view of how a debugger works.  The syntax is different 
from what they use, but it isn't hard to figure out.  I haven't touched MI in 
four or five years and it only took me a couple minutes to figure out how to do 
run to a breakpoint and print a variable:

-file-exec-and-symbols "/tmp/a.out"

-break-insert a.c:10




-var-create var1 * c

(I omitted the responses from all the command except -break-insert and 
-var-create to make it easier to read, but go run lldb-mi and play with it 
yourself or read the gdb MI documentation on the web.  It's a really simple 
format to work in.)

Several of the proposal have been special commands in the lldb command line 
driver that have non-changing output styles or the like.  Why re-invent 
something that already exists?  This makes a ton more sense on every front.  
Yes, it means you can't copy and paste your lldb debug session into a test case 
--- but we've all agreed that that's not something we want anyone doing anyway.

I'm in favor of lit + lldb-mi and think this would add a lot of value as an 
additional way for people to write test cases.


> On Sep 15, 2016, at 7:40 PM, Zachary Turner <ztur...@google.com> wrote:
> One thing I wonder about. It seems like everyone is mostly on the same page 
> about command line output .
> What about input? Would anyone have objections to a test which ran a few 
> commands to get the debugger into a particular state before doing something 
> to verify the output? Let's assume I'm waving my hands about this last step 
> but that it doesn't involve scraping the output of a debugger command 
> Maybe this will never even be an issue, but I just want to make sure everyone 
> is on the same page and that the objections are:
> a) specific to command OUTPUT, not input (i.e. it's ok to have a test run 
> "break set -n main")
> b) specific to *non trivial* output (checking that "p 5" displays 5 is ok)
> c) specific to the the output of the *user* command line interface, so that 
> some hypothetical other command line interface (which again I'm being hand 
> wavy about) would not be subject to the same objections.
> On Thu, Sep 15, 2016 at 7:28 PM Jason Molenda <jmole...@apple.com> wrote:
> > On Sep 15, 2016, at 8:02 AM, Zachary Turner <ztur...@google.com> wrote:
> >
> >
> > It sounds like your goal is also "tests have to use the SB API and no other 
> > API", which if so I think that's counterproductive.   More productive, IMO, 
> > would be being open to any alternative that addresses the concerns you have 
> > with command-line tests.  There are more than 2 ways to skin a cat, so to 
> > speak.
> Thinking about this a bit, another approach would be to do lit tests on top 
> of lldb-mi.  MI is a structured format for the debugger and a UI to 
> communicate back and forth with a simple text markup language (it would be 
> JSON if it were being done today, but it was added to gdb eighteen years ago, 
> so it's not).  The commands correspond to the commands a debugger user would 
> think to use -- no need to understand the structure of how lldb is 
> implemented, like with the SB API.  The format is a little unusual for a 
> human to type, but back when we supported gdb at Apple we worked in MI all 
> the time (it was used to communicate between Xcode, our IDE, and gdb) by hand 
> when testing and it was fine. "-exec-run" instead of run, that kind of thing. 
>  I think there are four dozens different commands.
> lldb-mi itself uses SB API.  And the concerns about hardcoding command line 
> UI don't apply, it's a key-value format intended for computers, no one is 
> going to add an extra space character to anything -- the most it changes is 
> that new key-value pairs are added to responses.
> I agree there are many acceptable ways to write lit tests that don't involve 
> lldb command line scraping, and I'm all in favor of using lit with tests that 
> don't do that.  Of course the patch we're discussing has lit test examples 
> that contradict our own policy on writing tests.  Once lit is supported in 
> lldb, are we going to reject any testsuite additions that depend on the text 
> output from the command line lldb?  If we're all on the same page here, then 
> I have no reservations.
> Just to say out loud the future I can easily see:  We add lit, then we have 
> people helpfully write a few dozen tests in lit that depend on the command 
> line debugger output.  Those patches have to be rejected.
> J

lldb-commits mailing list

Reply via email to