I thought about this more overnight and I'm more convinced that lit and lldb-mi
make a great pair. lldb-mi is a programmatic text format that isn't subject to
the whims of command-line UI design over the years; well tests written in terms
of MI will be resilient and stable. lldb-mi MUCH more closely matches a
non-lldb developer's view of how a debugger works. The syntax is different
from what they use, but it isn't hard to figure out. I haven't touched MI in
four or five years and it only took me a couple minutes to figure out how to do
run to a breakpoint and print a variable:
-file-exec-and-symbols "/tmp/a.out"
-break-insert a.c:10
=breakpoint-modified,bkpt={number="6",type="breakpoint",disp="keep",enabled="y",addr="0x0000000100000f70",func="main",file="a.c",fullname="/tmp/a.c",line="10",times="0",original-location="a.c:10"}
-exec-run
-thread-info
-exec-next
-var-create var1 * c
^done,name="var1",numchild="0",value="12",type="int",thread-id="1",has_more="0"
(I omitted the responses from all the command except -break-insert and
-var-create to make it easier to read, but go run lldb-mi and play with it
yourself or read the gdb MI documentation on the web. It's a really simple
format to work in.)
Several of the proposal have been special commands in the lldb command line
driver that have non-changing output styles or the like. Why re-invent
something that already exists? This makes a ton more sense on every front.
Yes, it means you can't copy and paste your lldb debug session into a test case
--- but we've all agreed that that's not something we want anyone doing anyway.
I'm in favor of lit + lldb-mi and think this would add a lot of value as an
additional way for people to write test cases.
J
> On Sep 15, 2016, at 7:40 PM, Zachary Turner <[email protected]> wrote:
>
> One thing I wonder about. It seems like everyone is mostly on the same page
> about command line output .
>
> What about input? Would anyone have objections to a test which ran a few
> commands to get the debugger into a particular state before doing something
> to verify the output? Let's assume I'm waving my hands about this last step
> but that it doesn't involve scraping the output of a debugger command
>
> Maybe this will never even be an issue, but I just want to make sure everyone
> is on the same page and that the objections are:
>
> a) specific to command OUTPUT, not input (i.e. it's ok to have a test run
> "break set -n main")
> b) specific to *non trivial* output (checking that "p 5" displays 5 is ok)
> c) specific to the the output of the *user* command line interface, so that
> some hypothetical other command line interface (which again I'm being hand
> wavy about) would not be subject to the same objections.
>
>
> On Thu, Sep 15, 2016 at 7:28 PM Jason Molenda <[email protected]> wrote:
>
> > On Sep 15, 2016, at 8:02 AM, Zachary Turner <[email protected]> wrote:
> >
> >
> > It sounds like your goal is also "tests have to use the SB API and no other
> > API", which if so I think that's counterproductive. More productive, IMO,
> > would be being open to any alternative that addresses the concerns you have
> > with command-line tests. There are more than 2 ways to skin a cat, so to
> > speak.
>
> Thinking about this a bit, another approach would be to do lit tests on top
> of lldb-mi. MI is a structured format for the debugger and a UI to
> communicate back and forth with a simple text markup language (it would be
> JSON if it were being done today, but it was added to gdb eighteen years ago,
> so it's not). The commands correspond to the commands a debugger user would
> think to use -- no need to understand the structure of how lldb is
> implemented, like with the SB API. The format is a little unusual for a
> human to type, but back when we supported gdb at Apple we worked in MI all
> the time (it was used to communicate between Xcode, our IDE, and gdb) by hand
> when testing and it was fine. "-exec-run" instead of run, that kind of thing.
> I think there are four dozens different commands.
>
> lldb-mi itself uses SB API. And the concerns about hardcoding command line
> UI don't apply, it's a key-value format intended for computers, no one is
> going to add an extra space character to anything -- the most it changes is
> that new key-value pairs are added to responses.
>
>
> I agree there are many acceptable ways to write lit tests that don't involve
> lldb command line scraping, and I'm all in favor of using lit with tests that
> don't do that. Of course the patch we're discussing has lit test examples
> that contradict our own policy on writing tests. Once lit is supported in
> lldb, are we going to reject any testsuite additions that depend on the text
> output from the command line lldb? If we're all on the same page here, then
> I have no reservations.
>
> Just to say out loud the future I can easily see: We add lit, then we have
> people helpfully write a few dozen tests in lit that depend on the command
> line debugger output. Those patches have to be rejected.
>
> J
_______________________________________________
lldb-commits mailing list
[email protected]
http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits
