I thought about this more overnight and I'm more convinced that lit and lldb-mi make a great pair. lldb-mi is a programmatic text format that isn't subject to the whims of command-line UI design over the years; well tests written in terms of MI will be resilient and stable. lldb-mi MUCH more closely matches a non-lldb developer's view of how a debugger works. The syntax is different from what they use, but it isn't hard to figure out. I haven't touched MI in four or five years and it only took me a couple minutes to figure out how to do run to a breakpoint and print a variable:
-file-exec-and-symbols "/tmp/a.out" -break-insert a.c:10 =breakpoint-modified,bkpt={number="6",type="breakpoint",disp="keep",enabled="y",addr="0x0000000100000f70",func="main",file="a.c",fullname="/tmp/a.c",line="10",times="0",original-location="a.c:10"} -exec-run -thread-info -exec-next -var-create var1 * c ^done,name="var1",numchild="0",value="12",type="int",thread-id="1",has_more="0" (I omitted the responses from all the command except -break-insert and -var-create to make it easier to read, but go run lldb-mi and play with it yourself or read the gdb MI documentation on the web. It's a really simple format to work in.) Several of the proposal have been special commands in the lldb command line driver that have non-changing output styles or the like. Why re-invent something that already exists? This makes a ton more sense on every front. Yes, it means you can't copy and paste your lldb debug session into a test case --- but we've all agreed that that's not something we want anyone doing anyway. I'm in favor of lit + lldb-mi and think this would add a lot of value as an additional way for people to write test cases. J > On Sep 15, 2016, at 7:40 PM, Zachary Turner <ztur...@google.com> wrote: > > One thing I wonder about. It seems like everyone is mostly on the same page > about command line output . > > What about input? Would anyone have objections to a test which ran a few > commands to get the debugger into a particular state before doing something > to verify the output? Let's assume I'm waving my hands about this last step > but that it doesn't involve scraping the output of a debugger command > > Maybe this will never even be an issue, but I just want to make sure everyone > is on the same page and that the objections are: > > a) specific to command OUTPUT, not input (i.e. it's ok to have a test run > "break set -n main") > b) specific to *non trivial* output (checking that "p 5" displays 5 is ok) > c) specific to the the output of the *user* command line interface, so that > some hypothetical other command line interface (which again I'm being hand > wavy about) would not be subject to the same objections. > > > On Thu, Sep 15, 2016 at 7:28 PM Jason Molenda <jmole...@apple.com> wrote: > > > On Sep 15, 2016, at 8:02 AM, Zachary Turner <ztur...@google.com> wrote: > > > > > > It sounds like your goal is also "tests have to use the SB API and no other > > API", which if so I think that's counterproductive. More productive, IMO, > > would be being open to any alternative that addresses the concerns you have > > with command-line tests. There are more than 2 ways to skin a cat, so to > > speak. > > Thinking about this a bit, another approach would be to do lit tests on top > of lldb-mi. MI is a structured format for the debugger and a UI to > communicate back and forth with a simple text markup language (it would be > JSON if it were being done today, but it was added to gdb eighteen years ago, > so it's not). The commands correspond to the commands a debugger user would > think to use -- no need to understand the structure of how lldb is > implemented, like with the SB API. The format is a little unusual for a > human to type, but back when we supported gdb at Apple we worked in MI all > the time (it was used to communicate between Xcode, our IDE, and gdb) by hand > when testing and it was fine. "-exec-run" instead of run, that kind of thing. > I think there are four dozens different commands. > > lldb-mi itself uses SB API. And the concerns about hardcoding command line > UI don't apply, it's a key-value format intended for computers, no one is > going to add an extra space character to anything -- the most it changes is > that new key-value pairs are added to responses. > > > I agree there are many acceptable ways to write lit tests that don't involve > lldb command line scraping, and I'm all in favor of using lit with tests that > don't do that. Of course the patch we're discussing has lit test examples > that contradict our own policy on writing tests. Once lit is supported in > lldb, are we going to reject any testsuite additions that depend on the text > output from the command line lldb? If we're all on the same page here, then > I have no reservations. > > Just to say out loud the future I can easily see: We add lit, then we have > people helpfully write a few dozen tests in lit that depend on the command > line debugger output. Those patches have to be rejected. > > J _______________________________________________ lldb-commits mailing list lldb-commits@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-commits