> On Mar 12, 2015, at 6:00 PM, Zachary Turner <ztur...@google.com> wrote: > > Well, like I said. I'm just thinking :) No need to worry > > Back to the original question, is it as easy as it seems to just create one > target in Xcode that manually includes each file recursively in the subtree? > So it just builds one executable? >
I don't think so. You can ADD a folder of sources to a project, but that just makes all the files available to the project. You then have to manually tell Xcode which files build into which targets. That's pretty easy to do, but I don't know of a way to get it to "include all .c files in the current target." Jim (I removed Chandler & Justin 'cause I doubt they care about Xcode...) > On Thu, Mar 12, 2015 at 5:56 PM <jing...@apple.com> wrote: > The lldbinline tests are an okay way to write a very simple class of tests. > But they will not suffice for many of the tests we need to write. I am > actually not a big fan of these tests because when they fail it is a royal > pain to reproduce the steps that led to the failure. I don't think making a > wholly different runner to run this is going to make that situation any > better. > > Jim > > > > On Mar 12, 2015, at 5:38 PM, Zachary Turner <ztur...@google.com> wrote: > > > > Well, as a quick example of where I think there's a considerable amount of > > overlap between the high level model of how the test operates is the case > > of the lldbinline tests. > > > > On Thu, Mar 12, 2015 at 5:28 PM <jing...@apple.com> wrote: > > > > > On Mar 12, 2015, at 5:06 PM, Zachary Turner <ztur...@google.com> wrote: > > > > > > Wasn't really trying to get into an extended discussion about this, but > > > FWIW I definitely realize that lldb's tests are more complicated than > > > what lit currently supports. But that's why I said "even if it meant > > > extending lit". It was mostly just a general comment about how it's nice > > > if everyone is focused on making one thing better instead of everyone > > > having different things. > > > > > > > Depending on how different the different things are. Compiler tests tend > > to have input, output and some machine that converts the input to the > > output. That is one very particular model of testing. Debugger tests need > > to do: get to stage 1, if that succeeded, get to stage 2, if that > > succeeded, etc. Plus there's generally substantial setup code to get > > somewhere interesting, so while you are there you generally try to test a > > bunch of similar things. Plus, the tests often have points where there are > > several success cases, but each one requires a different "next action", > > stepping being the prime example of this. These are very different models > > and I don't see that trying to smush the two together would be a fruitful > > exercise. > > > > Jim > > > > > As for specifics, my understanding is that lit parallelizes better (so > > > running tests is faster), understands how to build programs (so doesn't > > > require makefiles), and has a richer language for specifying how and > > > under what circumstances different tests should be run. It's also > > > familiar to other LLVM developers (so encourages cross-collaboration), > > > and allows one to write self-contained tests with the program to test and > > > the check in a single file (less maintenance). > > > > > > In any case, I'm really not an expert on lit, so +bogner and +chandlerc > > > in case they want to chime in. I do think it's at least worth thinking > > > about whether lit *could* be extended to meet LLDB's needs -- if nothing > > > else as a thought exercise, and maybe learning more about how it works > > > would give us some ideas to make our own test suite better. > > > > > > > > > On Thu, Mar 12, 2015 at 4:39 PM <jing...@apple.com> wrote: > > > > > > > On Mar 12, 2015, at 4:08 PM, Zachary Turner <ztur...@google.com> wrote: > > > > > > > > Oh I'm all for reusing as much of the existing mechanism as possible. > > > > Was just stating how the CMake worked as a discussion point. Another > > > > possibility would be to just have the Xcode project build one > > > > executable that pulls in sources recursively from the entire subtree. > > > > Is this as easy in Xcode as just adding all sources from a subfolder to > > > > a single target? > > > > > > > > > > > One day far off in the future it would be nice if all of LLDB's tests > > > > were ported to lit (even if that meant extending lit to make it do what > > > > we needed it to do), > > > > > > Why would this be nice? It looks like lit is a good test runner for > > > tests that have some input, do something with the input, produce an > > > output and check that output is matches some pattern. That is not at all > > > what the lldb tests look like. They often have to do complex dances - > > > for instance depending on how the line tables come out there are many > > > "correct" ways to step through code. If you are going to test this > > > you've got to do "step, if I got to a close bracket, step again, if I got > > > past it don't. Etc... > > > > > > I see no benefit in extending a simple runner like lit to do the complex > > > dances the lldb testsuite sometimes has to do. I'm all for sharing, but > > > it is also okay to have two implementations of some functionality if the > > > two uses are sufficiently different, and this certainly seems like one of > > > those cases. > > > > > > > so I can definitely see some value in hooking lit up to the Xcode build > > > > so it does everything the CMake build does. I'll have to look into > > > > exactly what steps the CMake and/or autoconf build are taking, but I > > > > suspect it's going to involve running CMake from a script, so not very > > > > desirable. I'm still learning a lot of this stuff though, so there may > > > > be a better way. Either way, I'll have to look into it a little bit. > > > > > > > > > Jim > > > > > > > > > > > > > > In the meantime, if running unit tests from Xcode is not part of > > > > anyone's usual workflow, can I remove it for now? > > > > > > > > On Thu, Mar 12, 2015 at 4:01 PM <jing...@apple.com> wrote: > > > > I'm not sure if this is what you meant, but I don't see a lot of value > > > > in making an Xcode project that has targets for each of the gtest > > > > binaries, and then tries to run the tests. Seems to me it would be > > > > better if the gtest project just invokes whatever mechanism the cmake > > > > build would do to run the tests. That's just another set of things to > > > > keep in sync. > > > > > > > > It is sufficient to have a target that just does whatever steps > > > > cmake/lit do to build the gtests & run them, if that is possible. I > > > > guess if you can't do this without running cmake in the lldb top-level > > > > directory that would be a problem. But it still seems better to me to > > > > wire that up, than to have to add tests to both Xcode & cmake. > > > > > > > > Jim > > > > > > > > > > > > > > > > > > > > > > > > > On Mar 12, 2015, at 3:46 PM, Zachary Turner <ztur...@google.com> > > > > > wrote: > > > > > > > > > > So I'm guessing the scheme runs do-gtest.py. I'd like to delete that > > > > > file as well as all the Makefiles in the directory if possible. It > > > > > seems like these files should be built using the normal Xcode build > > > > > system the same way the rest of LLDB is built. > > > > > > > > > > The way the CMake does it is that each test folder generates a new > > > > > executable. So right now it will build HostTests.exe, > > > > > ProcessLinuxTests.exe, and UtilityTests.exe. And then CMake will > > > > > invoke lit (the LLVM test runner) to run each of the executables one > > > > > by one and print the output. > > > > > > > > > > I'm not sure if that's easy or feasible to do in the Xcode build. I > > > > > kind of don't want to leave this do-gtest.py and Makefiles in the > > > > > build though, because the more of this stuff we have the more > > > > > maintenance it is, and things tend to rot. > > > > > > > > > > > > > > > On Thu, Mar 12, 2015 at 3:23 PM <jing...@apple.com> wrote: > > > > > Xcode has "projects" and then "workspaces" and "schemes". Workspaces > > > > > aggregate projects. Schemes exist in both workspaces and projects > > > > > and are the way to say "do something with some of the stuff referred > > > > > to by this project/workspace." So the way to do this formally is to > > > > > have the gtest scheme build & run the tests from the gtest project. > > > > > > > > > > The lldb.xcworkspace file does reference the gtest xcode project, and > > > > > it has a scheme for the gtest. > > > > > > > > > > Not sure what the scheme does yet, I'll look in a few minutes if > > > > > nobody beats me to it, I'm in the middle of things right now. > > > > > > > > > > Jim > > > > > > > > > > > > > > > > > > > > > On Mar 12, 2015, at 2:41 PM, Zachary Turner <ztur...@google.com> > > > > > > wrote: > > > > > > > > > > > > In lldb/gtest there is a gtest.xcodeproj folder with what I guess > > > > > > is an Xcode project. If I understand the way Xcode works, the way > > > > > > to use this is by opening this in another instance of Xcode > > > > > > separate from your normal LLDB project, and then building it. Is > > > > > > this right? > > > > > > > > > > > > I have a patch that moves some files around, and if nobody is using > > > > > > this Xcode project, I would like to delete it. Then, after I get > > > > > > the tests up and running in the CMake build, we can add it to the > > > > > > "real" Xcode project as a separate target similar to how you > > > > > > currently run the LLDB Test suite. > > > > > > > > > > > > Any objections to deleting the Xcode project? > > > > > > > > > > > > > > > _______________________________________________ lldb-dev mailing list lldb-dev@cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
