I don't think new test authors really need to add CMake any more so than they currently need to understand Make. Which is to say, not very much. Most Makefiles are currently 1-2 lines of code that simply does nothing other than include the common Makefile.
On the other hand, CMake defines a lot of constructs designed to support portable builds, so actually writing and maintaining that common CMake build file would be much easier. The existing Makefile-based system already doesn't require you to understand the specific compiler invocations you want. Here's 3 random Makefiles, which is hopefully representative given that I pulled them completely at random. breakpoint-commands/Makefile: LEVEL = ../../../make CXX_SOURCES := nested.cpp include $(LEVEL)/Makefile.rules functionalities/inferior-assert: LEVEL = ../../make C_SOURCES := main.c include $(LEVEL)/Makefile.rules types: LEVEL = ../make # Example: # # CXX_SOURCES := int.cpp include $(LEVEL)/Makefile.rules None of this is particularly interesting. There are a very few tests that need to do something weird. I opened 10 other random Makefiles and still didn't find any. I don't believe it would be hard to support those cases. So now instead of "understand Make" it becomes "understand CMake". Whic is already a requirement of building LLVM. If our test suite was lit-based where you actually had to write compiler invocations into the test files, I would feel differently, but that isn't what we have today. We have something that is almost a direct mapping to using CMake. On Wed, Jan 17, 2018 at 2:22 PM Adrian Prantl <apra...@apple.com> wrote: > > > > On Jan 17, 2018, at 1:45 PM, Vedant Kumar <v...@apple.com> wrote: > > > > I would prefer having all of our test dependencies tracked by CMake for > all the reasons Zach brought up, but I think we should defer that > undertaking until after the bots are more stable. We have some immediate > problems caused by stale in-tree test artifacts. As a first milestone, it'd > be great to not have to run `git clean -fdx` anymore. > > I'm pretty sure I do not want to go all the way to CMake right now, but I > am curious about your motivation: Why do you think that using CMake to > build the tests in the testsuite is a good idea? In my opinion this adds a > layer of complexity to the tests that makes it harder to understand what > exactly is happening and test authors now need to understand CMake *and* > the compiler invocations they want to execute in their tests. Do you also > share Zachary's point of view that the tests should be build artifacts that > should be kept after an incremental build? > > -- adrian > > > > > > >> On Jan 17, 2018, at 1:13 PM, Davide Italiano via lldb-dev < > lldb-dev@lists.llvm.org> wrote: > >> > >> On Wed, Jan 17, 2018 at 1:02 PM, Davide Italiano <dccitali...@gmail.com> > wrote: > >>> On Wed, Jan 17, 2018 at 12:32 PM, Adrian Prantl via lldb-dev > >>> <lldb-dev@lists.llvm.org> wrote: > >>>> Hi lldb-dev! > >>>> > >>>> I've been investigating some spurious LLDB test suite failures on > http://green.lab.llvm.org/green/ that had to do with build artifacts from > previous runs lying around in the test directories and this prompted me to > ask a couple of general noob questions about the LLDB testsuite. > >>>> > >>>> My understanding is that all execution tests are compiled using using > `make` in-tree. I.e.: the test driver (dotest.py) effectively executes > something equivalent to `cd $srcdir/packages/.../mytest && make`. And it > does this in a serial fashion for all configurations (dwarf, dSYM, dwo, > ...) and relies on the `clean` target to be implemented correctly. > >>>> > >>>> I don't understand all the design decisions that went into the LLDB > testsuite, but my naive intuition tells me that this is sub-optimal > (because of the serialization of the configurations) and dangerous (because > it relies on make clean being implemented correctly). It seems to me that a > better approach would be to create a separate build directory for each test > variant and then invoke something like `cd $builddir/test/mytest.dwarf && > make -C $srcdir/packages/.../mytest`. This way all configurations can build > in parallel, and we can simply nuke the build directory afterwards and this > way get rid of all custom implementations of the `clean` target. > > > > This sgtm as a starting point. > > > > vedant > > > >>>> > >>>> - Is this already possible, and/or am I misunderstanding how it works? > >>>> - Would this be a goal that is worthwhile to pursue? > >>>> - Is there a good reason why we are not already doing it this way? > >>>> > >>> > >>> As we're discussing lldb test suite changes, another detail that I > >>> find a little weird is that every time you execute the test suite you > >>> get a new build directory named after the time at which you run the > >>> test. > >>> It would be much much better IMHO to just have a `log/` generic > >>> directory where the failures are logged, and those who want to > >>> override this setting can just pass a flag. > >>> > >> > >> (The logs should also be moved out of tree, FWIW). > >> > >> -- > >> Davide > >> _______________________________________________ > >> lldb-dev mailing list > >> lldb-dev@lists.llvm.org > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev > >
_______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev