I would prefer having all of our test dependencies tracked by CMake for all the reasons Zach brought up, but I think we should defer that undertaking until after the bots are more stable. We have some immediate problems caused by stale in-tree test artifacts. As a first milestone, it'd be great to not have to run `git clean -fdx` anymore.
> On Jan 17, 2018, at 1:13 PM, Davide Italiano via lldb-dev > <lldb-dev@lists.llvm.org> wrote: > > On Wed, Jan 17, 2018 at 1:02 PM, Davide Italiano <dccitali...@gmail.com> > wrote: >> On Wed, Jan 17, 2018 at 12:32 PM, Adrian Prantl via lldb-dev >> <lldb-dev@lists.llvm.org> wrote: >>> Hi lldb-dev! >>> >>> I've been investigating some spurious LLDB test suite failures on >>> http://green.lab.llvm.org/green/ that had to do with build artifacts from >>> previous runs lying around in the test directories and this prompted me to >>> ask a couple of general noob questions about the LLDB testsuite. >>> >>> My understanding is that all execution tests are compiled using using >>> `make` in-tree. I.e.: the test driver (dotest.py) effectively executes >>> something equivalent to `cd $srcdir/packages/.../mytest && make`. And it >>> does this in a serial fashion for all configurations (dwarf, dSYM, dwo, >>> ...) and relies on the `clean` target to be implemented correctly. >>> >>> I don't understand all the design decisions that went into the LLDB >>> testsuite, but my naive intuition tells me that this is sub-optimal >>> (because of the serialization of the configurations) and dangerous (because >>> it relies on make clean being implemented correctly). It seems to me that a >>> better approach would be to create a separate build directory for each test >>> variant and then invoke something like `cd $builddir/test/mytest.dwarf && >>> make -C $srcdir/packages/.../mytest`. This way all configurations can build >>> in parallel, and we can simply nuke the build directory afterwards and this >>> way get rid of all custom implementations of the `clean` target. This sgtm as a starting point. vedant >>> >>> - Is this already possible, and/or am I misunderstanding how it works? >>> - Would this be a goal that is worthwhile to pursue? >>> - Is there a good reason why we are not already doing it this way? >>> >> >> As we're discussing lldb test suite changes, another detail that I >> find a little weird is that every time you execute the test suite you >> get a new build directory named after the time at which you run the >> test. >> It would be much much better IMHO to just have a `log/` generic >> directory where the failures are logged, and those who want to >> override this setting can just pass a flag. >> > > (The logs should also be moved out of tree, FWIW). > > -- > Davide > _______________________________________________ > lldb-dev mailing list > lldb-dev@lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev _______________________________________________ lldb-dev mailing list lldb-dev@lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
