Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-14 Thread Craig Scott
I've implemented the change to not run regular tests if they require a fixture with a setup test that fails. The merge request has been reopened and updated. Hopefully this can go through before the feature freeze for 3.7. ;) On Mon, Sep 12, 2016 at 11:09 PM, Brad King wrote: > On 09/10/2016 11

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-12 Thread Brad King
On 09/10/2016 11:34 AM, Craig Scott wrote: > have a crack at adding the required functionality to my fixtures branch > so that regular tests are skipped if setup tests fail [snip] > limit the new functionality just to fixtures where the required > behaviour is well defined. Sounds good to me. Ple

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-10 Thread Craig Scott
On Fri, Sep 9, 2016 at 5:14 AM, Daniel Pfeifer wrote: > On Thu, Sep 8, 2016 at 5:52 PM, Brad King wrote: > > I think if we introduce the notion of tests requiring other tests > > then a new model of test selection and enablement needs to be > > designed. Some kind of test DAG could be defined w

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-10 Thread Craig Scott
On Thu, Sep 8, 2016 at 11:52 PM, Brad King wrote: > On 09/08/2016 10:15 AM, Craig Scott wrote: > > adding a DEPENDS_ON_SUCCESS test property or something similar > > which would implement the perhaps more intuitive behaviour of not > > running dependent tests when a dependee fails. If that was do

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-08 Thread Daniel Pfeifer
On Thu, Sep 8, 2016 at 5:52 PM, Brad King wrote: > On 09/08/2016 10:15 AM, Craig Scott wrote: >> the current behaviour of DEPENDS. At the moment, if test B depends >> on test A, test B still executes if test A fails. >> It is unexpected because I'd initially have thought of DEPENDS as >> meaning I

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-08 Thread Brad King
On 09/08/2016 10:15 AM, Craig Scott wrote: > the current behaviour of DEPENDS. At the moment, if test B depends > on test A, test B still executes if test A fails. > It is unexpected because I'd initially have thought of DEPENDS as > meaning I can't run test B if test A fails, after all, B depends

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-08 Thread Craig Scott
I should also point out that another reason for not implementing the "skipping tests if the setup fails logic" relates to the current behaviour of DEPENDS. At the moment, if test B depends on test A, test B still executes if test A fails. This is both useful and unexpected at the same time. It is u

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-09-08 Thread Craig Scott
Merge request implementing this feature is now up for review here: https://gitlab.kitware.com/cmake/cmake/merge_requests/88 I ended up going with FIXTURE_... test property names rather than GROUP_... since it seemed more specific. I have not implemented the logic for skipping regular tests if any

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-08-31 Thread Craig Scott
Actually, we can't really re-use the RESOURCE_LOCK for the proposed RESOURCE_SETUP and RESOURCE_CLEANUP functionality since that would force all the tests using that resource to be serialised. So yes, a separate GROUP or something similar would seem to be needed. Let me amend my earlier proposal (w

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-08-31 Thread Craig Scott
On Tue, Aug 23, 2016 at 4:00 PM, Rolf Eike Beer wrote: > Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > > Cheeky way to get me more involved in contributing, but okay, I'll bite. > ;) > > Switching discussion to the dev list. > > > > So how would you want the feature to work? I'd s

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-08-31 Thread Craig Scott
In my original thinking, I was of the view that if a setup/cleanup step needed to be executed for each test rather than for the overall test run as a whole, then perhaps the test itself should handle that rather than CMake. The existing RESOURCE_LOCK functionality could then be used to prevent mult

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-08-25 Thread Ben Boeckel
On Tue, Aug 23, 2016 at 08:00:09 +0200, Rolf Eike Beer wrote: > Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > > So how would you want the feature to work? I'd suggest an initial set of > > requirements something like the following: > > > >- Need to support the ability to define

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-08-22 Thread Rolf Eike Beer
Am Dienstag, 23. August 2016, 10:06:01 schrieb Craig Scott: > Cheeky way to get me more involved in contributing, but okay, I'll bite. ;) > Switching discussion to the dev list. > > So how would you want the feature to work? I'd suggest an initial set of > requirements something like the following

Re: [cmake-developers] [CMake] Setup/tear down steps for CTest

2016-08-22 Thread Craig Scott
Cheeky way to get me more involved in contributing, but okay, I'll bite. ;) Switching discussion to the dev list. So how would you want the feature to work? I'd suggest an initial set of requirements something like the following: - Need to support the ability to define multiple setup and/or te