Re: [CMake] CTEST_CUSTOM_TESTS_IGNORE regex?
On 2013-01-31 01:19, Leif Walsh wrote:
In CTestCustom.cmake, I use CTEST_CUSTOM_MEMCHECK_IGNORE and
CTEST_CUSTOM_TESTS_IGNORE to turn some long-running tests on and off (on
for nightlies, off for development test cycles).
I recently broke apart a rather large shell script that was used to run a
test under many scenarios into something like
foreach (a 0 1)
foreach (b 0 1)
foreach (c 0 1)
add_test(NAME big_test_${a}${b}${c} COMMAND big_test ${a} ${b} ${c})
endforeach (c)
endforeach (b)
endforeach (a)
...you get the idea. It generates somewhere in the realm of 2000 test
cases. Now they run in parallel, which is great, but I can no longer turn
off this suite of tests easily.
It would be great to do something like
list(APPEND CTEST_CUSTOM_TESTS_IGNORE 'big_test_.*')
Is this or something like it supported?
Well, since we're talking about regular expressions, the list
separator is '|'. Have you tried:
set(CTEST_CUSTOM_TESTS_IGNORE
${CTEST_CUSTOM_TESTS_IGNORE}|big_test_.*)
...?
(Supporting an *actual* list of regular expressions seems like a nice
enhancement request, but hopefully the above can work around it for now.)
--
Matthew
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTEST_CUSTOM_TESTS_IGNORE regex?
This variable seems to be interpreted as a normal cmake ;-delimited list. I
currently use it that way for other tests. I haven't tried building it as a
regex but I kind of doubt that would work, since I know it works as a list.
If I have to I can duplicate the loops in CTestCustom.cmake but that has other
issues (code duplication, size of output when running with -V, maybe speed).
Sent from my iPhone
On Jan 31, 2013, at 14:02, Matthew Woehlke matthew.woeh...@kitware.com wrote:
On 2013-01-31 01:19, Leif Walsh wrote:
In CTestCustom.cmake, I use CTEST_CUSTOM_MEMCHECK_IGNORE and
CTEST_CUSTOM_TESTS_IGNORE to turn some long-running tests on and off (on
for nightlies, off for development test cycles).
I recently broke apart a rather large shell script that was used to run a
test under many scenarios into something like
foreach (a 0 1)
foreach (b 0 1)
foreach (c 0 1)
add_test(NAME big_test_${a}${b}${c} COMMAND big_test ${a} ${b} ${c})
endforeach (c)
endforeach (b)
endforeach (a)
...you get the idea. It generates somewhere in the realm of 2000 test
cases. Now they run in parallel, which is great, but I can no longer turn
off this suite of tests easily.
It would be great to do something like
list(APPEND CTEST_CUSTOM_TESTS_IGNORE 'big_test_.*')
Is this or something like it supported?
Well, since we're talking about regular expressions, the list separator is
'|'. Have you tried:
set(CTEST_CUSTOM_TESTS_IGNORE
${CTEST_CUSTOM_TESTS_IGNORE}|big_test_.*)
...?
(Supporting an *actual* list of regular expressions seems like a nice
enhancement request, but hopefully the above can work around it for now.)
--
Matthew
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTEST_CUSTOM_TESTS_IGNORE regex?
On 2013-01-31 14:16, Leif Walsh wrote:
This variable seems to be interpreted as a normal cmake ;-delimited list.
Ah, I see. I misread your message as it is already a regex and didn't
take lists. (Which is a little odd that it *isn't* a regex, as the CTest
command line takes a regex...)
If you have a custom dashboard script, you might be able to change the
test section to run ctest with '-E'... but I've never attempted to do
such a thing.
Or you could try running '${CTEST_COMMAND} -N' and parsing the output to
build the list of tests to ignore...
--
Matthew
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTEST_CUSTOM_TESTS_IGNORE regex?
Ah I see. Yep, it would both make sense and work perfectly if it was a regex.
Sadly it would also break stuff. Maybe CTEST_CUSTOM_TESTS_IGNORE_REGEX would be
a sane feature request.
Sent from my iPhone
On Jan 31, 2013, at 14:51, Matthew Woehlke matthew.woeh...@kitware.com wrote:
On 2013-01-31 14:16, Leif Walsh wrote:
This variable seems to be interpreted as a normal cmake ;-delimited list.
Ah, I see. I misread your message as it is already a regex and didn't take
lists. (Which is a little odd that it *isn't* a regex, as the CTest command
line takes a regex...)
If you have a custom dashboard script, you might be able to change the test
section to run ctest with '-E'... but I've never attempted to do such a thing.
Or you could try running '${CTEST_COMMAND} -N' and parsing the output to
build the list of tests to ignore...
--
Matthew
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
[CMake] CTEST_CUSTOM_TESTS_IGNORE regex?
In CTestCustom.cmake, I use CTEST_CUSTOM_MEMCHECK_IGNORE and
CTEST_CUSTOM_TESTS_IGNORE to turn some long-running tests on and off (on
for nightlies, off for development test cycles).
I recently broke apart a rather large shell script that was used to run a
test under many scenarios into something like
foreach (a 0 1)
foreach (b 0 1)
foreach (c 0 1)
add_test(NAME big_test_${a}${b}${c} COMMAND big_test ${a} ${b} ${c})
endforeach (c)
endforeach (b)
endforeach (a)
...you get the idea. It generates somewhere in the realm of 2000 test
cases. Now they run in parallel, which is great, but I can no longer turn
off this suite of tests easily.
It would be great to do something like
list(APPEND CTEST_CUSTOM_TESTS_IGNORE 'big_test_.*')
Is this or something like it supported?
I haven't tried it yet so maybe it just works, but I kind of doubt it.
I didn't see anything pop out in the docs but maybe I missed it.
--
Cheers,
Leif
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
