Re: [CMake] CTest / CDash integration
Sorry, but I have to keep complaining about the documentation. Please
excuse the ranting.
Just one example from the manual:
ctest --build-options: Add extra options to the build step.
This option must be the last option with the exception of
--test-command
End of available information.
I have not been able to figure out which CTEST_* variable these options
end up in, or how I could make them actually *affect* the build.
I am also still looking for a reference of variables used by CTest.
Quite obviously it is required to set both CTEST_SOURCE_DIRECTORY and
CTEST_BINARY_DIRECTORY in your script, as CTest keeps telling you if
you
try to execute an empty script. I wonder if it wouldn't be easier if
CTest
would assume the current directory as BINARY_DIR and the script
directory
as SOURCE_DIR by default, but that might be just me.
That I have to set CTEST_COMMAND manually when I obviously just called
the
executable, conveniently passing a working invocation in argv[0]).
Strange.
CTEST_SITE / CTEST_BUILD_NAME constitute the identification sent to the
CDash server, as I found out perusing some example scripts on the Wiki,
none of which really explaining what is done. Nice, but why ain't I
told
in plain English?
As SVN user, I have to set CTEST_CHECKOUT_COMMAND myself (to something
resembling svn checkout ${MY_REPOSITORY} ${CTEST_SOURCE_DIRECTORY}.
Figured it out, works. But apparently CTEST_UPDATE_COMMAND is something
else entirely, as CTest insists on appending update --non-interactive
on its own accord...
It is possible to trial error your way around all this, but I would
REALLY prefer to have a reference list, telling me which features I
might
be missing, or how I should escape quotation marks and semicolons in a
-DCMAKE_PREFIX_PATH= statement. Or why a statement that works when
given
as CTEST_CONFIGURE_COMMAND fails when passed in ctest_configure(
OPTIONS ).
Hence, again my question: Is the man page and the half-dozen pages in
the
Wiki really all there is in freely available documentation on CTest?
Regards,
--
Martin Baute
so...@rootdirectory.de
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
Probably someone else can give you a better answer, but I ended up doing the
following:
ctest \
--build-and-test $SOURCE_DIR $BUILD_DIR \
--build-generator Unix Makefiles --build-makeprogram make \
-j $PARALLEL_JOBS \
--build-options -DCMAKE_CXX_COMPILER=$CXX -DCMAKE_C_COMPILER=$CC \
--test-command ctest -D Nightly
I.e. the CTEST_SOURCE_DIRECTORY and CTEST_BINARY_DIRECTORY variables are set
by the --build-and-test sourcedir builddir argument, and the build options
are only needed to pass extra variables to your build system.
HTH,
Johannes
On Wednesday, 10. April 2013, 16:09:10, Martin Baute wrote:
Sorry, but I have to keep complaining about the documentation. Please
excuse the ranting.
Just one example from the manual:
ctest --build-options: Add extra options to the build step.
This option must be the last option with the exception of
--test-command
End of available information.
I have not been able to figure out which CTEST_* variable these options
end up in, or how I could make them actually *affect* the build.
I am also still looking for a reference of variables used by CTest.
Quite obviously it is required to set both CTEST_SOURCE_DIRECTORY and
CTEST_BINARY_DIRECTORY in your script, as CTest keeps telling you if
you
try to execute an empty script. I wonder if it wouldn't be easier if
CTest
would assume the current directory as BINARY_DIR and the script
directory
as SOURCE_DIR by default, but that might be just me.
That I have to set CTEST_COMMAND manually when I obviously just called
the
executable, conveniently passing a working invocation in argv[0]).
Strange.
CTEST_SITE / CTEST_BUILD_NAME constitute the identification sent to the
CDash server, as I found out perusing some example scripts on the Wiki,
none of which really explaining what is done. Nice, but why ain't I
told
in plain English?
As SVN user, I have to set CTEST_CHECKOUT_COMMAND myself (to something
resembling svn checkout ${MY_REPOSITORY} ${CTEST_SOURCE_DIRECTORY}.
Figured it out, works. But apparently CTEST_UPDATE_COMMAND is something
else entirely, as CTest insists on appending update --non-interactive
on its own accord...
It is possible to trial error your way around all this, but I would
REALLY prefer to have a reference list, telling me which features I
might
be missing, or how I should escape quotation marks and semicolons in a
-DCMAKE_PREFIX_PATH= statement. Or why a statement that works when
given
as CTEST_CONFIGURE_COMMAND fails when passed in ctest_configure(
OPTIONS ).
Hence, again my question: Is the man page and the half-dozen pages in
the
Wiki really all there is in freely available documentation on CTest?
Regards,
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
ctest has long been quite under-documented
There are some open issues in the bug tracker:
http://public.kitware.com/Bug/view.php?id=10392
http://public.kitware.com/Bug/view.php?id=13836
What's needed is someone to go in and do the work to document it well.
Until that happens, your best bet is to ask specific questions here to fill in
the gaps...
HTH,
David C
On Apr 10, 2013, at 10:09 AM, Martin Baute so...@rootdirectory.de wrote:
Sorry, but I have to keep complaining about the documentation. Please
excuse the ranting.
Just one example from the manual:
ctest --build-options: Add extra options to the build step.
This option must be the last option with the exception of --test-command
End of available information.
I have not been able to figure out which CTEST_* variable these options
end up in, or how I could make them actually *affect* the build.
I am also still looking for a reference of variables used by CTest.
Quite obviously it is required to set both CTEST_SOURCE_DIRECTORY and
CTEST_BINARY_DIRECTORY in your script, as CTest keeps telling you if you
try to execute an empty script. I wonder if it wouldn't be easier if CTest
would assume the current directory as BINARY_DIR and the script directory
as SOURCE_DIR by default, but that might be just me.
That I have to set CTEST_COMMAND manually when I obviously just called the
executable, conveniently passing a working invocation in argv[0]). Strange.
CTEST_SITE / CTEST_BUILD_NAME constitute the identification sent to the
CDash server, as I found out perusing some example scripts on the Wiki,
none of which really explaining what is done. Nice, but why ain't I told
in plain English?
As SVN user, I have to set CTEST_CHECKOUT_COMMAND myself (to something
resembling svn checkout ${MY_REPOSITORY} ${CTEST_SOURCE_DIRECTORY}.
Figured it out, works. But apparently CTEST_UPDATE_COMMAND is something
else entirely, as CTest insists on appending update --non-interactive
on its own accord...
It is possible to trial error your way around all this, but I would
REALLY prefer to have a reference list, telling me which features I might
be missing, or how I should escape quotation marks and semicolons in a
-DCMAKE_PREFIX_PATH= statement. Or why a statement that works when given
as CTEST_CONFIGURE_COMMAND fails when passed in ctest_configure( OPTIONS ).
Hence, again my question: Is the man page and the half-dozen pages in the
Wiki really all there is in freely available documentation on CTest?
Regards,
--
Martin Baute
so...@rootdirectory.de
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
[CMake] CTest / CDash integration
Hi there, one thing I haven't been able to figure out yet. Quite obviously the idea is that CMake, CTest and CDash should go hand in hand. But documentation-wise, there is a big rift once we are done digging through the CMake docs. We are told that include( CTest ) in CMakeLists.txt does add CTest support to a CMake build, and indeed it does. We are told that we can download a CTestConfig.cmake from our CDash server to get our build data uploaded, and indeed it works. But when it comes to customizing, things get fuzzy. There is lots of information to be had about how you could script and configure CTest, but somehow I could neither locate a list of supported variable settings (i.e., there is no such thing as ctest --help-variables), nor does there seem to exist an explanation on how to build in a clean binary directory. I *did* figure out that there is a variable CTEST_START_WITH_EMPTY_BINARY_DIRECTORY, and that adding it to my CTestCustom.ctest indeed wipes out my binary directory - including the CMakeCache with the platform-specific settings (like CMAKE_PREFIX_PATH). I'm not (really) asking about having my hand held through all these (minor) issues, but apparently I have either missed an important part of the documentation, or it is indeed sorely lacking. Which one is it? (I've been at CMake_Testing_With_CTest and CMake_Scripting_Of_CTest in the Wiki, but somehow it doesn't really fit together all that well.) Regards, -- Martin Baute so...@rootdirectory.de -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
I realize I haven't been all that clear about my intentions. I want to keep the amount of configuration files at a minimum. Lots of the information that is required for CTest (like config settings, source path, binary path etc.) or CDash (server address, project name) are already given in CMakeLists.txt, so naturally I am looking to keep it all together instead of spreading it over a number of different files cluttering my project. The information available in the documentation is focussing on various details, but doesn't really explain the right (intended) way of putting together a CMake-built project that supports uploading CTest results to a CDash server, so I was wondering if I was missing something. Regards, -- Martin Baute so...@rootdirectory.de -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
On Tuesday, April 09, 2013 04:11:06 PM Martin Baute wrote: I realize I haven't been all that clear about my intentions. I want to keep the amount of configuration files at a minimum. Lots of the information that is required for CTest (like config settings, source path, binary path etc.) or CDash (server address, project name) are already given in CMakeLists.txt, so naturally I am looking to keep it all together instead of spreading it over a number of different files cluttering my project. The CDash settings go into the CTestConfig.cmake file and that file resides next to the CMakeLists.txt file. Its in a separate file because one can use CTest/CDash without using CMake. How are the config settings and binary path for CTest already given in the CMakeLists.txt file? Normally, a CMakeLists.txt file does not have a hardcoded binary path, nor does it have fixed config settings. Between different computers, I can set a different binary path, and set different config settings. Because those aren't known in the CMakeLists.txt file, and they can vary between machines, a machine specific CTest script can provide that information. I keep a separate repository for CTest scripts for use by various machines. Each machine uses one of the scripts and can get updates to that script before getting updates to the source it is going to work on. I also keep the bulk of the CTest script in shared files, and the machine specific scripts have minimal code before including a shared script. The information available in the documentation is focussing on various details, but doesn't really explain the right (intended) way of putting together a CMake-built project that supports uploading CTest results to a CDash server, so I was wondering if I was missing something. Regards, -- Clinton Stimpson Elemental Technologies, Inc Computational Simulation Software, LLC www.csimsoft.com -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
Am 09.04.2013 16:29, schrieb Clinton Stimpson: How are the config settings and binary path for CTest already given in the CMakeLists.txt file? Normally, a CMakeLists.txt file does not have a hardcoded binary path, nor does it have fixed config settings. Wrong wording, sorry. They are, after successful configuration, in the CMakeCache.txt. Now I'm facing two options regarding CTest: - Doing incremental builds only, omitting information e.g. on the number of compiler warnings from the data sent to CDash. - Wiping the binary directory clean - including the platform-specific configuration. Now, I could set up a CTest script that does the CMake configuration for me. As it's platform specific, it would have to have lots of if / else constructs, duplicating most of what CMakeLists.txt does. OR I could create a CTestScript.cmake.in and forward the necessary settings from CMakeLists.txt (using configure_file()). All this strikes me as somehow needlessly complicated when lots of the necessary settings *could* be handled by the code that I add when I say include( CTest ). Add to that the somewhat lackluster documentation of CTest itself (no variable docs, several command-line options under- documented etc.), and I get the feeling that I'm simply *missing* something, like, a crucial page in the Wiki or an integration guide or somesuch. From what I've seen from the CMake / CTest / CDash trinity so far, it's really big on out of the box functionality, which is why I find this issue so confusing. Regards, -- Martin Baute so...@rootdirectory.de -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
On Tuesday, April 09, 2013 04:42:57 PM Martin Baute wrote: Am 09.04.2013 16:29, schrieb Clinton Stimpson: How are the config settings and binary path for CTest already given in the CMakeLists.txt file? Normally, a CMakeLists.txt file does not have a hardcoded binary path, nor does it have fixed config settings. Wrong wording, sorry. They are, after successful configuration, in the CMakeCache.txt. Now I'm facing two options regarding CTest: - Doing incremental builds only, omitting information e.g. on the number of compiler warnings from the data sent to CDash. - Wiping the binary directory clean - including the platform-specific configuration. Now, I could set up a CTest script that does the CMake configuration for me. As it's platform specific, it would have to have lots of if / else constructs, duplicating most of what CMakeLists.txt does. OR I could create a CTestScript.cmake.in and forward the necessary settings from CMakeLists.txt (using configure_file()). All this strikes me as somehow needlessly complicated when lots of the necessary settings *could* be handled by the code that I add when I say include( CTest ). Add to that the somewhat lackluster documentation of CTest itself (no variable docs, several command-line options under- documented etc.), and I get the feeling that I'm simply *missing* something, like, a crucial page in the Wiki or an integration guide or somesuch. From what I've seen from the CMake / CTest / CDash trinity so far, it's really big on out of the box functionality, which is why I find this issue so confusing. Regards, Perhaps it would help if I explained these 2 different ways of using CTest. 1. non scripted mode $ cmake /path/to/source $ ctest -D Experimental This goes through the start/configure/build/test/submit phases using information from the CMakeLists.txt and CMakeCache.txt files. In other words, CMake information is given to CTest. 2. scripted mode $ ctest -S myscript.cmake where myscript.cmake could simply go through the start/configure/build/test/submit phases, or it can do more complex things. You'll also notice that I did not run cmake manually to set up the build tree. That is driven by myscript.cmake. That is the reason the config settings and the binary path go in myscript.cmake, because it is driving. In other words, CTest information is given to CMake. When used this way, I would not try to reverse that and feed CMake information into CTest. Apart from #1 and #2... the include(CTest) and add_test() commands in the CMakeLists.txt files only generate information files for use by CTest in scenario #1 or #2. It is to bridge the CTest and CMake tools (which can be used independently). -- Clinton Stimpson Elemental Technologies, Inc Computational Simulation Software, LLC www.csimsoft.com -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
Re: [CMake] CTest / CDash integration
Perhaps it would help if I explained these 2 different ways of using CTest. /me slaps forehead... Indeed, this clarifies quite a few things. I hadn't considered those two ways as distinct, but merely complementing each other, and wondered about the apparently conflicting documentation. I will mull this over. Regards, -- Martin Baute so...@rootdirectory.de -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
