Re: [CMake] Ctest building with Multiple Processors
I wrote this blog post a couple years ago claiming one way to run a dashboard on windows: http://cmake.blogspot.com/2012/09/one-way-to-run-dashboard-on-windows.html As with all such things, there are, of course, multiple ways to solve the problem. At the time I wrote it, I trusted Visual Studio to do its best at building the projects in parallel without passing any explicit flags to do so. If you really need to squeeze out the best build time, but you still want to use the VS compiler, I would recommend setting up an environment for using ninja in conjunction with the VS compiler. If you need more advice than all this once you hit a specific issue, do ask further questions once you have them. HTH, David C. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
Is there a blog or other web site that would walk me through setting up a Build Bot on Windows for automated building/testing and submission to CDash? Thanks Mike Jackson On Sep 5, 2014, at 2:40 PM, Bill Hoffman bill.hoff...@kitware.com wrote: On 9/5/2014 11:49 AM, Chuck Atkins wrote: Unfortunately on Windows and VS builds there's not really a simple answer. My blog might be of help: http://www.kitware.com/blog/home/post/434 -Bill -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
No, they do not behave differently. -j and --parallel are aliases for each other, using either one goes through exactly the same command line argument processing code... HTH, David C. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
Actually my interpretation of the manual says that the _tests_ will be run in
parallel. What I want is the _Build_ to be run in parallel.
Thanks
Mike Jackson
On Sep 4, 2014, at 5:14 PM, Iosif Neitzke iosif.neitzke+cm...@gmail.com wrote:
Does ctest -j jobs, or ctest --parallel jobs behave differently?
http://www.cmake.org/cmake/help/v3.0/manual/ctest.1.html
On Wed, Sep 3, 2014 at 2:31 PM, Michael Jackson
mike.jack...@bluequartz.net wrote:
I am exploring CTest/CDash with out project and I was trying to figure out
how to have CTest use all my cores for builds. So far no luck with some
snippets from Google. So far I have the following in a file called
CTestConfig.cmake at the top level of my project.
set(CTEST_PROJECT_NAME DREAM3D)
set(CTEST_NIGHTLY_START_TIME 01:00:00 UTC)
set(CTEST_DROP_METHOD http)
set(CTEST_DROP_SITE my.cdash.org)
set(CTEST_DROP_LOCATION /submit.php?project=DREAM3D)
set(CTEST_DROP_SITE_CDASH TRUE)
# Use multiple CPU cores to build
include(ProcessorCount)
ProcessorCount(N)
if(NOT N EQUAL 0)
if(NOT WIN32)
set(CTEST_BUILD_FLAGS -j${N})
endif(NOT WIN32)
set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
endif()
And then I execute CTest like this from my terminal
ctest -D Experimental
but monitoring my processors says that only a single processor is being used.
Thanks for any help
---
Mike Jackson www.bluequartz.net
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
Actually my interpretation of the manual says that the _tests_ will be run in parallel. What I want is the _Build_ to be run in parallel. Indeed that is just the parallel test execution. Parallel build is usually generator specific, controlled through CTEST_BUILD_FLAGS. For example: if you're using Unix Makefiles: ... set(CTEST_SITE wopr.kitwarein.com) set(CTEST_BUILD_NAME Fedora20_Intel-2013-SP1-U3) set(CTEST_BUILD_CONFIGURATION Release) set(CTEST_CMAKE_GENERATOR Unix Makefiles) *set(CTEST_BUILD_FLAGS -j8)* ... This will call make -j8 for the build instead of just make. If using the ninja generator, the build will be parallel by default, or you can specify a -j flag to override that number. Visual Studio gets a bit tricky though since you can paralleling at both the target and file level so if not careful, you can end up with N^2 build processes. What generator are you using? -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
What generator? All of them of course. I am trying to setup our build bots on several environments including OS X, Linux and Windows. I have ninja built and installed on the Windows machine. I would _prefer_ to use Visual Studio and ninja so that we have a sanity check to make sure both types of project files can be built correctly, configured, build and the software packaged. We are eventually going to try to get nightlies of our software going also. Thanks Mike Jackson dream3d.bluequartz.net github.com/dream3d/dream3d On Sep 5, 2014, at 9:33 AM, Chuck Atkins chuck.atk...@kitware.com wrote: Actually my interpretation of the manual says that the _tests_ will be run in parallel. What I want is the _Build_ to be run in parallel. Indeed that is just the parallel test execution. Parallel build is usually generator specific, controlled through CTEST_BUILD_FLAGS. For example: if you're using Unix Makefiles: ... set(CTEST_SITE wopr.kitwarein.com) set(CTEST_BUILD_NAME Fedora20_Intel-2013-SP1-U3) set(CTEST_BUILD_CONFIGURATION Release) set(CTEST_CMAKE_GENERATOR Unix Makefiles) set(CTEST_BUILD_FLAGS -j8) ... This will call make -j8 for the build instead of just make. If using the ninja generator, the build will be parallel by default, or you can specify a -j flag to override that number. Visual Studio gets a bit tricky though since you can paralleling at both the target and file level so if not careful, you can end up with N^2 build processes. What generator are you using? -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
The easiest option with visual studio is to use the /mp compiler flag by setting it in the CFLAGS and CXXFLAGS environment variables in your ctest script. This will parallelize each target across all CPUs. Target-level parallelism may or may not also happen since that's dependent on your local configuration and VS version; Older versions used devenv.exe to drive the build, in which case it's target-level parallelism is only configurable through the IDE, where as newer VS versions can be driven with MSBuild directly and use the /maxcpucount:N switch in the CTEST_BUILD_FLAGS. Unfortunately on Windows and VS builds there's not really a simple answer. - Chuck On Fri, Sep 5, 2014 at 11:09 AM, Michael Jackson mike.jack...@bluequartz.net wrote: What generator? All of them of course. I am trying to setup our build bots on several environments including OS X, Linux and Windows. I have ninja built and installed on the Windows machine. I would _prefer_ to use Visual Studio and ninja so that we have a sanity check to make sure both types of project files can be built correctly, configured, build and the software packaged. We are eventually going to try to get nightlies of our software going also. Thanks Mike Jackson dream3d.bluequartz.net github.com/dream3d/dream3d On Sep 5, 2014, at 9:33 AM, Chuck Atkins chuck.atk...@kitware.com wrote: Actually my interpretation of the manual says that the _tests_ will be run in parallel. What I want is the _Build_ to be run in parallel. Indeed that is just the parallel test execution. Parallel build is usually generator specific, controlled through CTEST_BUILD_FLAGS. For example: if you're using Unix Makefiles: ... set(CTEST_SITE wopr.kitwarein.com) set(CTEST_BUILD_NAME Fedora20_Intel-2013-SP1-U3) set(CTEST_BUILD_CONFIGURATION Release) set(CTEST_CMAKE_GENERATOR Unix Makefiles) set(CTEST_BUILD_FLAGS -j8) ... This will call make -j8 for the build instead of just make. If using the ninja generator, the build will be parallel by default, or you can specify a -j flag to override that number. Visual Studio gets a bit tricky though since you can paralleling at both the target and file level so if not careful, you can end up with N^2 build processes. What generator are you using? -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
On 9/5/2014 11:49 AM, Chuck Atkins wrote: Unfortunately on Windows and VS builds there's not really a simple answer. My blog might be of help: http://www.kitware.com/blog/home/post/434 -Bill -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] Ctest building with Multiple Processors
Does ctest -j jobs, or ctest --parallel jobs behave differently?
http://www.cmake.org/cmake/help/v3.0/manual/ctest.1.html
On Wed, Sep 3, 2014 at 2:31 PM, Michael Jackson
mike.jack...@bluequartz.net wrote:
I am exploring CTest/CDash with out project and I was trying to figure out
how to have CTest use all my cores for builds. So far no luck with some
snippets from Google. So far I have the following in a file called
CTestConfig.cmake at the top level of my project.
set(CTEST_PROJECT_NAME DREAM3D)
set(CTEST_NIGHTLY_START_TIME 01:00:00 UTC)
set(CTEST_DROP_METHOD http)
set(CTEST_DROP_SITE my.cdash.org)
set(CTEST_DROP_LOCATION /submit.php?project=DREAM3D)
set(CTEST_DROP_SITE_CDASH TRUE)
# Use multiple CPU cores to build
include(ProcessorCount)
ProcessorCount(N)
if(NOT N EQUAL 0)
if(NOT WIN32)
set(CTEST_BUILD_FLAGS -j${N})
endif(NOT WIN32)
set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
endif()
And then I execute CTest like this from my terminal
ctest -D Experimental
but monitoring my processors says that only a single processor is being used.
Thanks for any help
---
Mike Jackson www.bluequartz.net
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
[CMake] Ctest building with Multiple Processors
I am exploring CTest/CDash with out project and I was trying to figure out how
to have CTest use all my cores for builds. So far no luck with some snippets
from Google. So far I have the following in a file called CTestConfig.cmake at
the top level of my project.
set(CTEST_PROJECT_NAME DREAM3D)
set(CTEST_NIGHTLY_START_TIME 01:00:00 UTC)
set(CTEST_DROP_METHOD http)
set(CTEST_DROP_SITE my.cdash.org)
set(CTEST_DROP_LOCATION /submit.php?project=DREAM3D)
set(CTEST_DROP_SITE_CDASH TRUE)
# Use multiple CPU cores to build
include(ProcessorCount)
ProcessorCount(N)
if(NOT N EQUAL 0)
if(NOT WIN32)
set(CTEST_BUILD_FLAGS -j${N})
endif(NOT WIN32)
set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
endif()
And then I execute CTest like this from my terminal
ctest -D Experimental
but monitoring my processors says that only a single processor is being used.
Thanks for any help
---
Mike Jackson www.bluequartz.net
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
