2018-01-12 11:14 GMT+01:00 Eric Noulard <[email protected]>: > > > 2018-01-12 10:58 GMT+01:00 Alan W. Irwin <[email protected]>: > >> I am looking into the practicality of using clusters containing ~5 >> cheap ARM 8-core computers (such as the Banana Pi M3) to rapidly build >> and test software (since even with ccache I am currently spending a >> lot of my time waiting for tests to complete as I develop my >> software). Such clusters would give you ~40 cores which is a lot of >> computer power for ~$500 or so. But, of course, the issue with >> clusters is how to use them efficiently? >> >> For one 8-core M3 you would expect >> >> make -j<NNN> target >> >> to scale linearly with NNN up to the saturation limit of roughly NNN=8 >> > A side note on that. Sometimes you want to control the 'load' more than the "parallelism". With make (or ninja) you can ask for **load** limitation using:
make -l <N> and ninja -l <N> i.e. if you have 8 core and you want to keep 1 of them "calm" then make -l 7 is interesting. > > AFAIU what you want is "distributed ctest" not really distributed build > right? > For distributed build you may have a look at distcc (http://distcc.org/) > which seems already used > by CMake-enabled project like ccache is > (https://stackoverflow.com/questions/32752446/using- > compiler-prefix-commands-with-cmake-distcc-ccache/34317588) > > Now I'm not quite sure to understand why you speak of MPI-awareness ? > Are your test including MPI which would mean that some executable are > already eating up more than one core? > Or do you have a big collection of mono-processor tests you want to run on > the cluster? > Concerning high-level distributed execution on a cluster I would have a look at something like OpenPBS: http://www.rguha.net/writing/pub/opbs.html (now apparently http://pbspro.org/). Once you have your PBS cluster setup I would say that you would need to write a small test driver that would launch the tests on the cluster instead of launching them locally. -- Eric
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
