Hi Alan, Thanks, I wasn't aware of the buildenv EasyBlock either, but it seems this may solve some problems I encountered in the past with compilers or flags that were not always correctly picked up by CMake. In the past, I sometimes passed these manually in the EasyConfigs, but that's a bit ugly. So, do I understand correctly that you load the buildenv EasyConfigs whenever you build new software through EasyBuild? Do you then add it as a build dependency to your existing EasyConfigs (which involves customizing existing EasyConfigs), or do you simply load the module before you run your 'eb some_easyconfig.eb'?
And another question: if I need to load these modules to get CMake to do what I intend, what if a user uses e.g. the foss toolchain to build some of his own software that relies on CMake? Do you instruct your users to load the buildenv module in those cases? Finally, it makes me wonder if this (i.e. loading a compiler toolchain + seperate buildenv module) is the right design. To me, it seems very counterintuitive that loading the toolchain itself is not sufficient to have e.g. CMake pick up the correct compilers. Of course, toolchains are mostly loaded as dependencies for running production codes, in which case setting these variables would be unnecessary pollution of the environment. But other than that: is there any reason why the toolchains modules themselves don't set these build environments by default? (Kenneth, was this a conscious design decision, or did it just 'grow' like this?) Curious to hear what you (and others) think. Also, apart from Juelich, are other institutes running into the same problems that we are, and is are the buildenv blocks/configs also the way they solve it? For uniformity and reusability of EasyBlocks/Configs I think it would be good if 'we, the community' adopt a uniform solution for this issue. Cheers, Caspar From: "Alan O'Cais" <[email protected]> To: "easybuild" <[email protected]> Sent: Tuesday, 10 July, 2018 16:02:44 Subject: Re: [easybuild] Loading toolchain module and CC/CXX env vars Hi Paul, There's a simple easyblock that implments this (for the toolchain): [ https://github.com/easybuilders/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/buildenv.py | https://github.com/easybuilders/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/buildenv.py ] You can see example uses in [ https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/b/buildenv | https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/b/buildenv ] Alan On Tue, 10 Jul 2018 at 15:56, Paul Melis < [ mailto:[email protected] | [email protected] ] > wrote: Hi, When developing software (in contrast to simply building and installing it with EB) it makes sense to leverage the existing toolchains for setting up a build environment. E.g. I load foss/2017b and other needed EB-built dependencies while developing and use CMake to build by hand. When the package gets to a stable point creating an EB config is then (relatively) easy and predictable, based on the loaded modules used. Rebuilding by hand is also a lot quicker than doing a full EB build and install each time. But loading a toolchain like foss/2017b doesn't completely set up the environment in a way that makes it directly usable by CMake and friends. Particularly, CC and CXX are not set so it's up to the build tool to figure out which compiler to use. CMake does this somewhat silly in that it searches for (in order) cc, gcc, cl, bcc, ... when CC is not set (and similarly c++, g++, aCC, cl, ... when CXX is not set). But this is a breadth-first search of "cc" in a set of directories. So in our case it actually finds /usr/bin/cc (and /usr/bin/c++ next to it) which are old, GCC 4.8.5, and it doesn't find the foss/2017b binaries. This appears to be because GCC installations don't contain the "cc" command (even though they have a "c++" one), which is what CMake searches for first. Setting CC and CXX to the foss/2017b compilers will fix this, which kind of makes me wonder why these aren't set by default when loading GCC(core) and similar compiler modules? Has this ever been discussed? Regards, Paul -- Paul Melis | Visualization group leader & developer | SURFsara | | Science Park 140 | 1098 XG Amsterdam | | T 020 800 1312 | [ mailto:[email protected] | [email protected] ] | [ http://www.surfsara.nl/ | www.surfsara.nl ] | -- Dr. Alan O'Cais E-CAM Software Manager Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: [ mailto:[email protected] | [email protected] ] WWW: [ http://www.fz-juelich.de/ias/jsc/EN | http://www.fz-juelich.de/ias/jsc/EN ] ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Prof. Dr. Sebastian M. Schmidt ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
