Hi Caspar On Wed, 11 Jul 2018 at 13:27, Caspar van Leeuwen < [email protected]> wrote:
> Hi Alan, > > Thanks, I wasn't aware of the buildenv EasyBlock either, but it seems this > may solve some problems I encountered in the past with compilers or flags > that were not always correctly picked up by CMake. In the past, I sometimes > passed these manually in the EasyConfigs, but that's a bit ugly. So, do I > understand correctly that you load the buildenv EasyConfigs whenever you > build new software through EasyBuild? Do you then add it as a build > dependency to your existing EasyConfigs (which involves customizing > existing EasyConfigs), or do you simply load the module before you run your > 'eb some_easyconfig.eb'? > No, I only created this as a way to help give our users a working build environment that reflects the EB toolchains (at least for an Autotools scenario), it's not related to our EB setup at all. It's quite limited in that sense, you get a lot more useful information from `eb -x <easyconfg>` but of course that changes from config to config > > And another question: if I need to load these modules to get CMake to do > what I intend, what if a user uses e.g. the foss toolchain to build some of > his own software that relies on CMake? Do you instruct your users to load > the buildenv module in those cases? > We tell them it's there but it's hard to know if any of them use it at all (we don't keep track of module loads). Specifically with CMake, the easyblock sets a few additional variables <https://github.com/easybuilders/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/cmakemake.py#L67> so even buildenv won't give you everything you probably need. > > Finally, it makes me wonder if this (i.e. loading a compiler toolchain + > seperate buildenv module) is the right design. To me, it seems very > counterintuitive that loading the toolchain itself is not sufficient to > have e.g. CMake pick up the correct compilers. Of course, toolchains are > mostly loaded as dependencies for running production codes, in which case > setting these variables would be unnecessary pollution of the environment. > But other than that: is there any reason why the toolchains modules > themselves don't set these build environments by default? (Kenneth, was > this a conscious design decision, or did it just 'grow' like this?) > See above and the `-x` option, all these variables are indeed set, all buildenv does is make it so that the user can set them in their own environment. > > Curious to hear what you (and others) think. Also, apart from Juelich, are > other institutes running into the same problems that we are, and is are the > buildenv blocks/configs also the way they solve it? For uniformity and > reusability of EasyBlocks/Configs I think it would be good if 'we, the > community' adopt a uniform solution for this issue. > > Cheers, > > Caspar > > > ------------------------------ > *From: *"Alan O'Cais" <[email protected]> > *To: *"easybuild" <[email protected]> > *Sent: *Tuesday, 10 July, 2018 16:02:44 > *Subject: *Re: [easybuild] Loading toolchain module and CC/CXX env vars > > Hi Paul, > > There's a simple easyblock that implments this (for the toolchain): > > https://github.com/easybuilders/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/buildenv.py > You can see example uses in > > https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/b/buildenv > > Alan > > On Tue, 10 Jul 2018 at 15:56, Paul Melis <[email protected]> wrote: > >> Hi, >> >> When developing software (in contrast to simply building and installing >> it with EB) it makes sense to leverage the existing toolchains for >> setting up a build environment. E.g. I load foss/2017b and other needed >> EB-built dependencies while developing and use CMake to build by hand. >> When the package gets to a stable point creating an EB config is then >> (relatively) easy and predictable, based on the loaded modules used. >> Rebuilding by hand is also a lot quicker than doing a full EB build and >> install each time. >> >> But loading a toolchain like foss/2017b doesn't completely set up the >> environment in a way that makes it directly usable by CMake and friends. >> Particularly, CC and CXX are not set so it's up to the build tool to >> figure out which compiler to use. >> >> CMake does this somewhat silly in that it searches for (in order) cc, >> gcc, cl, bcc, ... when CC is not set (and similarly c++, g++, aCC, cl, >> ... when CXX is not set). But this is a breadth-first search of "cc" in >> a set of directories. So in our case it actually finds /usr/bin/cc (and >> /usr/bin/c++ next to it) which are old, GCC 4.8.5, and it doesn't find >> the foss/2017b binaries. This appears to be because GCC installations >> don't contain the "cc" command (even though they have a "c++" one), >> which is what CMake searches for first. >> >> Setting CC and CXX to the foss/2017b compilers will fix this, which kind >> of makes me wonder why these aren't set by default when loading >> GCC(core) and similar compiler modules? Has this ever been discussed? >> >> Regards, >> Paul >> >> -- >> >> Paul Melis >> | Visualization group leader & developer | SURFsara | >> | Science Park 140 | 1098 XG Amsterdam | >> | T 020 800 1312 | [email protected] | www.surfsara.nl | >> > > > -- > Dr. Alan O'Cais > E-CAM Software Manager > Juelich Supercomputing Centre > Forschungszentrum Juelich GmbH > 52425 Juelich, Germany > > Phone: +49 2461 61 5213 > Fax: +49 2461 61 6656 > E-mail: [email protected] > WWW: http://www.fz-juelich.de/ias/jsc/EN > > > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > Forschungszentrum Juelich GmbH > 52425 Juelich > Sitz der Gesellschaft: Juelich > Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 > Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher > Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), > Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, > Prof. Dr. Sebastian M. Schmidt > > ------------------------------------------------------------------------------------------------ > > ------------------------------------------------------------------------------------------------ > > > -- Dr. Alan O'Cais E-CAM Software Manager Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: [email protected] WWW: http://www.fz-juelich.de/ias/jsc/EN
