I have been struggling with creating a clean environment in which to build with EasyBuild and test easyconfigs.
Our build host is used to build other software and has a large number of "-dev" packages installed. I have found this to interfere with EasyBuild builds and it is currently blocking my easyconfig work. My initial attempts in setting up an environment have been successful, and I have a list of things I have found that I did not expect. When I am done, I hope to address those issues. My current question involves builddependencies and pkg-config. First, pkg-config. In CentOS it appears that pkg-config is installed in the base server install. This is not the case for Ubuntu/Debian and Opensuse. Without an explicit dependency on pkg-config, easyconfigs will end up using the system's pkg-config, which I think may be a vector for pollution from system/OS packages (I observed this with HarfBuzz - pkg-config found a system version first, before the EasyBuilt version). Should pkg-config be included at the toolchain level? Otherwise, it should probably be called out in osdependencies for most easyconfigs (configuremake only?). I have a suspicion that adding pkg-config to the toolchain, and thus replacing the system pkg-config, may cause a cascade of dependencies. Maybe that is why this has not been done? BTW, M4 seems to be similar - it should probably be included in the toolchain. Second, my immediate issue stems from this. I am trying to build HarfBuzz with cairo support. Clearly, this depends on cairo, which has builddependencies to renderproto, xproto, and xextproto. Without these modules loaded, pkg-config (my OS verison - I decided to go ahead and use it for now) returns an error when the configure script for HarfBuzz runs `pkg-config --cflags "cairo >= 1.8.0"`. This causes the wrong include dir to be specified so the HarfBuzz compilation fails (cairo installs cairo.h and other include files in .../include/cairo/ not .../include/ - non standard, thus the need for a working pkg-config). It looks to me like the only builddependencies that are loaded are those specified in the easyconfig being built, and dependent packages' builddependencies are not loaded. This makes sense. I can see two solution to this issue: - add those *proto packages as builddependencies for HarfBuzz - load builddependencies for the whole dependent package chain The first solution seems wrong to me and does not fix the problem for anyone else dependent on cairo to build. I suppose those builddependencies could be converted to dependencies in cairo, but they are not really run-time dependencies, they are true build dependencies, but they will be needed to build things that include cairo. I suspect this happens in other places with other packages as well, which is why I think the systemic solution of loading _all_ builddependencies during package build. Incidentally, I am using LXD currently to generate a clean Ubuntu containers, into which I am bootstrapping EasyBuild with minimal packages. I hope to have the whole thing automated with Jenkins in the next week or two. I have tested CentOS and Opensuse containers as well, so I plan to incorporate those for full testing. I also intend to have our "production" easybuilds run in these containers, with our distribution file system mounted. As always, thank you for EasyBuild! Ben McGough System Administrator Center IT/Scientific Computing O 206.667.7818 [email protected] 1100 Fairview Ave. N. P.O. Box 19024 Seattle, WA 98109 Fred Hutch / Cures Start Here fredhutch.org
