[easybuild] Problems installing Amber 12 with Easybuild - install dir gets devoured

[Ben Roberts]

Hi list,

I’m trying to install Amber 12 with Easybuild. My current version of the 
framework is the latest checkout of the “develop” branch from git: commit 
3c4976c5f0ea5dad572f6ed3c014380b299e2d7a.

I’m having trouble because the Amber installation process, being inherently 
non-standard, is interacting strangely with the Easybuild process.

An overview of the Amber build process is as follows:

1. Download and extract the Amber tarball (extracts to “amber12”)
2. Download and extract the AmberTools tarball (also extracts into “amber12”)
3. Change into the “amber12” directory
4. Set the AMBERHOME environment variable to the amber12 directory
5. Run “configure” with options for a serial build
6. Run “make install” — note that this combines the functionality of “make” and 
“make install” into a single step, and builds AmberTools and Amber in situ
7. Run “make test”
8. Run “make clean” to delete serial-specific object files, etc.
9. Run “configure” again with options for a parallel build
10. Run “make install”
11. Run “make test.parallel”
12. Run “make clean”
13. Run “configure” again with options for a CUDA build
14. Run “make install”
15. Run “make test.cuda”
16. Optionally, run “make clean”

I’ve attached my easyconfig and easyblock for your information.

The particular issue I’m having is the interplay of three features of this 
build. One is that the build and install directories are the same, so that I’ve 
used the “buildininstalldir” option in my easyconfig. The other is that it’s 
necessary to keep at least some parts of the serial installation around for 
parallel and CUDA builds, so that deleting the install directory between builds 
is not an option. The third is the use of multiple configure calls, to reflect 
the staged nature of the build.

You can probably already see where I’m going with this, which is the method 
“make_builddir” in easybuild-framework/easybuild/framework/easyblock.py. This 
method, if “buildininstalldir” is set, will force a “cleanupoldbuild” and veto 
a “keeppreviousinstall” (line 641ff.), regardless of any options to the 
contrary that I set in the easyconfig.

How can I get around this? Is there an option that can be implemented to deal 
with staged builds of this sort, the prospect of overloading the make_builddir 
method in my easyblock, or some other solution that I haven’t floated yet?

Thanks,
Ben

Amber-12.21-AmberTools-13.26-ictce-4.1.13.eb
Description: Binary data

from easybuild.framework.easyconfig import CUSTOM, MANDATORY, BUILD
from easybuild.tools.filetools import run_cmd
from easybuild.easyblocks.generic.configuremake import ConfigureMake
import os

class Amber(ConfigureMake):
    """Easyblock for building and installing Amber"""


    def __init__(self, *args, **kwargs):
        super(Amber, self).__init__(*args, **kwargs)
        self.already_extracted = False

    @staticmethod
    def extra_options(extra_vars=None):
        """Extra easyconfig parameters specific to ConfigureMake."""
        supersuperclass = super(ConfigureMake, ConfigureMake)
        extra_vars = dict(supersuperclass.extra_options(extra_vars))
        extra_vars.update({
            # 'Amber': [True, "Build Amber in addition to AmberTools", CUSTOM],
            'patchlevels': ["latest", "(AmberTools, Amber) updates to be applied", CUSTOM],
        })
        # Don't include the ConfigureMake extra options since this configure_step doesn't handle them
        return supersuperclass.extra_options(extra_vars)

    def extract_step(self):
        """Only extract from the tarball if this has not already been done."""
        if (self.already_extracted == True):
            cmd = '%(prebuildopts)s make clean' % {
                'prebuildopts': self.cfg['prebuildopts']
            }
            run_cmd(cmd, log_all=True, simple=False)
        else:
            super(Amber, self).extract_step()
            self.already_extracted = True

    def configure_step(self):
        #cmd = '%(preconfigopts)s AMBERHOME="%(installdir)s" ./configure --no-updates %(configopts)s' % {
        cmd = '%(preconfigopts)s %(prebuildopts)s ./configure --no-updates %(configopts)s' % {
            'prebuildopts': self.cfg['prebuildopts'],
            'preconfigopts': self.cfg['preconfigopts'],
            'installdir': self.installdir,
            'configopts': self.cfg['configopts']
        }
        run_cmd(cmd, log_all=True, simple=False)

    def patch_step(self, **kw):
        if self.cfg['patchlevels'] == "latest":
            cmd = "%(prebuildopts)s ./update_amber --update" % {
                'prebuildopts': self.cfg['prebuildopts']
            }
            run_cmd(cmd, log_all=True)
        else:
            for (tree, patch_level) in zip(['AmberTools', 'Amber'], self.cfg['patchlevels']):
                if patch_level == 0: continue
                cmd = "%s ./update_amber --update-to %s/%s" % (self.cfg['prebuildopts'], tree, patch_level) 
                run_cmd(cmd, log_all=True)
        return super(Amber, self).patch_step(**kw)

    def test_step(self):
        cmd = "%s make test" % (self.cfg['prebuildopts'])
        run_cmd(cmd, log_all=True)

    def make_module_extra(self):
        """Add module entries specific to Amber/AmberTools"""
        txt = super(Amber, self).make_module_extra()
        #cmd = "AMBERHOME=`pwd` ./update_amber -v"
        #(out, _) = run_cmd(cmd, log_all=True, simple=False)
        #txt += self.moduleGenerator.set_environment('AMBER_VERSION', out)
        return txt


    def install_step(self):
        """In Amber, installation is conflated with building,
        so that 'make install' is done during the build step."""
        pass