You can solve this by using an alias for the `eb` command, see https://gist.github.com/ocaisa/a24059bbb08b040d7710cd9184c7e95e#file-userinstallations_easybuild-lua-L49 for an example that is pretty similar for what you are trying to achieve.
Alan On Fri, 2 Jul 2021 at 20:06, Kenneth Hoste <[email protected]<mailto:[email protected]>> wrote: On 02/07/2021 19:38, Davide Vanzo wrote: > Hello EasyBuilders! > > I am trying to configure EasyBuild to allow users to install software > locally while using the existing cluster-wide stack (distributed via > CVMFS) for dependencies. > > Here is the relevant EB configuration on the clients: > > allow-loaded-modules (F) = EasyBuild > > detect-loaded-modules (F) = purge > > envvars-user-modules (F) = USER_STACK_ROOT, HOME > > installpath (F) = /home/hpcadmin/EasyBuild > > minimal-toolchains (F) = True > > module-depends-on (F) = True > > module-naming-scheme (F) = HierarchicalMNS > > prefix (F) = /home/hpcadmin/EasyBuild > > robot-paths (F) = > <cvmfs_stack_root>/EasyBuild/custom_easyconfigs, > <cvmfs_stack_root>/EasyBuild/software/EasyBuild/4.4.0/easybuild/easyconfigs > > rpath (F) = True > > subdir-user-modules (F) = EasyBuild/modules > > The cluster-wide stack root path is indicated above as: cvmfs_stack_root > > When a user wants to build and locally install a piece of software and > the toolchain is already available in the cluster-wide stack, the > expectation is that EB pulls the existing dependencies from the stack > and only builds/installs locally the missing dependencies and finally > the desired software. The generated local module will then appear in > Lmod thanks to the subdir-user-modules option. > > When EasyBuild tries to resolve the dependencies, it pulls the list of > installed modules from: > > * the paths in MODULEPATH > * the <installpath>/<subdir_modules> as specified in the EB configuration > > In my case this correspond to the following paths: > > * <cvmfs_stack_root>/EasyBuild/modules/all/Core > * /home/hpcadmin/EasyBuild/modules/all > > The problem with this is that EB only considers stack modules in the > Core directory and not on the rest of the hierarchical tree (Compiler, > MPI), causing the entire toolchain to be re-built locally. > > The solution would be to add <cvmfs_stack_root>/EasyBuild/modules/all to > MODULEPATH at build time. > > What am I missing? Nothing, that's the correct approach currently. EasyBuild relies on what we call the "full" module name (Core/GCC/9.3.0 for example, or Compiler/GCC/9.3.0/OpenMPI/4.0.2, where the "short" (user-facing) module names are GCC/9.3.0 and OpenMPI/4.0.2) to check whether modules are already installed in a hierarchy, and there's a good reason for it. It may be possible to avoid this, but this requires a bit of work to be done in framework, since we probably rely on this in a number of places... We discussed this recently in the EasyBuild conf call, and we agreed that a short-term easy solution could be to let EasyBuild automatically add <prefix> to $MODULEPATH when it notices <prefix>/Core is in there. See also https://github.com/easybuilders/easybuild-framework/issues/3703 regards, Kenneth > > Davide > -- Dr. Alan O'Cais E-CAM Software Manager Juelich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Juelich, Germany Phone: +49 2461 61 5213 Fax: +49 2461 61 6656 E-mail: [email protected]<mailto:[email protected]> WWW: http://www.fz-juelich.de/ias/jsc/EN ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------ Forschungszentrum Juelich GmbH 52425 Juelich Sitz der Gesellschaft: Juelich Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498 Vorsitzender des Aufsichtsrats: MinDir Volker Rieke Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender), Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt, Dr. Astrid Lambrecht, Prof. Dr. Frauke Melchior ------------------------------------------------------------------------------------------------ ------------------------------------------------------------------------------------------------
