I was basing my approach on the first answer given here:
https://www.reddit.com/r/HPC/comments/y1gdch/how_to_manage_easybuild_on_hpc_with_multiple_cpu/ But I see now that the problem being solved there is slightly different to mine, as two existing clusters were being merged, so the software for the two different architectures had already been built. "Alan O'Cais" <alan.oc...@cecam.org> writes: > Sorry, wrong name: --installpath-modules OK, thanks. Is there no corresponding environment variable? > If you are not using this explicitly in your config, then you are relying on > EASYBUILD_PREFIX to set the default, see > https://docs.easybuild.io/en/latest/Configuration.html#mandatory-configuration-settings > for details. By setting _just_ EASYBUILD_INSTALLPATH, you are > only changing where software gets installed, everything else is remaining the > same (and EasyBuild relies only on the module files to know if something > is installed or not). Ah, OK. Is there a recommend directory layout? I can think of easybuild/ |-- build |-- ebfiles_repo |-- moduleData |-- modules | `-- arch | |-- intel | `-- amd |-- software | `-- arch | |-- intel | `-- amd `-- sources or easybuild/ |-- build |-- ebfiles_repo |-- moduleData |-- arch | |-- intel | | |-- software | | `-- modules | `-- amd | | |-- software | | `-- modules `-- sources Are there any factors which would make one preferable to the other? > On Fri, 2 Dec 2022 at 09:55, Loris Bennett <loris.benn...@fu-berlin.de> wrote: > > Hi Alan, > > Is EASYBUILD_MODULEPATH documented anywhere? It is not set in my > environment and my search on the web didn't turn up any results. > > Cheers, > > Loris > > "Alan O'Cais" <alan.oc...@cecam.org> writes: > > > You are changing the EASYBUILD_INSTALLPATH, but not changing > EASYBUILD_MODULEPATH. This means you are overwriting the existing module > tree. You > don't > > want this, instead you build a new module tree somewhere else. Since the > tree will be empty, EasyBuild will also install all the deps. > > > > On Fri, 2 Dec 2022 at 09:27, Loris Bennett <loris.benn...@fu-berlin.de> > wrote: > > > > Hi Ole, > > > > Ole Holm Nielsen <ole.h.niel...@fysik.dtu.dk> writes: > > > > > Hi Loris, > > > > > > On 12/2/22 08:27, Loris Bennett wrote: > > >> How do I force a total rebuild of, say, a foss toolchain for a > different > > >> CPU architecture? > > >> Up to now I had a homogeneous cluster with Intel Xeon CPUs, now we > > >> have > > >> acquired some nodes with AMD Epyc CPUs for which I need to build > > >> software. > > >> I have modified EASYBUILD_INSTALLPATH to point to directory for the > > >> new > > >> architecture and prepended a corresponding directory for the modules to > > >> MODULEPATH. However, running EasyBuild with the option --force just > > >> rebuilds the package specified, not the dependencies. > > >> What is the correct way to go about this? > > > > > > Probably there are multiple ways to set up modules for multiple > > > architectures :-) My choice was to create completely different module > > > trees for each type of hardware (we have 4 generations of Intel Xeon). > > > My notes are in this Wiki page: > > > > https://wiki.fysik.dtu.dk/Niflheim_system/EasyBuild_modules/#automounting-the-cpu-architecture-dependent-modules-directory > > > > > > IHTH /Ole > > > > Thanks for the link. That looks like an good way of dealing with > > switching between different architectures. > > > > However, I am not yet quite that far. My question was more to do with > > how to actually rebuild the software in a second branch. If I change > > EASYBUILD_INSTALLPATH and MODULEPATH as described abouve and then run > > EasyBuild with force I get, say, the following: > > > > [build@a001 ~]$ eb Bison-3.8.2.eb --robot --force --dry-run > > == Temporary log file in case of crash > /tmp/eb-akrn565i/easybuild-0vsmv95q.log > > == found valid index for > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs, so > using it... > > == found valid index for > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs, so > using it... > > Dry run: printing build status of easyconfigs and dependencies > > * [x] > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs/m/M4/M4-1.4.19.eb > (module: M4/1.4.19) > > * [F] > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs/b/Bison/Bison-3.8.2.eb > (module: Bison/3.8.2) > > > > How to I get EasyBuild to build M4 automatically as well? > > > > Cheers, > > > > Loris > > > > -- > > Dr. Loris Bennett (Herr/Mr) > > ZEDAT, Freie Universität Berlin > > > -- > Dr. Loris Bennett (Herr/Mr) > ZEDAT, Freie Universität Berlin > -- Dr. Loris Bennett (Herr/Mr) ZEDAT, Freie Universität Berlin
