Ever configuartion setting can be done in multiple ways, see https://easybuilders.github.io/easybuild-tutorial/2022-isc22/configuration/ for a good overview.
The easiest way to do arch support is to have use .../easybuild/<arch> as the --prefix for each node type. The only stuff you should really worry about have multiple copies of are the sources, so you can set --sourcepath to .../easybuild/sources to avoid that duplication. For each node then you configure the profile so that it detects the right path(s) for `module use /path/to/<arch>/modules/all` On Fri, 2 Dec 2022 at 11:01, Loris Bennett <[email protected]> wrote: > > I was basing my approach on the first answer given here: > > > https://www.reddit.com/r/HPC/comments/y1gdch/how_to_manage_easybuild_on_hpc_with_multiple_cpu/ > > But I see now that the problem being solved there is slightly different > to mine, as two existing clusters were being merged, so the software for > the two different architectures had already been built. > > "Alan O'Cais" <[email protected]> writes: > > > Sorry, wrong name: --installpath-modules > > OK, thanks. Is there no corresponding environment variable? > > > If you are not using this explicitly in your config, then you are > relying on EASYBUILD_PREFIX to set the default, see > > > https://docs.easybuild.io/en/latest/Configuration.html#mandatory-configuration-settings > for details. By setting _just_ EASYBUILD_INSTALLPATH, you are > > only changing where software gets installed, everything else is > remaining the same (and EasyBuild relies only on the module files to know > if something > > is installed or not). > > Ah, OK. Is there a recommend directory layout? I can think of > > easybuild/ > |-- build > |-- ebfiles_repo > |-- moduleData > |-- modules > | `-- arch > | |-- intel > | `-- amd > |-- software > | `-- arch > | |-- intel > | `-- amd > `-- sources > > or > > easybuild/ > |-- build > |-- ebfiles_repo > |-- moduleData > |-- arch > | |-- intel > | | |-- software > | | `-- modules > | `-- amd > | | |-- software > | | `-- modules > `-- sources > > Are there any factors which would make one preferable to the other? > > > On Fri, 2 Dec 2022 at 09:55, Loris Bennett <[email protected]> > wrote: > > > > Hi Alan, > > > > Is EASYBUILD_MODULEPATH documented anywhere? It is not set in my > > environment and my search on the web didn't turn up any results. > > > > Cheers, > > > > Loris > > > > "Alan O'Cais" <[email protected]> writes: > > > > > You are changing the EASYBUILD_INSTALLPATH, but not changing > EASYBUILD_MODULEPATH. This means you are overwriting the existing module > tree. You > > don't > > > want this, instead you build a new module tree somewhere else. Since > the tree will be empty, EasyBuild will also install all the deps. > > > > > > On Fri, 2 Dec 2022 at 09:27, Loris Bennett < > [email protected]> wrote: > > > > > > Hi Ole, > > > > > > Ole Holm Nielsen <[email protected]> writes: > > > > > > > Hi Loris, > > > > > > > > On 12/2/22 08:27, Loris Bennett wrote: > > > >> How do I force a total rebuild of, say, a foss toolchain for a > different > > > >> CPU architecture? > > > >> Up to now I had a homogeneous cluster with Intel Xeon CPUs, now we > > > >> have > > > >> acquired some nodes with AMD Epyc CPUs for which I need to build > > > >> software. > > > >> I have modified EASYBUILD_INSTALLPATH to point to directory for > the > > > >> new > > > >> architecture and prepended a corresponding directory for the > modules to > > > >> MODULEPATH. However, running EasyBuild with the option --force > just > > > >> rebuilds the package specified, not the dependencies. > > > >> What is the correct way to go about this? > > > > > > > > Probably there are multiple ways to set up modules for multiple > > > > architectures :-) My choice was to create completely different > module > > > > trees for each type of hardware (we have 4 generations of Intel > Xeon). > > > > My notes are in this Wiki page: > > > > > https://wiki.fysik.dtu.dk/Niflheim_system/EasyBuild_modules/#automounting-the-cpu-architecture-dependent-modules-directory > > > > > > > > IHTH /Ole > > > > > > Thanks for the link. That looks like an good way of dealing with > > > switching between different architectures. > > > > > > However, I am not yet quite that far. My question was more to do > with > > > how to actually rebuild the software in a second branch. If I change > > > EASYBUILD_INSTALLPATH and MODULEPATH as described abouve and then run > > > EasyBuild with force I get, say, the following: > > > > > > [build@a001 ~]$ eb Bison-3.8.2.eb --robot --force --dry-run > > > == Temporary log file in case of crash > /tmp/eb-akrn565i/easybuild-0vsmv95q.log > > > == found valid index for > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs, > so using it... > > > == found valid index for > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs, > so using it... > > > Dry run: printing build status of easyconfigs and dependencies > > > * [x] > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs/m/M4/M4-1.4.19.eb > (module: M4/1.4.19) > > > * [F] > /trinity/shared/easybuild/software/EasyBuild/4.6.2/easybuild/easyconfigs/b/Bison/Bison-3.8.2.eb > (module: Bison/3.8.2) > > > > > > How to I get EasyBuild to build M4 automatically as well? > > > > > > Cheers, > > > > > > Loris > > > > > > -- > > > Dr. Loris Bennett (Herr/Mr) > > > ZEDAT, Freie Universität Berlin > > > > > -- > > Dr. Loris Bennett (Herr/Mr) > > ZEDAT, Freie Universität Berlin > > > -- > Dr. Loris Bennett (Herr/Mr) > ZEDAT, Freie Universität Berlin >
