Hi Fotis,
Fotis Georgatos wrote:
Hi Martin,
On Jul 12, 2016, at 7:08 PM, Martin MOKREJŠ <[email protected]> wrote:
Finally, I discovered Gentoo::RAP which has a bit cryptic documentation and the
table at the top of thehttps://wiki.gentoo.org/wiki/Prefix/libc page drives me
always crazy (I have more questions than which the table tries to
document&explain). Gentoo::RAP uses Gentoo's MAIN portage repository (so less
confusion for me but issues fixed in Gentoo::Prefix repository may still be broken
in the MAIN tree ... ;)). You cannot get everything at once. What is very important
for me, on Gentoo::RAP I got glibc-2.23 installed and more or less all apps
including lots of java stuff can be run there. So, it is an amazing system how to
get all the bells and whistles of Gentoo Linux on almost any platform.
wow, I didn’t know that this works:
* Prefix/libc essentially delivers one more isolation layer, at the level of
glibc. [1]
The distribution/vector combinations here are quite a rich list:
https://wiki.gentoo.org/wiki/Prefix/libc
This could have also helped to build Clang/3.4 on RHEL6.x w/out a patch, since
glibc was the bottleneck!
I have sys-devel/clang-3.8.0-r100:0/3.8::gentoo installed on the Gentoo::RAP
"system" (having the glibc-2.23)
but also sys-devel/clang-3.7.1-r100::gentoo_prefix I have just compiled against
the old glibc-2.12 on the
Gentoo:Prefix approach. Maybe you want to backport some patches to EasyBuild if
you have have issue to get
clang compiled on old libc systems. So, it is doable. ;)
In any case, EasyBuild is a variation on build systems, with a focus on HPC
aims (ie. performance).
As an appetiser, have a check about how your FFTW library is provided via
alternative build systems;
it may seem a triviality, but that is a prereq for, say, Molecular Dynamics
codes on really large machines.
(namd, gromacs and the like)
Well, I think I am not up to answer all your good questions, but here is what
the fftw package in Gentoo is able to handle. USE flags with a minus are those
I have disabled on my laptop, but I could ask the system to enable any of them
(almost any).
# emerge -pv fftw
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] sci-libs/fftw-3.3.4:3.0/3::gentoo USE="fortran mpi openmp threads (-altivec) -doc
(-neon) -quad -static-libs {-test} (-zbus)" ABI_X86="32 (64) (-x32)" CPU_FLAGS_X86="avx
sse sse2 -fma3 -fma4" 0 KiB
#
The USE flags are set globally in a make.conf file, but can be overridden per
package in package.use config
file. So, I can enable globally mpi support in all apps but disable mpi support
in a few at my will.
I can see why Prefix can get someone enthusiastic, yet if you had to pay the
bill across 1000s of nodes,
you’d rather spend far more time in ensuring optimal maintainable builds.
Furthermore, how would you do
rebuild a tree of tens of HPC applications, against a complex new toolchain
(fi. cgmvolf) w/out a tool
I did not play with this myself but the emerge tool can create tarballs/rpms of
the $DESTDIR with installed
files for each package, before that is move to the / of the filesystem (or to
the $EPREFIX which is the
relocated path on Gentoo:Prefix installations. So, I think one can have easily
one compile host and deploy
the compiled packages on a cluster. Moreover, one can use cccache and similar
tools, also define hosts
which can serve precompiled packages (merely as I already noted above), so I
think one can manage a cluster
even by deploying individual packages on each node (although deploying a whole
image is a more common way
IMHO).
which can master such concepts? (esp. consider the difference CC vs MPICC). In
EB, that is an one-liner.
Gentoo simply does not know things like MPI stack built with CUDA support over
a given GCC and such.
For me, the following command enables MPI support in all packages I have:
USE=mpi emerge world
The emerge program respects the usual CC and CXX and F77 and F90 env variables
to learn the compiler to be used,
so I can set CC=mpicc but IMHO it also works with CC=gcc because the emerge
application passes to configure
--enable-mpi anyways thanks to the USE=mpi anyway.
To conclude, there is room for common approaches and more sharing of efforts
across different build systems,
however we should not too quickly equate them and, always check the features of
each one with each other.
Absolutely, but Gentoo gives you fully tuned binaries to your system if you
care to set your CFLAGS/CXXFLAGS
properly, so the Gentoo:RAP approach gave me fully tuned binaries, resolved the
dependencies, and abstracted
me from the RHEL6 stuff. Much faster then writing up hundreds of easybuild
package definitions.
Now I will just recompile all apps with icc and if any succeeds the gcc-made
binaries will be re-written on
my installation.
CC=icc CXX=icc USE="fftw fortran95 int64 mpi" emerge world
# Note: the 'emerge world' is the best approach, I need to learn some other
arguments how to really get all currently installed apps, but the command would
be quite similar
I will ask (CC:) the more advanced Gentoo devs if they would like to chim in
into this discussion a promote
a bit the Gentoo features and nice building system, probably more advanced then
LinuxFromScratch, etc. ;-)
Martin
