Joachim,
> First of all, a big thanks for looking into this. I have been banging
> at FZJs doors multiple times for you to support it in EB.
So far, there have only been minor version bumps (e.g., Score-P 3.0
vs. Score-P 3.1) which have been left as an exercise (you know
`--try-software-version`, don't you? ;-)). But now there are a few
more significant changes in terms of dependencies.
In any case, depending on which features you want/need (e.g.,
sampling, CUDA, OpenCL, etc.), you have to modify the upstream
easyconfig anyway.
> I am not sure I fully get the issues.
I'd like to keep things as compatible as possible with what
is already there. However, currently Qt5 and (some of) its
dependencies are only available on the foss/intel level --
which isn't really necessary. Thus, I'm trying to move it
down to GCCcore, based on the existing foss easyconfigs. In
principle, this isn't a big problem, except for the Mako build
dependency of Mesa which depends (at least) on a bare Python
and setuptools, both on the GCCcore level.
> If I see it correctly, set-up-tools doesn’t require MPI4PY or a BLAS
> lib.
Yep.
> So it could in principle be in a Python at GCC core level.
Yep.
> So solutions might be:
>
> * create a set-up-tool config to build a set-up-tool module to go with
> the bare Python. We have (had?) configs for matplotlib and mpi4py
> in that way for regular python installs.
> * use an Anaconda package for you python needs. They are at core level.
>
>
> The downside of these approaches might be, if a users wants to profile a
> package requiring Python in it’s own (in addition to C and Fortran) the
> Anaconda or bare Pyhon you load with Scalasca might stand in the way.
Note that Python is a *dep* of Mako which is a *build dep* of Mesa.
That is, neither Score-P nor Scalasca nor Cube require Python directly.
But now that you say it, the OTF2 package actually comes with a Python
binding, i.e., if someone wants to use it, OTF2 needs Python as a
dependency... I haven't taken that into account yet.
> You get additional fun for the Python 2 vs Python 3 issues here (some
> people deserve getting shot for this mess).
It's not only Python2 vs. Python3. That there is no proper versioning
scheme for Python packages is broken by design, too.
> When I build Scalasca in the past, I had to build it against a specific
> compiler and MPI lib, which means foss/intel level. Has that been
> relaxed?
Nope, this requirement is still there. But the Cube component (which
I was talking about) does not depend on MPI and doesn't benefit much
from being built with Intel, so it can easily be moved to the GCCcore
level and shared between multiple compiler/MPI-specific installations
of Score-P/Scalasca.
> Is it now (almost?) like other tools (e.g. ARM forge which
> installs at Core level and works with anything but OpenMPI 2.1) that you
> would build Scalasca at core level and it then figures the compiler/MPI
> by itself?
No, no, and no ;-)
> If you still need to build Scalasca against a compiler/MPI
> combination, I think taking the Python at the level would be preferable.
When you talk about Scalasca, you are referring to at least seven
different components now:
Scalasca
Score-P
OPARI2
OTF2
CubeWriter
CubeLib
CubeGUI
Plus all the (transitive) dependencies such as compiler, MPI, Qt, X11,
Mesa, LLVM, SIONlib, libunwind, PAPI, etc. A dry-run for Scalasca
shows me 71 packages in total... (OK, this also includes stuff needed
for bootstrapping the compiler, etc.)
Markus
>> On 27 Apr 2018, at 09:51, Markus Geimer <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> All,
>>
>> I'm currently preparing updated easyconfigs for the upcoming
>> Score-P/Scalasca releases (incl. dependencies). Since many
>> people are using a hierarchical MNS now, I started with a
>> "minimal toolchain" setup. However, I ran into the following
>> issue where I would like to get your feedback.
>>
>> Basically, I have the following dependency chain:
>>
>> Cube (GCCcore)
>> dep> Qt5 (GCCcore)
>> dep> libGLU (GCCcore)
>> dep> Mesa (GCCcore)
>> builddep> Mako (GCCcore)
>> dep> Python (foss/intel)
>>
>> The Problem is that Mako seems to require setuptools, thus
>> I can't use Python-bare instead of the full Python package.
>> But that would require to lift the whole thing up the
>> hierarchy onto the foss/intel level -- which is obviously
>> crazy. So what is the preferred way to deal with such a
>> situation?
>>
>> I know about [1], but it isn't there yet and the discussion
>> in the PR is quite controversial.
>>
>> Cheers,
>> Markus
>>
>>
>> [1] https://github.com/easybuilders/easybuild-easyconfigs/pull/5072
--
Dr. Markus Geimer
Juelich Supercomputing Centre
Institute for Advanced Simulation
Forschungszentrum Juelich GmbH
52425 Juelich, Germany
Phone: +49-2461-61-1773
Fax: +49-2461-61-6656
E-Mail: [email protected]
WWW: http://www.fz-juelich.de/jsc
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
