Hi,
I'm still struggling a bit with the installation of the new toolchain.
There are chains/software called *mpich* but there is also e.g.
mvapich2. It seems I couldn't decide if I would use the 2 or not, and
now I get this error:
ERROR: EasyBuild encountered an exception (at easybuild/main.py:833 in
build_and_install_software): autoBuild Failed (last 300 chars):
EasyBuild crashed with an error (at
easybuild/tools/toolchain/toolchain.py:328 in prepare): List of
toolchain dependency modules and toolchain definition do not match
(set(['MPICH', 'GCC', 'FFTW', 'ScaLAPACK', 'OpenBLAS']) vs set(['GCC',
'MPICH2', 'FFTW', 'ScaLAPACK', 'OpenBLAS']))
I got the MPICH vs. MPICH2 error before, thought I resolved it, but
clearly I haven't. Would you have an idea what I did wrong? I attached
some files that might be of interest.
A second question I have concerns github. Eb does not support github
sources, right? Is the solution then to download the source as a zip
file and make a link to the local zip?
Thank your for all the work on the custom module naming, and good luck
with the new version!
Vriendelijke groet,
Bart
On 20/09/13 18:38, Kenneth Hoste wrote:
Hi Bart,
On 20 Sep 2013, at 05:34, Bart Verleye wrote:
Dear All,
We are testing out easybuild and I've managed to install some software with it,
very easily indeed.
Good to hear, and welcome to the wonderful world of EasyBuild. ;-)
However, I'm trying now to build software which is not yet included in the
easybuild configs, e.g. a software with the toolchain gmpich2.
For that, I need the versions of this toolchain that are available.
But where can I find these?
The support for a gmpich2 toolchain is available in the framework, but we're
currently not shipping any easyconfigs for it, that's true.
To be clear: gmpich2 is intended to be a subtoolchain, it should only be used
(in principal) to construct other toolchains (just like gompi, gmvapich2, ...).
It should be used to construct a gmpolf toolchain, which also includes
OpenBLAS, LAPACK, FFTW, ...
This is quite involved already, especially because the support for a gmpolf
toolchain is not yet present in the framework.
The good news is that we have (good, up-to-date) documentation on compiler
toolchains, see https://github.com/hpcugent/easybuild/wiki/Compiler-toolchains .
If you want to go ahead and tackle this, please do (and make a pull request
when it's done to contribute it to the EasyBuild codebase).
Let us know if you have questions or if something is not clear, and help out.
We can set up a Skype session if that would help you more quickly.
E.g. now I get the error:
== temporary log file in case of crash /tmp/easybuild-rjNEcI.log
== resolving dependencies ...
ERROR: EasyBuild crashed with an error (at easybuild/main.py:545 in
resolve_dependencies): Dependencies not met. Cannot resolve [('gmpich2',
'1.1.6'), ('libpng', '1.5.13-gmpich2-1.1.6'), ('Python',
'2.7.3-gmpich2-1.1.6'), ('zlib', '1.2.7-gmpich2-1.1.6'), ('freetype',
'2.4.11-gmpich2-1.1.6')]
Which version of gmpich2 can I use? In this example, should I provide an eg
file for e.g. libpng-1.5.13-gmpich2-1.1.6 myself, or will the robot do that?
For now, you'll need to provide gmpich2-1.1.6 (or gmpolf-x.y.z) easyconfig
files for the toolchain, software and all the dependencies yourself (or let EB
do it for you, see below).
If you do that, please contribute them back so others can use them as well if
they want to.
We have command line options like --try-toolchain that should allow you to do
that easily, but that doesn't work recursively yet (i.e. it will not
automagically apply the --try-toolchain to missing dependencies too).
"eb Python-2.7.3-goolf-1.4.10.eb --try-toolchain=gmpich2-1.1.6" will look for
the goolf easyconfig file for Python, and rewrite it for the gmpich2-1.1.6 toolchain
before performing the build.
If you list multiple easyconfigs on the command line, the --try-toolchain will
apply to all of them:
eb Python-2.7.3-goolf-1.4.10.eb libpng-1.5.13-goolf-1.4.10.eb
zlib-1.2.7-goolf-1.4.10.eb freetype-2.4.11-goolf-1.4.10.eb
--try-toolchain=gmpich2-1.1.6
Again, using gmpich directly is probably not what you want, especially not to
be build Python since that toolchain doesn't include BLAS/LAPACK libraries,
so the numpy/scipy Python packages that will be build along with Python will be
very slow (and the build may even fail because a performance test fails).
I might be missing out on something completely...perhaps a link to the right
documentation could help heaps already.
It's basically just a set of missing easyconfig files. We only ship easyconfigs
for builds we have actually tested...
We have some documentation in place on the wiki
https://github.com/hpcugent/easybuild/wiki/, but parts of it are out-of-date
and need a significant amount of love (soon, hopefully, but I keep saying that).
regards,
Kenneth
PS: Any chance you and your colleague will be attending SC'13 in Denver?
--
Centre for e-Research
Level G, Room 409-G21
24 SYMONDS ST
Auckland 1010
New Zealand
+64 (0) 9 923 9740 ext 89740
name = "GCC"
version = '4.8.1'
homepage = 'http://gcc.gnu.org/'
description = """The GNU Compiler Collection includes front ends for C, C++,
Objective-C, Fortran, Java, and Ada,
as well as libraries for these languages (libstdc++, libgcj,...)."""
toolchain = {'name': 'dummy', 'version': 'dummy'}
source_urls = [
'http://ftpmirror.gnu.org/%(namelower)s/%(namelower)s-%(version)s', # GCC
auto-resolving HTTP mirror
'http://ftpmirror.gnu.org/gmp', # idem for GMP
'http://ftpmirror.gnu.org/mpfr', # idem for MPFR
'http://www.multiprecision.org/mpc/download', # MPC official
]
sources = [
SOURCELOWER_TAR_GZ,
'gmp-5.1.2.tar.bz2',
'mpfr-3.1.2.tar.gz',
'mpc-1.0.1.tar.gz',
]
languages = ['c', 'c++', 'fortran', 'lto']
# building GCC sometimes fails if make parallelism is too high, so let's limit
it
maxparallel = 4
moduleclass = 'compiler'
##
# Copyright 2013 Ghent University
#
# This file is triple-licensed under GPLv2 (see below), MIT, and
# BSD three-clause licenses.
#
# This file is part of EasyBuild,
# originally created by the HPC team of Ghent University (http://ugent.be/hpc/en),
# with support of Ghent University (http://ugent.be/hpc),
# the Flemish Supercomputer Centre (VSC) (https://vscentrum.be/nl/en),
# the Hercules foundation (http://www.herculesstichting.be/in_English)
# and the Department of Economy, Science and Innovation (EWI) (http://www.ewi-vlaanderen.be/en).
#
# http://github.com/hpcugent/easybuild
#
# EasyBuild is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation v2.
#
# EasyBuild is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with EasyBuild. If not, see <http://www.gnu.org/licenses/>.
##
"""
EasyBuild support for gmpolf compiler toolchain (includes GCC, MPICH2, OpenBLAS, LAPACK, ScaLAPACK and FFTW).
@author: Dmitri Gribenko (National Technical University of Ukraine "KPI") (copy from...)
@author: Bart Verleye (University of Auckland)
"""
from easybuild.toolchains.compiler.gcc import Gcc
from easybuild.toolchains.fft.fftw import Fftw
from easybuild.toolchains.linalg.openblas import OpenBLAS
from easybuild.toolchains.linalg.scalapack import ScaLAPACK
from easybuild.toolchains.mpi.mpich2 import Mpich2
class Gmpolf(Gcc, Mpich2, OpenBLAS, ScaLAPACK, Fftw):
"""Compiler toolchain with GCC, MPICH2, OpenBLAS, ScaLAPACK and FFTW."""
NAME = 'gmpolf'
name = 'MPICH'
version = '3.0.4'
homepage = 'http://www.mpich.org/'
description = """MPICH v3.x is an open source high-performance MPI 3.0
implementation.
It does not support InfiniBand (use MVAPICH2 with InfiniBand devices)."""
toolchain = {'name': 'GCC', 'version': '4.8.1'}
sources = [SOURCELOWER_TAR_GZ]
source_urls = ['http://www.mpich.org/static/tarballs/%s' % version]
# MPICH configure wants F90/F90FLAGS to be renamed to FC/FCFLAGS.
preconfigopts = 'export FC="$F90"; export FCFLAGS="$F90FLAGS"; unset F90; unset
F90FLAGS; '
sanity_check_paths = {
'files': ['bin/mpicc', 'bin/mpicxx', 'bin/mpic++', 'bin/mpif77',
'bin/mpif90',
'bin/mpiexec', 'bin/mpirun',
'include/mpi.h', 'include/mpi.mod', 'include/mpif.h'],
'dirs': [],
}
moduleclass = 'mpi'
