Re: [easybuild] undefined reference to 'omp_get_num_threads' despite openmp=True

[Ward Poelmans]

On 23/03/2020 14:08, Loris Bennett wrote:
> So the libraries seem to get build OK, but linking the client against
> them seems to fail, despite the fact that all the calls to 'gcc' seem to
> have the option '-fopenmp' set.

The issue is that the last one (--mode=link) does not have -fopenmp.
Either you need to add it to the link flags or manually add -lgomp to
the libraries to link to.

Ward

Re: [easybuild] undefined reference to 'omp_get_num_threads' despite openmp=True

[Ward Poelmans]

Hi Loris,

On 23/03/2020 13:25, Loris Bennett wrote:
> 
> ../lib/.libs/libgretl-1.0.so: error: undefined reference to 
> 'omp_get_num_threads'
> ../lib/.libs/libgretl-1.0.so: error: undefined reference to 'GOMP_barrier'
> ../lib/.libs/libgretl-1.0.so: error: undefined reference to 
> 'omp_get_thread_num'
> ../lib/.libs/libgretl-1.0.so: error: undefined reference to 'GOMP_parallel'
> ../lib/.libs/libgretl-1.0.so: error: undefined reference to 'omp_get_wtime'
> ../lib/.libs/libgretl-1.0.so: error: undefined reference to 
> 'omp_set_num_threads'
> collect2: error: ld returned 1 exit status

This means the linker is not pulling in `-lgomp`. I guess because
`-fopenmp` is not used? The lines you show here above the errors are
probably unrelated (can happen if you're not using -j1).

Ward

Re: [easybuild] CP2K-6.1-intel-2018a.eb builds broken version?

[Ward Poelmans]

Hi Loris,

On 25/06/19 11:02, Loris Bennett wrote:
> According to the following page
> 
>   https://www.cp2k.org/dev:compiler_support
> 
> CP2K is broken ("fails at runtime") if built with the Intel 18.0.1
> compiler.  However, that is the compiler version used for the easyconfig
> 
>   CP2K-6.1-intel-2018a.eb
> 
> Should this easyconfig be deprecated?

I've also seen this but my experience contradicts what CP2K claims. They
say intel/2018a should not work but intel/2018b would. In my tests it's
the other way round. Many (almost all) regtests fail if you try
intel/2018b. So I would stick to the 2018a version for now.

Or even better: use foss. For CP2K it seems to pass a lot more tests.

Ward

Re: [easybuild] CP2K psmp?

[Ward Poelmans]

Hi Loris,

On 17/06/19 16:23, Loris Bennett wrote:
> 
> Is there any particular reason why the buildopts don't just specify, say,
> 
>   VERSION="sopt popt ssmp psmp"
> 

There is no specific reason for this. I shouldn't be very hard to adjust
the easyblock to accept this.

Ward

Re: [easybuild] Non-standard PATH?

[Ward Poelmans]

Hi Loris,

On 27/03/19 13:35, Loris Bennett wrote:

> 
> However, the binaries are not directly under 'bin', but in the
> subdirectories shown below:
> 
> [build@admin bin]$ tree -L 1
> .
> ├── em64t-unknown-linux-gnu
> ├── em64t-unknown-linux-gnu_mpi
> ├── em64t-unknown-linux-gnu_smp
> ├── x86_64-unknown-linux-gnu
> ├── x86_64-unknown-linux-gnu_mpi
> └── x86_64-unknown-linux-gnu_smp
> 
> What is the standard way of tweaking the PATH variable in such a case?

modextrapaths = {'PATH': ['bin/x86_64-unknown-linux-gnu']}

would work. But it it's a lot of path, maybe a easyblock is nicer.

Ward

Re: [easybuild] Checksum errors while installing GCCcore 6.4.0

[Ward Poelmans]

Hi Sona,


On 13/03/19 10:22, Ms. Sona CP wrote:

> I am getting this checksum error message while installing GCCcore-6.4.0
> file.
> I have cross checked the easyconfig files but couldn;t find a solution.
> Can any one please help with this?
> I have attached my log file.

It's a bit of a cryptic error? I only find:
make[2]: *** [s-attrtab] Killed

I'm not seeing any errors related to checksums?

Could it be that you've hit an out of memory situation where the kernel
kills a process?

Ward

Re: [easybuild] Specifying location of configure

[Ward Poelmans]

On 22/02/19 14:49, Loris Bennett wrote:
> Hi,
> 
> Just writing my first easyconfigs from scratch.  The first went OK, but
> with the second, 'configure' is not in the root of the tarball, but in
> the subdirectory 'src'.  I assume this is described the documentation,
> but I wasn't able to find it.
> 
> So how do I indicate the location of 'configure' in the easyconfig?

Hi Loris,

You should look at the 'start_dir' setting for easyconfig. EasyBuild
will change to that directory after extracting and before running the
configure step.

Ward

Re: [easybuild] Fix incorrect paths in modules

[Ward Poelmans]

On 14/01/19 11:07, Loris Bennett wrote:
> 
> I assume easiest solution is just to rebuild all the module files as
> user 'build'.  Is that correct?

Yes, paths may be put in all kind of places. The only way to be sure is
to rebuild.

Ward

Re: [easybuild] next EasyBuild conf call: Wed Dec 12th 2018 (*today*), 5pm CET

[Ward Poelmans]

On 13/12/18 00:46, Jack Perdue wrote:
> I think that would be wonderful if we could just
> give the user the option of flat or hierarchical instead
> of imposing one or the other on all of them.

Just add all the modules paths to $MODULELIST and you've switched from
hierarchical to flat, no?

Ward

Re: [easybuild] Retrospectively hiding modules / hiding by default

[Ward Poelmans]

On 23/11/18 10:57, Ole Holm Nielsen wrote:

> 
> It's also possible to hide modules by configuring Lmod, see the
> isVisibleHook function in
> https://lmod.readthedocs.io/en/latest/170_hooks.html

We hide modules from older toolchain with the hooks:
https://github.com/sisc-hpc/Lmod-UGent/blob/master/SitePackage.lua#L214

You can use regex, Lmod properties, etc to hide modules.

Ward

Re: [easybuild] VMD-1.9.3-intel-2016b-Python-2.7.12.eb segfaults

[Ward Poelmans]

Hi John,

On 18-12-17 15:52, John Donners wrote:
> 
> the problem seems to be in LLVM compiled with the intel compiler when
> used as backend for Mesa.
> It can be circumvented when using
> 
> GALLIUM_DRIVER=swr vmd
> 
> but that is slower, so not preferrable. Did anyone get VMD running with
> LLVM as driver for gallium (which is the default)?

I would first try a more recent version of Mesa (and LLVM). In some
cases the swr backend can be considerable faster (but you need at least
AVX).



Ward

Re: [easybuild] zlib is not statically linked

[Ward Poelmans]

Hi Raphaël,

On 25-09-17 09:33, Philippart Raphaël wrote:

> Do you have an explanation for my zlib problem and his statically link
> problem ?
> I deleted in the .eb the zlib’s dependency and I have the same result 

I'm seeing things like:
checking for library containing zlibVersion... no
checking for library containing dlopen... none required
checking zlib.h usability... no
checking for spawnvpe... yes
checking zlib.h presence...  bg da de eo es fi fr ga hu id it ja ms nl
pt_BR ro ru rw sr sv tr uk vi bg da de eo es fi fr ga hu id it ja ms nl
pt_BR ro ru rw sr sv tr uk vi
checking whether NLS is requested... yes
checking for msgfmt... yes
checking for zlib.h... yes

Can you verify that libz.a is actually present in the zlib module?
And share the 'config.log' of binutils?

I also see a warning about the static library. Normally should libz.a be
compiled with -fPIC, can you check in the build log?

It's a strange case, the build commands seem fine to me. Can you also
check if the libraries in the lib directory don't link to libz.so?

Ward

Re: [easybuild] Mesa and vglrun

[Ward Poelmans]

On 22-09-17 13:39, alexi.riv...@chalmers.se wrote:
> So I'm wondering if there are any recommendations for how to compile software
> with 3D-acceleration?

Hi Alexi,

The Mesa easyconfig of EasyBuild currently doesn't build with support
for hardware 3D acceleration. It requires the presence of a whole bunch
of libraries and kernel stuff. You could do it but it would suggest to
use a recent enough kernel for it.

Ward

Re: [easybuild] zlib is not statically linked

[Ward Poelmans]

On 20-09-17 10:56, Philippart Raphaël wrote:

> I’m trying to install the package ROOT and bam-readcount. I have in the
> 2 scenarii the same error : 
> 
> zlib is not statically linked in
> /cm/shared/apps/easyb/.local/easybuild/software/binutils/2.25-GCCcore-4.9.3/bin/addr2line:
> \tlinux-vdso.so.1

Hi Raphaël,

Can you share the debug logs with us? zlib should indeed be linked in
statically for binutils but it seems to fail for some reason.

Ward

Re: [easybuild] custom glibc

[Ward Poelmans]

On 24-08-17 15:53, Jure Pečar wrote:
> 
> Hi all,
> 
> I found this thread from about two years ago:
> www.mail-archive.com/easybuild@lists.ugent.be/msg01795.html
> 
> Have things changed since? Would newer glibc still present a pain from 
> easybuild point of view?

Not really. It's not impossible but you will be treading on unexplored
paths.

> New glibc goodies that got me to think in this direction are thread cache and 
> vectorized basic math functions.

If you want thread cache etc, just use jemalloc? If you want the
vectorized math, use the intel compiler?

Ward

Re: [easybuild] Spider output - whatis vs help

[Ward Poelmans]

On 04-07-17 08:43, Alvarez, Damian wrote:
> I think there is no reason to break backwards compatibility. But we still can 
> make it better adding templates like %(pyver)s and allowing whatis to be 
> defined using them. For instance:
> 
> whatis = ["Homepage: %s" % homepage, "Description: %s" % description, “List 
> of extensions: %(extension_list)s”]

What would be really nice if that when people do `ml spider numpy`, they
get the Python module (where numpy is in the extensions list).

Ward

Re: [easybuild] building mxnet

[Ward Poelmans]

On 27-06-17 11:41, Jure Pečar wrote:
> On Mon, 8 May 2017 12:27:55 +0200

> Just haswell remaining ... and boom. Sanity check for python fails with:
> 
> == 2017-06-27 11:05:27,003 extension.py:181 WARNING Extension: MXNet failed 
> to install, cmd 
> '/g/easybuild/x86_64/CentOS/7/haswell/software/Python/2.7.12-foss-2016b/bin/python
>  -c "import mxnet"' (stdin: None) output: terminate called after throwing an 
> instance of 'std::system_error'
>   what():  Operation not permitted
> 
> Which is something I'm seeing for the first time. Google tells me that is is 
> somehow related to -pthread or a bug in the code itself. Both seems unlikely 
> since builds for different architectures passed.
> 
> Any hint how to figure out this one?

Unfortunately, no. You might try running it with strace but I'm not sure
if that will tell you much. You can also try a newer version of MXNet?
If it's a bug in de code, it might have been fixed already.

We've build it on haswell without issue.


Ward

Re: [easybuild] Local user intstallation

[Ward Poelmans]

You can decide to hide a module based on it's full name, short name or
path to the module file. So yes.

Ward

On 28-06-17 09:55, Alan O'Cais wrote:
> So you're saying you can select to hide all  modules called
> Core/Compiler/MPI (in a HMNS) which you pick up
> from $EB_GLOBAL_PREFIX/modules/all ?
> 
> On 28 June 2017 at 09:46, Kenneth Hoste  > wrote:
> 
> Hi Davide & Miguel,
> 
> With Lmod, there's another option: you can selective hide modules
> from users, without breaking existing 'module load' commands, to
> actively discourage them from using old installations for example.
> 
> This can be done in various ways, e.g. by putting .modulerc files in
> place, or by implementing the isVisible hook (cfr. Ward's hook
> implementations at
> https://github.com/TACC/Lmod/tree/master/contrib/more_hooks
> )
> 
> 
> regards,
> 
> Kenneth
> 
> 
> On 28/06/2017 01:01, Miguel Costa wrote:
>> Hello Davide,
>>
>> On 28 Jun 2017 02:43, "Vanzo, Davide" > > wrote:
>>
>> So, I gave it a try and it works fine.
>>
>> The only annoyance (since it is more a matter of style) is
>> that once I add the $EB_GLOBAL_PREFIX/modules/all to the
>> $MODULEPATH, Lmod output for "module avail" gets pretty crowded.
>>
>>
>> I know what you mean :)
>>
>> I was simplifying, I don't actually expose modules/all to normal
>> users by default (they can always add it later, either by changing
>> MODULEPATH directly or running "module use ...")
>>
>> Besides the module classes and hidden modules, what works for me
>> is grouping the modules by toolchain (could this be added as a
>> feature?) and exposing those groups separately 
>>
>> This has allowed me to, for instance, "deprecate" toolchains that
>> created problems in many of our applications (again, users can add
>> them back manually)
>>
>> You call also use the ordering in MODULEPATH to put a desired
>> group at the top or bottom of the output of "module avail" (it
>> would be interesting if lmod had a customizable "module shortlist"
>> command...)
>>
>> Don't know how all this changes if you're using HMNS (see Alan's
>> message)
>>
>> Miguel
>>
>>
>>
>>
>> -- 
>> Davide Vanzo, PhD
>> Application Developer
>> Advanced Computing Center for Research and Education (ACCRE)
>> Vanderbilt University - Hill Center 201
>> (615)-875-9137 
>> www.accre.vanderbilt.edu 
>>
>> On Jun 27 2017, at 10:43 am, Vanzo, Davide
>> > > wrote:
>>
>> Thank you all for the great suggestions!
>>
>> DV
>>
>>
>> On Jun 27 2017, at 3:14 am, Miguel Dias Costa
>> > > wrote:
>>
>>
>>
>> On 27/06/17 15:42, Alan O'Cais wrote:
>>> That is a neat trick with the sourcepaths that I will
>>> try out today.
>>
>> I have to credit boegel, he told me about it, I had
>> simply assumed that would not work :)
>>
>>>
>>> I think there is a bit more to a user install space
>>> than what's been mentioned when you have a HMNS,
>>> see 
>>> https://github.com/hpcugent/easybuild-framework/issues/2143
>>> 
>>> 
>>
>> ah... because there are then two roots to the
>> hierarchy, the global one and the local one?
>>
>> Miguel
>>
>>>
>>> On 27 June 2017 at 03:09, Fotis Georgatos
>>> > wrote:
>>>
>>>
>>> Hi Miguel,
>>>
>>> Yeap! i’d also fully applause this approach, it
>>> is very similar to what we’ve been doing since 2012!
>>> - $MODULEPATH can include both global and local
>>> builds, you just need to decide their exact order
>>>   * fyi. recent versions of Lmod have in
>>> ./contrib a pair of use.own.eb modulefiles,
>>> implementing the above
>>> - the global install, is nothing more than by yet
>>> another user, which just happens to be steered by
>>> admin
>>> - i also encourage people to add a versionsuffix
>>> with their initials and maybe a date, to keep
>>> things apart
>>>
>>>  

Re: [easybuild] Strategy for hiding modules

[Ward Poelmans]

If you're still interested, the isVisible() hook has been merged. An
example on how to use it can be found at
https://github.com/TACC/Lmod/pull/276/files

You can dynamically hide a module based on any criteria of your desire,
time, toolchain version, ...

Ward

On 07-04-17 14:32, Alan O'Cais wrote:
> Funnily enough, there's a PR for Lmod that suits exactly your use case:
> https://github.com/TACC/Lmod/pull/248
> 
> 
> On 7 Apr 2017 3:05 pm, "Andreas Hilboll"  > wrote:
> 
> Hi all,
> 
> when I started using EasyBuild, I didn't know about the option to hide
> modules.  Slowly, the list of installed modules becomes too long for
> `module avail` to be useful for the user.  So I would like to hide
> modules.  The question is, how should I do this?
> 
> 1. Re-generate all modules (can I use EasyBuild to regenerate modules,
>hiding them if I want to, without recompiling the code?)
> 
> 2. Manually hide modules (probably a bad idea)
> 
> 3. ???
> 
> It would be nice to have a best-practices document somewhere, but I
> couldn't find any. [*]
> 
> Also, from what I understand, using minimal toolchains is generally a
> good idea to keep the number of modules low -- correct?
> 
> I'd appreciate any guidance on how I can clean up my installation.
> 
> Cheers,
>   Andreas.
> 
> PS: I'm using lmod
> 
> [*] I'd be happy to write a blog post from a non-expert's perspective
> (I'm no sysadmin, and our cluster is rather smallish, almost no
> technical staff to maintain the software).
> 
> 
> 
> 
> 
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> 
> 
> 

Re: [easybuild] building mxnet

[Ward Poelmans]

On 02-05-17 17:13, Jure Pečar wrote:
> On Tue, 2 May 2017 15:36:27 +0200
> Ward Poelmans <wpoel...@gmail.com> wrote:
> 
>> Hi Jure,
>>
>> There are now an easyblock [1] and easyconfig [2] for MXNet.
>>
>> Can you give them a spin?
> 
> Nice ... I'll have to learn from this example how to handle submodules :)
> 
> 0d64855f741e04.tar.gz I find downloaded in the sources dir has a hash of 
> 4d83e41d97419ce04e8394dbb8893042, not fb04cb1694f1e66591a31a18bc92b451 ... 
> any idea why?

OK, they did something I didn't expect. Apparently commit 0d64855f741e04
for nnvm was also added to mxnet itself. This makes that EB downloads
the wrong tarball...

I've changed the commit I use of nnvm for now. I will look into the
framework so we can specify from which url which tarball should be
downloaded.

Ward

Re: [easybuild] Software/easyconfigs update avail check

[Ward Poelmans]

On 04-05-17 10:39, Jens Timmerman wrote:
> 
> On 04/05/2017 08:31, Henkel, Andreas wrote:
>> Hi, 
>>
>> Recently we started using eb for our new cluster. Yesterday, in our group 
>> meeting a question was raised concerning updates and security patches for 
>> installed software similar to apt update/upgrade, yum update,...?
>> Or is there a routine to check for newer releases of installed easyconfigs? 
> Short answer: No
> 
> There are no such systems in place, EasyBuild does not have a security
> team, so there are no systems in place for installing security updates.

There is currently one exception: openssl. We have an easyconfig for it
but we put it as a osdependency in all cases. So whatever security
updates happen to the OS provided version of openssl, Easybuild uses it.

Ward

Re: [easybuild] building mxnet

[Ward Poelmans]

On 25-01-17 11:51, Jure Pečar wrote:
> 
> Hi all,
> 
> I'm trying to build mxnet thru EasyBuild. It's quite a challenge ...

Hi Jure,

There are now an easyblock [1] and easyconfig [2] for MXNet.

Can you give them a spin?

[1] https://github.com/hpcugent/easybuild-easyconfigs/pull/4346
[2] https://github.com/hpcugent/easybuild-easyblocks/pull/1135

Ward

Re: [easybuild] mpi4py on OpenMPI 2.x (foss/2017a)

[Ward Poelmans]

Hi Joachim,

On 11-04-17 15:39, Joachim Hein wrote:
> Hi,
> 
> I have a user with MPI4PY issues.  MPI4PY out of the box wants to have a 
> multi threaded MPI library, which is not supported in OpenMPI 1.x on IB.  So 
> building on top of foss/2016b and foss/2016a is not an option.  foss/2017a is 
> an option, but there are currently no Python packages for this.  Could I get 
> a quick reply on status/issues around Python for foss/2017a and intel/2017a.  
> Is that something wrong in principle, is there something in the making or 
> should I just have a go at moving e.g. the Python config for foss/2016b to 
> foss/2017a?  I can engage here, but like to avoid duplication of effort, if 
> someone else is already on the case.


There is a 'Python-2.7.13-intel-2017a.eb' easyconfig in develop. Do
--try-toolchain on that for foss/2017a?

Ward

Re: [easybuild] GPG signing RPM in EasyBuild

[Ward Poelmans]

On 04-04-17 14:32, Siddiqui, Shahzeb wrote:
> Exactly, if there is a way to store GPG key like a GitHub token we could
> get this to work.
> 

I might be missing something here but why can't we handle the GPG key id
as an ordinary commandline argument?

The ID of GPG key is not sensitive information? We could just pass the
id to fpm and let the GPG agent handle all sensitive stuff.

Ward

Re: [easybuild] Extended package documentation

[Ward Poelmans]

On 14-02-17 20:08, Kenneth Hoste wrote:
> 
> I feel this sort of terse key/value style could be useful, but only for
> single-line info (e.g. homepage, contact, etc.).
> 
> For things like 'description' and 'usage', I'm not sure this would be
> useful or sensible.
> 
> So, maybe what you're looking for is some hybrid approach, where aspects
> like 'description' and 'usage' (and 'examples') are properly fleshed out
> sections, where all the other info is packed together in an easy to
> digest (for machines) form, like the key-value style you showed.

I don't really agree with that. The output show be nicely human
readably. If you want to parse it, we should let Lmod output json (or
some other serialization format). I wouldn't bother too much with making
it both human + machine readable at the same time.


> In an ideal world, we would have a sensible default output format, and
> you would support the necessary knobs and dials to tweak the output to
> your liking.

For Lmod we could probably get quite far by adding some hooks.

Ward

[easybuild] Lmod no autoswap error customization

[Ward Poelmans]

Hi all,

If you use Lmod as your module system, you can either allow or disable
'auto swap'. If you want to mimic Tmod behaviour as close as possible,
you usually don't want this to happen. A user might get confused by it
(especially in a flat module tree).

However, Lmod gives out a rather 'complex' error which might not
always make sense to the user. With hooks, we can give a 'nicer' error
for EasyBuild created modules:

$ ml purge
$ ml ncurses/5.9-intel-2015a
$ ml ncurses/5.9-intel-2015b
Lmod has detected the following error:  A different version of the
'ncurses' module is already loaded (see output of 'ml').


If you don't understand the warning or error, contact the helpdesk at 

$ ml libreadline/6.3-intel-2014b
Lmod has detected the following error:  A different version of the
'intel' module is already loaded (see output of 'ml').
You should load another 'libreadline' module for that is compatible
with the currently loaded version of 'intel'.
Use 'ml spider libreadline' to get an overview of the available versions.


If you don't understand the warning or error, contact the helpdesk at 
While processing the following module(s):
Module fullname  Module Filename
---  ---
libreadline/6.3-intel-2014b
/home/opt/easybuild/modules/all/libreadline/6.3-intel-2014b

The `SitePackage.lua` to do this can be found at
https://github.com/TACC/Lmod/tree/master/contrib/more_hooks

Ward

Re: [easybuild] configure: error: cannot run C compiled programs.

[Ward Poelmans]

Hi Satish,

On Mon, Jan 30, 2017 at 11:57 AM, Satish Sherikar
 wrote:
>
> Currently i am installing NCL, HDF5 applications everytime it is giving same
> error.
>
> can you please help me to resolve this issue.
>
> I have attached log file for the same.

Can you also post the `config.log` for HDF5? It's in the build dir.

Ward

Re: [easybuild] LUA modules: whatis statement split up across multiple lines when printed

[Ward Poelmans]

On Thu, Jan 19, 2017 at 10:38 PM, Ben Roberts  wrote:
> This is a problem for me and my site as we prepare our lists of modules by
> looking at the output of “module whatis”. While we could post-process that
> output, it is somewhat tedious and error prone to have to do so. The
> formatting may also be somewhat ugly in general when users consult a module
> whatis.

Why not use the json output of spider to process the list of modules?

In principle, you can even write a class to let Lmod write json
instead of bash/perl/... to the screen.

But I agree with Jack. We try to limit the number of character on a line to 120.

Ward

Re: [easybuild] Python-2.7.12-foss-2016b undefined references

[Ward Poelmans]

On 18-01-17 22:16, Vanzo, Davide wrote:
> Ward,
> the symbols exist in the libreadline.a, not in libncurses.a as you can
> see here:
> 
> $ readelf --syms
> /usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
> | grep 'tputs\|tgoto\|tgetnum\|PC\|BC\|UP\|tgetent\|tgetstr\|tgetflag'
>97:  0 NOTYPE  GLOBAL DEFAULT  UND tputs
>   127:  0 NOTYPE  GLOBAL DEFAULT  UND tgoto
>70:  0 NOTYPE  GLOBAL DEFAULT  UND tgetnum
>97:  0 NOTYPE  GLOBAL DEFAULT  UND PC
>99:  0 NOTYPE  GLOBAL DEFAULT  UND BC
>   100:  0 NOTYPE  GLOBAL DEFAULT  UND UP
>   101:  0 NOTYPE  GLOBAL DEFAULT  UND tgetent
>   102:  0 NOTYPE  GLOBAL DEFAULT  UND tgetstr
>   104:  0 NOTYPE  GLOBAL DEFAULT  UND tgetflag
>   118:  0 NOTYPE  GLOBAL DEFAULT  UND tputs

The 'UND' means that the library needs the symbol but doesn't have it.

> $ readelf --syms
> /usr/software/software/Core/ncurses/6.0/lib/libncurses.a | grep
> 'tputs\|tgoto\|tgetnum\|PC\|BC\|UP\|tgetent\|tgetstr\|tgetflag'
>28:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
>41:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
>14:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp
>25:  0 NOTYPE  GLOBAL DEFAULT  UND tputs_sp

This is weird, they should be here. It can be put the libtinfo library
but the current easyconfigs don't build it. Can you share the build log
of ncurses? And check if they are in the .so version?

Ward

Re: [easybuild] Python-2.7.12-foss-2016b undefined references

[Ward Poelmans]

Hi Davide,

On Wed, Jan 18, 2017 at 9:05 PM, Vanzo, Davide
 wrote:
> When trying to build Python-2.7.12-foss-2016b it fails with the error below.
>
> =
> /usr/software/software/Compiler/GCC/5.4.0-2.26/libreadline/6.3/lib/libreadline.a
> /usr/software/software/Core/ncurses/6.0/lib/libncurses.a   -lm
> ./libpython2.7.so: error: undefined reference to 'tputs'
> ./libpython2.7.so: error: undefined reference to 'tgoto'
> ./libpython2.7.so: error: undefined reference to 'tgetnum'
> ./libpython2.7.so: error: undefined reference to 'PC'
> ./libpython2.7.so: error: undefined reference to 'BC'
> ./libpython2.7.so: error: undefined reference to 'UP'
> ./libpython2.7.so: error: undefined reference to 'tgetent'
> ./libpython2.7.so: error: undefined reference to 'tgetstr'
> ./libpython2.7.so: error: undefined reference to 'tgetflag'
> collect2: error: ld returned 1 exit status
> make: *** [python] Error 1
>
> ==
>
> The weird thing is that libreadline.a contains such symbols when checking
> with readelf. Has anyone had this problem before?

Yes, I've seen this before. That's why we link libreadline to ncurses.
I'm surprise you get this error when both the static archive for
libreadline and ncurses are include.

So you are sure that those symbols exists in libncurses.a and it is defined?

Can you try what happens if you put the paths for libncurses and
libreadline before -lpython2.7?

> On a side note, while digginig into libreadline and ncurses, I noticed that
> although we want libreadline linked to our custom version of ncurses 6.0, it
> gets actually linked to the system ncurses library. This can be solved by
> modifying the easyconfig to read as follows:
>
> preconfigopts = 'env LDFLAGS="-lncurses -L$EBROOTNCURSES/lib"'
>
> Is it just me having this problem or can be something worth enhancing?

That's definitely a bug. Please open an issue on github (or pull request).

Ward

Re: [easybuild] Training session at S3IT: GC3Pie for programmers, January 23--27, 2017

[Ward Poelmans]

Hi Riccardo,

On 10-01-17 13:10, Riccardo Murri wrote:
> 
> S3IT is glad to announce a training session "GC3Pie for programmers".
> It is targeted at current and prospective users who want to develop their
> own scripts and workflows using GC3Pie.

Any chance of streaming/recording the training?

Ward

Re: [easybuild] A question on Intel 2017 cluster edition

[Ward Poelmans]

On 04-01-17 11:01, Kenneth Hoste wrote:
> 
> 
> On 04/01/2017 02:41, Jack Perdue wrote:
>> On 01/03/2017 06:55 PM, Exequiel Sepúlveda wrote:
>>>
>>> 1.- Why are GCC and binutils needed on the easyconfigure file
>>> intel-2017.00.eb
>>
>> See other response.  Additionally, the Intel compilers depend
>> upon an underlying GCC (with its own underlying binutils),
>> so needs to be included.

Correct. For example, to use C++11, you need a recent GCC version. So we
let Easybuild control everything.

>>> 2.- Because I have only a big tar.gz file with Intel Cluster Edition,
>>> how could I separate the required parts to use easyconfigs file for
>>> icc, ifort, imkl, etc.
>>
>> You need to download all the individual, offline, downloads
>> for the different packages from the Intel site, not the great big one.
> 
> This is the current situation, indeed, but we can work out a way to
> install icc, ifort, impi and imkl from the single big .tar.gz.


Well, actually it should work (but I haven't tried it).

We specify the list of components for icc and ifort so even if you use
the big tarball, it should just install what is needed.

Ward

Re: [easybuild] How to setup EB in the global /etc/profile.d/ ?

[Ward Poelmans]

On 13-10-16 15:20, Ole Holm Nielsen wrote:
> On 10/13/2016 01:54 PM, Ward Poelmans wrote:
>> On 13-10-16 13:48, Ole Holm Nielsen wrote:
>>
>> For loading Easybuild by default, I would use a Lmod default collection.
> 
> Could you possibly offer a specific example of what you mean?  Like a
> complete script to put into /etc/profile.d/ ?

I'm afraid I don't have one. We don't load Easybuild by default for our
users. But you should take a look at:
https://lmod.readthedocs.io/en/latest/070_standard_modules.html

The only thing we do is load Lmod itself (a symlink to init/bash in
/etc/profile.d).

Ward

Re: [easybuild] How to setup EB in the global /etc/profile.d/ ?

[Ward Poelmans]

On 13-10-16 13:48, Ole Holm Nielsen wrote:
> I would like to enable the EB/Lmod modules to all users in a global
> bash/tcsh setup script.  On CentOS 7.2 initialization is done using
> scripts in /etc/profile.d/
> 
> Basically I want every user shell to execute these commands:
> 
> export EASYBUILD_MODULES_TOOL=Lmod
> export EASYBUILD_PREFIX=/home/modules
> module use $EASYBUILD_PREFIX/modules/all
> module load EasyBuild

The settings you can put in a global config file? Much safer then
environment variables.

For loading Easybuild by default, I would use a Lmod default collection.

Ward

Re: [easybuild] OpenFOAM module file

[Ward Poelmans]

On 05-07-16 13:01, Erik Smeets wrote:
> 
> The problem in our opinion would be that a user wouldn't return to a clean 
> environment when unloading the modules. Furthermore, this complicates 
> switching between different versions. What's your thought on this and how do 
> you approach these requirements within your systems?

You have several options:
- re-implement all the logic from the bashrc file in the module file
itself. Downside: lots of work and maintain heavy.
- Create OpenFOAM wrapper that does the source for you. In that way the
environment is never polluted.
- Always go to a subshell before sourceing the file.

We don't have much trouble with it because it's used most often within a
job file and not that many people switch OpenFOAM modules.

But you're right, it's not very clean.

Ward

Re: [easybuild] How to name easyconfigs for alternative build options

[Ward Poelmans]

On 01-07-16 10:53, Andreas Hilboll wrote:
> 
> How should I go about naming the easyconfig, so that in the end I have
> two distinct module files, one with 'nonetcdf' somewhere in its name?

The current approach is to use a versionsuffix for this. In the
`nonetcdf` you simple add:
versionsuffix = '-nonetcdf'
and rename the easyconfig with the suffix.

Ward

Re: [easybuild] 2016b toolchains: proposal & test results

[Ward Poelmans]

On 30-06-16 13:52, Backeljauw Franky wrote:
> Hi Kenneth,
> 
> Two small questions:
> 
>   * Why don’t you condiser GCC 6.1 (or GCC 6.0) to be part of “all the
> latest & greatest”?
>   * Is it necessary to have identical version of GCC for the foss and
> intel toolchains?

Well, you could use a different GCC for both but that means a different
GCCcore for each. And one of the reason for using the same is that we
can have dependencies (like automake, libtool, ncurses, ...) that work
for both foss and intel. And intel only officially support GCC 5.x for now.

Furthermore, we didn't test GCC 6.1 heavily so far (anybody else how
did?). And as GCC changed the default C++ version from 98 to 11, it
might break quite a few things. The Fedora people found more broken
builds then usual after doing a GCC 6 rebuild.

So, the suggestion is to wait until GCC 6.2 (or greater) is released
before switching. Any objections?

Ward

Re: [easybuild] ccache support in EasyBuild?

[Ward Poelmans]

For ordinary EasyBuild use, I agree it's quite useless.


However, when creating new easyblocks/easyconfigs, it can be quite handy to 
speed things up. Usually, it will involved build the same software several 
times until the new easyblock fully works.


Ward


From: xavier.besse...@gmail.com  on behalf of Xavier 
Besseron 
Sent: Monday, June 27, 2016 2:56 PM
To: easybuild@lists.ugent.be
Subject: Re: [easybuild] ccache support in EasyBuild?

Hi Easybuilders,

My opinion is that ccache will be pretty useless for EasyBuild.
The typical usecase of ccache is when you build the same source (in the same 
directory), with the same compiler, with the same options.
Basically, it will speedup a "make ; make clean ; make".

I don't think that's a common workflow with EasyBuild.

https://ccache.samba.org/manual.html#_how_ccache_works

Bonus:
https://ccache.samba.org/performance.html



Best regards,

Xavier



On Thu, Jun 23, 2016 at 8:59 AM, Kenneth Hoste 
> wrote:
(migrated from Lmod-users mailing list)

Does anyone have experience with combining ccache with EasyBuild?

Would it make sense to add support for something like 'eb --use-ccache', to 
make EasyBuild create symlinks to ccache for each of the compilers it may be 
using, and shove the path to them first in $PATH?


regards,

Kenneth

On 22/06/16 21:56, Jack Perdue wrote:
/me ponders whether Robert (or Kenneth) might think this discussion
might not be better served on the EasyBuild list

jack (who's down with Adam... and used ccache a lot while
  grinding out RPMs on local hardware but doesn't mess
  with it it much these days on our clusters
  http://siliconslick.com/papitools/epel/6/SRPMS/ ).

On 06/22/2016 12:12 PM, Adam Huffman wrote:
Used to be very grateful for ccache a few years ago when I was
regularly building large Fedora RPMs at home.

ccache -s

shows you how well it's working.

It is something I've wondered about in connection with EasyBuild.

Does it impair strict reproducibility?

On Wed, Jun 22, 2016 at 5:13 PM, Orion Poplawski 
> wrote:
On 06/22/2016 10:06 AM, Kenneth Hoste wrote:
Hi Orion,

How much speedup have you seen with this setup, do you have any numbers on that?

How often is ccache able to recycle object files (I guess that's where the
biggest speedup is coming from) when different compilers are involved?
Sorry, I really don't have any numbers.  Would be nice to have though.

Any caveats at all with this, e.g. have you seen builds fail when using ccache
but work fine without it?
I've not seen any issues using ccache.

This looks like it could be an interesting feature in EasyBuild
(--use-ccache), which could injects the symlinks to ccache at runtime in
$PATH... :-)


regards,

Kenneth

On 22/06/16 17:27, Orion Poplawski wrote:
Perhaps others might find this useful.

ccache is a compiler cache that can speed up multiple builds of larger
projects.  In Fedora/EPEL the ccache package puts links of compiler names
pointing to ccache in /usr/lib{,64}/ccache and adds
/etc/profile.d/ccache.{c,}sh to prepend that directory to PATH.

Unfortunately when loading a compiler module, the compiler's directory gets
prepended to the PATH and ccache is no longer active.  Locally I install this
Core modulefile:

$ cat /opt/modulefiles/Core/ccache.lua
prepend_path{"PATH","/usr/lib64/ccache",priority=100}

With the ccache module loaded then it keeps the /usr/lib64/ccache path at the
front.  Adjust priority as needed for your installation.

And I also install links for icc/icpc, although I'm not particularly sure how
much ccache works with icc/icpc though I came across an (older) Intel web page
that mentioned using ccache with icc.

$ ls -l /usr/lib64/ccache/icc /usr/lib64/ccache/icpc
lrwxrwxrwx. 1 root root 16 Jun  2 15:03 /usr/lib64/ccache/icc ->
../../bin/ccache
lrwxrwxrwx. 1 root root 16 Jun  2 15:03 /usr/lib64/ccache/icpc ->
../../bin/ccache

HTH,

 Orion

--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane   or...@nwra.com
Boulder, CO 80301   http://www.nwra.com

--
Attend Shape: An AT Tech Expo July 15-16. Meet us at AT Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
___
Lmod-users mailing list
lmod-us...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lmod-users

Re: [easybuild] eb --job

[Ward Poelmans]

On 03-06-16 16:27, Riccardo Murri wrote:
>>> Can you please provide me with the full `qstat -f` output for a
>>> completed job?
>>
>> it is here
> 
> Thanks! (And thanks to Ward as well, who provided another example)
> 
> One quetion more: this is very similar to what PBSPro provides with its
> `qstat -x -f` (the `-x` is for including e(x)ited jobs in the output).
> 
> Does TORQUE's `qstat` also accept a `-x` option?  Or does it complain?

>From the man page:
   -xSpecifies that the output is to be displayed in XML
form.  This option is only valid with the -f option or by itself, which
will also specify the -f full status display.


Ward

Re: [easybuild] Option --dep-graph

[Ward Poelmans]

On 03-06-16 13:13, nan...@luis.uni-hannover.de wrote:
>  
>  Hi, 
> 
>  I get the following error when I try to use the option -dep-graph, thought I 
> have installed all necessary packages following the doc
> 
>>  eb foss-2016a.eb -r --dep-graph ~/foss-2016a.eb.png
> == temporary log file in case of crash /tmp/eb-1nclYZ/easybuild-O4tEAl.log
> == resolving dependencies ...
> Error: renderer for .png is unavailable
> Wrote dependency graph for 24 easyconfigs to /home/nanava/foss-2016a.eb.png
> 
> It says that  *.png has been created, but in fact it is not.
> 
> However if I create *.dot first, then converting to pdf|png works
> 
>> eb foss-2016a.eb -r --dep-graph ~/foss-2016a.eb.dot
> == temporary log file in case of crash /tmp/eb-eDeX_3/easybuild-o6SPIj.log
> == resolving dependencies ...
> Wrote dependency graph for 24 easyconfigs to /home/nanava/foss-2016a.eb.dot
> 
>> dot -Tpng ~/foss-2016a.eb.dot -o ~/foss-2016a.eb.png
> 
> Is this a known issue ?

I'm not sure what you mean? Easybuild doesn't output png, it gives you
the intput for the dot progam. You always need to make the
png/pdf/svg/... yourself with dot.

Ward

Re: [easybuild] [ANN] EasyBuild v2.8.1

[Ward Poelmans]

On 30-05-16 13:47, Kenneth Hoste wrote:

> I'm happy to announce the release of EasyBuild version 2.8.1 [1].
> Yes, that's right, EasyBuild v2.8.0 just got even better!

This really doesn't capture the importance of this release. In its
continuous quest for world dominance, EasyBuild took an import step: we
support our first window manager[1]! Granted, JWM is not Gnome or KDE
but step by step, we're getting there.

Ward

[1] https://github.com/hpcugent/easybuild-easyconfigs/pull/3007

Re: [easybuild] binutils for foss/2016a

[Ward Poelmans]

On 03-05-16 17:00, Heywood, Todd wrote:

> *** static library 
> /sonas-hs/it/hpc/home/easybuild/install_sbox/software/Core/zlib/1.2.8/lib/libz.a
>  is not portable!

That's weird. It seems -fPIC is needed when compiling zlib. Very strange
that this didn't happen automatically. On what system are you building this?


Ward

Re: [easybuild] configure flag for openmpi

[Ward Poelmans]

Hi Yann,


Just add the options to the `configopts` string in the easyconfigs of OpenMPI?


Ward

Re: [easybuild] Python packages?

[Ward Poelmans]

On 08-02-16 22:50, Elizabeth Fischer wrote:

> Is there any documentation on how EasyBuild interacts with Python
> packages?  I see there's an EasyBuild for matplotlib, but not for
> basemap.  What is the recommended way to go about installing stuff? 
> Pure EasyBuild?  EasyBuild + PIP?  EasyBuild + Conda?

basemap was added in 2.6? [1].

The method of installing depends on what you want. There is a generic
pythonpackage easyblock that you can use. But using easy_install/pip/etc
on top of a EB python install should also work.

Ward

[1] https://github.com/hpcugent/easybuild-easyconfigs/pull/2221

Re: [easybuild] Problems with installation of Clang under Cent OS 6.6/6.7

[Ward Poelmans]

Hi Victor,

I have retested the Clang 3.6 easyconfig on our SL6.6 and CO7 systems.
Both work fine. But the sanitizers tests can be very picky about their
environment and don't always get along on a HPC system.

You can download the new block:
https://raw.githubusercontent.com/hpcugent/easybuild-easyblocks/develop/easybuild/easyblocks/c/clang.py
and use it with the --include-easyblocks=clang.py option.

Add the line skip_sanitizer_tests=True to the easyconfig to skip the
tests. To keep the building directory, use the
--disable-cleanup-builddir to not delete the build dir. Keep in mind
that Clang is not build in the source dir (seperate build dirs per stage
are created).



Ward

On 14-01-16 11:57, ПРОСОФТ HPC wrote:
> Hi Ken, Fotis,
> 
> yesterday I googled the failed tests and found two related bug-reports:
> 
> https://llvm.org/bugs/show_bug.cgi?id=22760
> https://llvm.org/bugs/show_bug.cgi?id=22810
> 
> It looks like starting from the v3.5 release Clang requires Glibc v2.13 or 
> newer.. 
> 
> Anyway, I was able to succesfully install Clang-3.4.2-GCC-4.8.2.eb building 
> block. Now I am happy except only one thing. Actually, I need to install some 
> other LLVM modules such as SAFEcode, DragonEgg. The problem is that after 
> succesfull installation of a building block the building directory is 
> automatically cleaned up. Could you please advise me how to force EB to 
> prevent cleaning source as well as building directories?
> 
> Thanks you very much for your help and assistance!
> 
> With best regards, 
> Victor
> 
> 14.01.2016, 02:09, "Fotis Georgatos" <fo...@mail.cern.ch>:
>>  Hi Ken, Victor,
>>
>>  On Jan 13, 2016, at 3:25 PM, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
>>>   According to our findings, "ulimit -v" must be set to 'unlimited', while 
>>> "ulimit -s" must be set to something else than 'unlimited' for the tests to 
>>> work.
>>>   There may be more going on though, since that's exactly what you have 
>>> according to your debug log.
>>>   Maybe the stack size limit should be higher? Not sure…
>>
>>  I’ve seen this issue before. I hadn’t debugged it fully, but it appeared to 
>> relate to glibc-2.12 of RHEL6 and friends (hello Centos6, hello SL6);
>>
>>  AFAIK, the issue goes away with glibc 2.13 and anything greater. If I 
>> recall the story right, clang v3.4 and above will fail the sanitiser test,
>>  because they are picky for a situation that can indeed imply a race 
>> condition (if true, well done! :) - can someone else confirm this?
>>  btw. try to install either of clang/3.3 and clang/3.4 to verify the above 
>> statement, across glibc cases.
>>
>>>   Ward has issued a PR to add a new option to hard disable only the 
>>> sanitizer checks, see 
>>> https://github.com/hpcugent/easybuild-easyblocks/pull/806.
>>>   That should go in very soon, and so will be part of the next release 
>>> (EasyBuild v2.6.0).
>>
>>  Nice to have that as a tuneable.
>>
>>>   With that in place, adding "skip_sanitizer_tests = True" will be 
>>> sufficient to dance around this issue, as opposed to having to skip *all* 
>>> tests and disable bootstrapping, as I mentioned earlier.
>>>   We should consider having that enabled by default to skip these 
>>> troublesome tests... Ward?
>>
>>  I’m inclined to agree, however it’s good to have a better understanding if 
>> that introduces an “unsafe" situation.
>>  Basically, if behaviour is the same like it used to be with clang/3.3, it 
>> may be OK to disable as default even if imperfect;
>>  but if the v3.4 sanitiser checks are more strict for a strong reason (race 
>> conditions looming ahead), I’d be more skeptical...
>>  more input needed.
>>
>>  Fotis
>>
>>  --
>>  echo "sysadmin know better bash than english" | sed s/min/mins/ \
>>| sed 's/better bash/bash better/' # signal detected in a CERN forum


-- 
dr. ir. Ward Poelmans
High Performance Computing infrastructure unit
Ghent University
Krijgslaan 281 S9
B-9000 Gent
Belgium
Tel: +32 9 264 4817
http://www.ugent.be/hpc

Re: [easybuild] foss/2016a and intel/2016a common toolchains

[Ward Poelmans]

On 11-01-16 21:50, Kenneth Hoste wrote:

> ** GCC 4.9.3 vs GCC 5.3.0 **
> 
> The question here is if we stay with the latest release of GCC v4.x,
> which as it happens is still the same version as the one included in the
> 2015b toolchains, or if we make the jump to GCC 5.x.
> 
> My personal feeling is that it's (still) too early to jump to GCC 5.x,
> and that we should stick to GCC 4.9.3 for the time being.
> 
> The main reasons for this are that several Linux distributions are yet
> to pick up GCC 5.x as the main system compiler, and thus that there are
> many unknown issues that are bound to pop up left and right, and that
> the latest release of the Intel compilers is known not to be fully
> compatible yet with GCC 5.x (cfr. [4]).

I feel adventurous and would suggest that we do move to GCC 5. The major
issues are the new ABI of libstdc++ and that it defaults to c++11.

If we stick the default to c++98 and avoid the (
-D_GLIBCXX_USE_CXX11_ABI=0) we should be fine. Maybe some poorly written
build systems will assume that the major GCC version is always 4 but as
GCC already did a couple of major version bumps, I think it will be a
minor issue.


> * GCC version in foss vs intel
> 
> With the GCCcore concept that was introduced in EasyBuild v2.5.0, it is
> possible to use a common GCC version as a base for both the foss and
> intel toolchains (e.g. 4.9.3), while using a different GCC version in
> foss (e.g. 5.3.0).

What future do we see for GCCcore? I would keep it fixed at 4.9.3 for
quite some time in the future. As it is a base compiler, it not a big
deal that it's not the latest and greatest version?


Ward

Re: [easybuild] mpiicc and mpiifort, and --cleanup-builddir

[Ward Poelmans]

Hi Ben,

On 12-01-16 04:51, Ben Roberts wrote:

> I’m still trying to get my head around how I can use the Intel compilers with 
> MPI. Maybe this isn’t an issue now that I’m at EB 2.6.0.
> 
> In brief, it seems that whatever I do the compilers being used are mpicc and 
> mpif90. I understand that this uses the GCC compilers even though the Intel 
> toolchain has been requested.

As Jack already pointed out, we set the compiler that mpicxx and friends
should use.

> The program I’m trying to build (CPMD), unfortunately, incorporates the value 
> of the CC environment variable and so on into a Makefile. This means that if 
> these environment variables are reset between the configure and make 
> operations, the changes aren’t reflected in the build step.

That's annoying but unfortunately too common (See all the patches in the
tree). Patching the makefiles is the only way...

> This brings me to the second problem. I’m trying to find all the places where 
> CC and so on are explicitly set, so they can be commented out if need be 
> (thus forcing the use of the environment value of CC). But the build 
> directory is deleted at the end of the build. I’ve tried the 
> --cleanup-builddir option, and this doesn’t prevent the deletion, and 
> --cleanup-builddir=false gives me a rather terse warning about how 
> --cleanup-builddir doesn’t take options.
> 
> How can I keep the build dir for investigative purposes? “cleanupoldbuild” in 
> the easyconfig doesn’t seem quite right as this seems related to previous 
> builds, but maybe I’m missing something here.

You need --disable-cleanup-builddir.


Ward

-- 
dr. ir. Ward Poelmans
High Performance Computing infrastructure unit
Ghent University
Krijgslaan 281 S9
B-9000 Gent
Belgium
Tel: +32 9 264 4817
http://www.ugent.be/hpc

Re: [easybuild] Intel-2016.xx toolchains

[Ward Poelmans]

Hi Franky,

On 22-12-15 12:02, Backeljauw Franky wrote:
> 
> I’ve just installed EasyBuild 2.5.0 and noticed there are several 
> intel-2016b.xx toolchains now:

Yes, that's because we haven't decided about which one is going to be
intel/2016a. Kenneth was going to send a mail about it but he forgot.

But for the intel case, most likely the 2016.01 will be promoted to
2016a. On the foss toolchain we still have to decide whether to stick
with GCC 4.9.x or go to 5.x.

> Another final question: any idea how to get rid of the UserWarning?

Yes, that shouldn't be there. Open a bug report so we don't forget about it.


Ward

Re: [easybuild] [ANN] EasyBuild v2.5.0

[Ward Poelmans]

 in one way or
>>> another!
>>>
>>>
>>> To upgrade to EasyBuild v2.5.0, there are several options:
>>>
>>>(i) (re)bootstrap EasyBuild to obtain an EasyBuild/2.5.0 module to
>>> load [5]
>>>
>>>(ii) install EasyBuild v2.5.0 with a previous version of
>>> EasyBuild, using the easyconfig file available in [6]
>>>
>>>(iii) install EasyBuild v2.5.0 from PyPI, using one of the
>>> standard Python installation tools (easy_install, pip, ...)
>>>
>>>(iv) update your Git working copies of the different EasyBuild
>>> repositories
>>>
>>>
>>> Enjoy!
>>>
>>> regards,
>>>
>>> Kenneth
>>> EasyBuild release manager
>>>
>>> [1] http://pypi.python.org/pypi/easybuild
>>> [2]
>>> http://easybuild.readthedocs.org/en/latest/Controlling_compiler_optimization_flags.html
>>> [3] http://easybuild.readthedocs.org/en/latest/Packaging_support.html
>>> [4] http://easybuild.readthedocs.org/en/latest/Release_notes.html
>>> [5]
>>> http://easybuild.readthedocs.org/en/latest/Installation.html#bootstrapping-easybuild
>>> [6] https://github.com/hpcugent/easybuild-easyconfigs/pull/2238
>>>
>>
>> -
>> phone: +31 6 143 66 783
>> e-mail: pieter.neeri...@gmail.com <mailto:pieter.neeri...@gmail.com>
>> skype:  pieter.online
>> -
>>
> 


-- 
dr. ir. Ward Poelmans
High Performance Computing infrastructure unit
Ghent University
Krijgslaan 281 S9
B-9000 Gent
Belgium
Tel: +32 9 264 4817
http://www.ugent.be/hpc

Re: [easybuild] cannot build old GCC on recent Ubuntu

[Ward Poelmans]

On 30-10-15 10:41, Martin wrote:
> On Thu, Oct 29, 2015 at 2:37 AM, Riccardo Murri  > wrote:
> 
> > Why, oh why, would you need a GCC that old on a recent system?
> 
> To compile code that was known to compile correctly with GCC 4.5 and is
> no longer accepted by GCC 4.9.  (Later on, I found about `-fpermissive`
> but I still think that `.eb` files should either work or be removed from
> the repo if they cannot be built any more ;-))
> 
> 
> I thought the promise that we can rebuild things is one of the core
> promises of easybuild?
> 
> I have to admit I'm a bit worried. I *heavily* rely on this to be
> covered by the "reproducible build" argument that comes up every other
> thread.
> 
> So what if I decide to switch distros or create another cluster that
> replaces the current one, shouldn't I be able to rebuild the stuff
> without needing to worry about the underlying distribution?

Well, building GCC (especially older GCC on newer systems) has always
been difficult. If you would switch distro to something untested (like
Slackware), you probably bump into a few issues. Nothing we cannot solve
but sometimes, distro just do weird stuff. Like move a file to a
subdirectory in this case. Or lib/lib64 weirdness on opensuse.

Ward

Re: [easybuild] cannot build old GCC on recent Ubuntu

[Ward Poelmans]

On 29-10-15 02:37, Riccardo Murri wrote:

> 2. have `EB_GCC` supply `MAKEINFO=missing` in all stages when the
>build-dependencies are ignored.
> 
> If you don't have 1. then build will fail on all distributions that have
> texinfo >= 5.0.0; but 2. is needed because `builddependencies` are
> ignored in early GCC build stages.

In the past we used the missing solution.

>> Maybe sys/cdefs.h moved, and GCC 4.5 isn't aware of that yet?
> 
> Well, `sys/cdefs.h` is included by `/usr/include/features.h` which is
> *not* part of GCC.  So it looks more like a missing component in the std
> include path for `xgcc`.
> 
> 
>> Have you tried something like: CPATH=/usr/include/x86_64-linux-gnu:$CPATH eb
>> GCC-4.5.3.eb ?
> 
> Thanks for the suggestion!  Unfortunately, it did not work -- same error.

That's normal. We unset the CPATH before building GCC due to ubuntu
weirdness in the past. The problem is that ubuntu pusts `sys/cdefs.h` in
a subdirectory of `/usr/include` for some reason. On Fedora, it can be
found directly under `/usr/include`. The most dirty solution would be to
simply symlink the directory. The proper solution would be to adapt the
build system. We can add the directory to the include list in the
easyblock when detecting ubuntu?


Ward

Re: [easybuild] GROMACS-5.1-intel-2015b-hybrid.eb: error in test

[Ward Poelmans]

On 28-10-15 09:46, Backeljauw Franky wrote:
> Actually, we are not getting compilation errors when building GROMACS itself. 
> It only fails in the testing phase.
> But when we do this step manually, after having loaded the necessary modules, 
> it works fine.
> So that gives us the impression that it’s an EasyBuild issue instead…
Do you run the Easybuild run inside a jobscript? I've seen issues with tests of 
some software because of the job environment.

Ward

Re: [easybuild] GNU toolchain

[Ward Poelmans]

On 25-10-15 12:30, Markus Geimer wrote:
> I would have expected that binutils are a dependency (rather than a
> builddep) of GCC.  The reason is that with a hierarchical module
> naming scheme, it is perfectly valid to not expose the EB toolchains
> to users by not adding 'Core/toolchain' to the default modulepath.
> This means that when I (as a software developer) load GCC, I may end
> up with a broken installation that can't compile code optimized for
> my platform -- depending on the underlying system-provided binutils.
> This looks like a major flaw to me...

Hi Markus,

What happens when building the GNU toolchain is:
- Build binutils with system GCC
- Build GCC using the previous binutils and system GCC
- Build binutils with EB GCC
- Create GNU toolchain that loads both EB GCC and the previous binutils

This is the reason why binutils is a build dep for GCC, we only mean to
use that binutils once as it was built using the system GCC.

The flaw is that you try to load the GCC module itself while you should
be loading the GNU module. But the naming is indeed confusing.

Ward

-- 
dr. ir. Ward Poelmans
High Performance Computing infrastructure unit
Ghent University
Krijgslaan 281 S9
B-9000 Gent
Belgium
Tel: +32 9 264 4817
http://www.ugent.be/hpc

Re: [easybuild] Use of environment variables (CC, etc.) in patched configure scripts

[Ward Poelmans]

On Tue, Oct 13, 2015 at 7:57 PM, Ward Poelmans <ward.poelm...@ugent.be> wrote:
> On Tue, Oct 13, 2015 at 6:58 PM, Kenneth Hoste <kenneth.ho...@ugent.be> wrote:
>> No, but you can do the reverse (as you already know, I think), using the
>> 'unwanted_env_vars' easyconfig parameter, that takes a list of names of
>> environment variables that should be unset.
>> I think this is a better approach too: see what makes the build trip, and
>> unset that. $LIBS is a common one.
>
> That doesn't work. `unwanted_env_vars` only works for variables not set by EB.

Addition, you can unset a variable using buildopts etc:
buildopts = "unset LIBS && "
will work. The hard part is of course finding which variable is causing havoc.

Ward

Re: [easybuild] Maintaining patches

[Ward Poelmans]

Hi Adam,

On Wed, Oct 7, 2015 at 11:51 AM, Adam Huffman <adam.huff...@crick.ac.uk> wrote:
>
> Other than checking the code manually, is there a way of tracking why these 
> patches were applied? I see there’s a PR, as yet unmerged:

Not really. Our policy these days is that every patch should include a
top comment about what it does, why it is needed and who the author or
source is.

But for an older patch, the proof of the pudding is in the eating. If
it is has been mainlined, the patch_step will simply fail. The easiest
thing to do is just drop it and see if the sanity check passes.

Ward

-- 
dr. ir. Ward Poelmans
High Performance Computing infrastructure unit
Ghent University
Krijgslaan 281 S9
B-9000 Gent
Belgium
Tel: +32 9 264 4817
http://www.ugent.be/hpc

Re: [easybuild] Sourceforge download

[Ward Poelmans]

What about:

sources= ['%(name)s_%(version)s.tar.gz']
source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files/',
'download')]

Ward

On Thu, Sep 24, 2015 at 10:55 AM, Niek de Klein <niekdekl...@gmail.com> wrote:
> Hi easybuild list,
>
> I'm trying to install Bridger from sourceforge. The sourceforge link is:
>
>
> sourceforge.net/projects/rnaseqassembly/files/Bridger_r2014-12-01.tar.gz/download
>
> So I tried using
>
> name = 'Bridger'
> version = 'r2014.12.01'
> [..]
> source_urls = [SOURCEFORGE_SOURCE]
> sources = ['%%(name)s_%s.tar.gz' % '-'.join(version.split('.'))]
>
> but this does not find the file to download. Looking at other programs
> installed from sourceforge it looks for this
>
>
> sourceforge.net/projects/rnaseqassembly/files/bridger//2014.12.01/Bridger_2014-12-01.tar.gz/download
>
> so I think because this does not make use of
> // SOURCEFORGE_SOURCE won't work,
> correct?
>
> So then I tried with
>
> source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
>  sources = ['%%(name)s_%s.tar.gz/download' %
> '-'.join(version.split('.'))]
>
> But this gives the error
>  Unknown file type for file
> /apps/sources/b/Bridger/Bridger_r2014-12-01.tar.gz/download (['download'])")
>
> Without /download at the end it doesn't find the url.
>
> Other things I have tried:
>
>source_urls = ['sourceforge.net/projects/rnaseqassembly/files/']
>sources = [('%%(name)s_%s.tar.gz' %
> '-'.join(version.split('.')),'download')]
>
>
>   sources = [SOURCELOWER_TAR_BZ2]
>   source_urls = [('http://sourceforge.net/projects/rnaseqassembly/files/',
> 'download')]
>
> and
>
>   sources = [SOURCELOWER_TAR_BZ2]
>   source_urls = [('http://sourceforge.net/projects/rnaseqassembly/',
> 'download')]
>
> But then the files can't be found. I don't know what else to try, how can I
> get this downloaded from sourceforge? Below the full .eb file for
> replication:
>
> easyblock = 'MakeCp'
>
> name = 'Bridger'
> version = 'r2014.12.01'
>
> homepage = 'https://wiki.gacrc.uga.edu/wiki/Bridger'
> description = """Bridger is an efficient de novo trascriptome assembler for
> RNA-Seq data. """
>
> toolchain = {'name': 'goolf', 'version': '1.7.20'}
> toolchainopts = {'pic': True, 'usempi': True}
>
>
> sources = [SOURCELOWER_TAR_BZ2]
> source_urls = [('http://sourceforge.net/projects/rnaseqassembly/',
> 'download')]
>
>
> files_to_copy = [('Assemble','bin/')]
>
> moduleclass = 'bio'
>
>
>
>



-- 
dr. ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] 2015b toolchains : non-GCC- and GCC-versions

[Ward Poelmans]

On Wed, Sep 2, 2015 at 2:56 PM, Bode, Brett  wrote:
> How do you handle the binutils dependency for other software (like mpich)? I 
> have been using the GCC-4.9.2-binutils-2.25 version on a haswell node. 
> Binutils is required to build GCC, but it is also required to build other 
> software with GCC. However, those packages do not list binutils (either 
> version) as a dependency. My hack to get around the problem was change the 
> plain binutils module in a regular “dependency” rather than a 
> builddependency. Probably not the best solution but to be honest I think the 
> GCC-4.9.x-binutils-2.25 sort of implies that it is included.

The easiest thing to do is to use the foss toolchain
(compiler+blas/lapack+mpi) or the GNU toolchain (just gcc+binutils).
You can binutils for free with these.

Ward

Re: [easybuild] 2015b toolchains : non-GCC- and GCC-versions

[Ward Poelmans]

On Wed, Sep 2, 2015 at 2:50 PM, Backeljauw Franky
 wrote:
> I guess the first one can go (since it’s just a dependency to build 
> binutils/2.25 as you mentioned below), but what’s the difference between 
> GCC-4.9.3-binutils-2.25 and GCC-4.9.3-2.25?

M4/1.4.17 is a build dependency for binutils/2.25

M4/1.4.17-GCC-4.9.3-binutils-2.25 is a build dependencies for
binutils/2.25-GCC-4.9.3

GNU is the toolchain combining GCC+binutils

M4/1.4.17-GNU-4.9.3-2.25 is the M4 for the actual toolchain and used
by the other easyconfigs in the toolchain.

You can delete the first two without trouble (I think), the latter is a keeper.

Ward

Re: [easybuild] GCC-4.9.3-binutils-2.25 : invalid ELF header

[Ward Poelmans]

On Tue, Sep 1, 2015 at 4:44 PM, Backeljauw Franky
 wrote:
>
>
> The one I found back is about "CMake-3.3.0-intel-2014b : problem building”
> also on this mailing list, dated 28 October 2014.
> It was an issue with GPFS 3.5.0-20, currently we have GPFS 4.1.0-8, so it
> could be something ‘similar’…

I've seen too many weird thing happen on a network fs when building
with EB. I strongly recommend that you build only on a local fs or
/dev/shm.

Ward

Re: [easybuild] install from git clone

[Ward Poelmans]

On Fri, Aug 28, 2015 at 2:42 PM, Niek de Klein  wrote:
> git clone https://github.com/voutcn/megahit.git

They have releases so you can simplify use
https://github.com/voutcn/megahit/archive/v1.0.2.tar.gz as source url?

Ward

Re: [easybuild] Tarball Easyblock sanity_check fails

[Ward Poelmans]

On Mon, Aug 10, 2015 at 4:39 PM, Stolpe, Oliver
 wrote:

> When I comment out the sanity_check part, then it complains that lib and
> lib64 are missing, but I there is nothing to check for easybuild and I
> didn't request them?
>
> Sanity check failed: no dir of ('lib', 'lib64')

Dear Oliver,

That's odd indeed. Can you show us the debug log?

Ward

Re: [easybuild] Installations that require non-trivial patches

[Ward Poelmans]

On Fri, Jul 24, 2015 at 4:55 PM,   wrote:
>
> The first program is GROMACS that I need to patch with plumed. I need to have
> the GROMACS files un-tared, and then run the plumed patch command in their
> directory. During the patching plumed suggests several versions of GROMACS and
> asks which one you are trying to patch. It expects the number of the correct
> option as an answer.

I once started on this:
https://github.com/hpcugent/easybuild-easyconfigs/pull/1218

My solution would be to generate the patch for plumed on GROMACS
yourself and add that to EB, like in my PR.

The downside is that this patch needs to be regenerate for every
version change. You can write an easyblock to do this automatically
but that's quite a bit of work and I didn't have the time for that.

Ward

Re: [easybuild] difference between `--filter-deps` and `--hide-deps` ?

[Ward Poelmans]

On Fri, Jul 31, 2015 at 2:35 PM, Riccardo Murri  wrote:
> What's exactly the difference between the two?  Does "filter" consider
> the dependency satisfied (hence, skip building), whereas "hide" still
> builds it but does not provide a module file?

The --hide-deps will create a hidden module. The module will be
installed but invisible with module av.

The --filter-deps will simply skip the given dependencies and EB will
assumed it is supplied in some other way.

Ward

Re: [easybuild] conflicts with system libraries

[Ward Poelmans]

On Fri, Jul 31, 2015 at 5:49 AM, Riccardo Murri  wrote:
>
> Am I doing something wrong here?  Is there a well-known solution/fix?

Hi Riccardo,

It's a known issue: https://github.com/hpcugent/easybuild-easyconfigs/pull/1670

I still have to divide into the exact cause of this issue but you can
use that PR as a temporarily fix.

Ward

Re: [easybuild] (g)awk and easybuild

[Ward Poelmans]

On Fri, Jul 10, 2015 at 4:26 PM, Stolpe, Oliver
 wrote:
>
> $ gawk
> gawk: symbol lookup error:
> /tools/easybuild/software/libreadline/6.3-foss-2015a/lib/libreadline.so.6:
> undefined symbol: PC

Dear Oliver,

Please try this: https://github.com/hpcugent/easybuild-easyconfigs/pull/1670

It's for the intel toolchain but the same thing should work with the
foss toolchain.

Ward

Re: [easybuild] next EasyBuild conf call: Wed July 8th, 3pm CET

[Ward Poelmans]

On Wed, Jul 8, 2015 at 5:58 PM, Jack Perdue  wrote:
> But _something_ needs to be done to make sure that toolchains
> in EB can keep up with compiler/MPI/mathlib releases.

So basically you would like a way to generate a toolchain with the
gcc, icc, ifort, imkl, impi, ... of your choice? We could write a
script for that. It's not hard to make a custom toolchain, just
tedious.

The problem is of course the infinite number of possible combinations.
--try-toolchain will get you far but merging these easyconfigs back to
the main tree will be trouble.

Ward

Re: [easybuild] cd to a directory with easybuild

[Ward Poelmans]

On Thu, Jul 2, 2015 at 6:05 PM, Yann Sagon  wrote:
> I have copied the Makefile in my work directory as Makefile.o and Makefile
> Modified the Makefile.
> Created the diff : diff -u Makefile.o Makefile >
> Gadget-2.0.7-makefile-hardcoding.patch
>
> Error message: "Can't determine patch level for patch"

You can specify the patch level with a tuple:
patches = [('Gadget-2.0.7-makefile-hardcoding.patch', 1)]
will give a you a patch level of 1. An alternative is to specify the
relative directory path where to apply the patch, for example
patches = [('Gadget-2.0.7-makefile-hardcoding.patch', '..')]

Ward

Re: [easybuild] cd to a directory with easybuild

[Ward Poelmans]

Hi Yann,

Welcome to Easybuild.

On Thu, Jul 2, 2015 at 5:46 PM, Yann Sagon  wrote:
> I'm trying to write my first easyconfig file (for Gadget2). Once untarred, 
> the source of Gadget2 contains a directory named Gadget2, so I need to 
> instruct easybuild to cd to that directory. Is this possible or I need to 
> write my own easyblock? I'm using the easyblock MakeCp as there is no 
> configure.

You don't need to do anything. EB will first extract the provide
source and then cd in the directory that was created. It is only when
multiple directories are created that you need to help EB and tell
which one to use (with the start_dir variable in the easyconfig). If
you run eb with the -l option, you should see that it changes to the
extracted directory, the variable is called start_dir.

Ward

Re: [easybuild] wget GCC issue

[Ward Poelmans]

On Fri, Jun 26, 2015 at 7:04 PM, Todd Gamblin  wrote:
> In Spack, the compilers don't actually have to be loaded, so your new
> compilers stay out of the way of the system software.  The stuff compiled
> with Spack gets RPATH'd, so it knows how to find its libraries -- in the
> gcc build we actually modify gcc's spec to put an RPATH in to each
> compiler's runtime.  That's for compilers Spack actually built -- for
> system compilers, I'm still working on a way to make the Spack build
> automatically RPATH in compiler runtime libraries.  It *is* possible to do
> this.  CMake actually detects what it calls "implicit link libraries" for
> each compiler.  Most compilers can be coaxed into telling you what these
> are.

And what about the case where the user wants a compiler loaded to
compile some software himself? Is that handled by rpath's?

Ward

Re: [easybuild] wget GCC issue

[Ward Poelmans]

On Fri, Jun 26, 2015 at 3:47 PM, Stolpe, Oliver
 wrote:
> Hello List,
>
> I got another issue and wondered whether this is intentional or if there is
> a workaround for this. I am using easybuild on a debian right now. A lot of
> tools were compiled with the goolf-1.4.10 toolchain (GCC 4.7.2). When GCC is
> loaded, the command `wget` doesn't work anymore:
>
> $ wget
> wget: missing URL
> $ module load GCC/4.7.2
> $ wget
> wget: /tools/easybuild/software/GCC/4.7.2/lib64/libstdc++.so.6: version
> `CXXABI_1.3.8' not found (required by
> /usr/lib/x86_64-linux-gnu/libicuuc.so.52)
>
> Same happens for GCC/4.8.3. It works with GCC/4.9.2. I really wondered if
> this is intended behaviour. Thanks for any comments.

Hi Oliver,

That's because your Debian uses a newer GCC. Meaning: wget was
compiled with GCC-4.9 and if you load an older version of GCC, the
icuuc library will be looking for a symbol that did not yet exists in
that version of GCC. The only workaround here is to compile wget also
with EB, so that it can work with GCC/4.7.2. Or use a fully statically
linked version of wget.

Clashes between (older) version of base software are partially
unavoidable. The only real option is to have an environment completely
seperated from the distro libraries.

Ward

Re: [easybuild] 2015b common toolchains

[Ward Poelmans]

On Fri, Jun 19, 2015 at 12:05 PM, Martin  wrote:
> I'm not sure wether it should be part of the toolchain but to me "base
> tools/libs" should somehow be standardized on. I'm thinking along the lines
> of xz, bzip and such (I'm more along the lines of: Provide an environment as
> complete as possible for the users). Most of the time these tools are not
> technically required but I found that a lot of users expect the tools to
> simply be available and in a working state.

What kind of tools do you have in mind? xz and bzip2 already have to
be present as a requirement of EB.
It seems quite difficult to come up with a complete set? gawk, sed, grep, ...
I don't think we again much by adding these to EB?

I personally expect that any Linux computer has vim installed ;)
( and yes, Ubuntu, I'm looking at you )

Ward

Re: [easybuild] how to set dependency to not use compiler?

[Ward Poelmans]

Hi Niek,

On Tue, Jun 16, 2015 at 7:11 PM, Niek de Klein  wrote:
> ERROR: Irresolvable dependencies encountered: Weblogo/2.8.2-goolf-1.7.20
>
> when it should be using Weblogo/2.8.2.
>
> My dependencies in the HOMER.eb file look like this:
>
> dependencies = [
> ('Ghostscript', '9.16'),
> ('Weblogo', '2.8.2'),
> ('BLAT', '3.5'),
> ]

By default, EB will looks for easyconfig with exactly the same
toolchain. If you want another, you have to specify this. For the
dummy toolchain there is even a shortcut:
('Weblogo', '2.8.2', '', True)

For the full explanation:
https://easybuild.readthedocs.org/en/latest/Writing_easyconfig_files.html#dependencies

Ward

Re: [easybuild] Sending text to the command line during installation

[Ward Poelmans]

Hi Niek,

We have support for installing software through a question/answer
system. But you will need a custom easyblock for it. For example, take
a look at the Mathematica easyblock:
https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/m/mathematica.py

The install path can be access by `%(installdir)s` variable in an easyconfig.

Ward

On Thu, Jun 11, 2015 at 10:31 AM, Niek de Klein <niekdekl...@gmail.com> wrote:
> Hi all,
>
> I'm trying to write an .eb file for Anacona, which has a binary file
> installer. During installation you have to send to the command line:
> , yes, path/to/install/. I use easyblock = 'Binary'.
> The first two steps (pressing enter and saying yes) I can do by using
>
>   install_cmd = 'sh %(name)s-%(version)s.sh | yes'
>
> However, I don't know how to send the install path to the command line after
> pressing yes. Is there already support in EasyBuild for sending text to the
> command line during installation?
>
> Thanks,
> Niek



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] (Recursive) github checkout with easybuild

[Ward Poelmans]

On Wed, Jun 10, 2015 at 12:01 PM, Stolpe, Oliver
 wrote:
> Ok, can I somehow execute commands before it tries to do the fetching step? 
> What is about the 'skipsteps' parameter for the config file? I can't get from 
> the manual what values it accepts. Can I trick it by putting empty files in 
> the sources or will it try to unpack them?

The skipsteps works exactly as you expect. It accepts: fetch, ready,
source, patch, prepare, configure, build, test, install, extensions,
package, postproc, sanitycheck, cleanup, module, testcases.

But I don't understand what the problem is? Download the freebayes
sources with the git checkout, tar/zip it and let EB use that file?

If you put empty files, EB will try to use them (aka unpack).

Ward

Re: [easybuild] (Recursive) github checkout with easybuild

[Ward Poelmans]

On Wed, Jun 10, 2015 at 11:33 AM, Stolpe, Oliver
 wrote:
> How do I skip the fetching step in an EasyBlock, which seems to be mandatory?

The fetching step will be skipped if all files in `sources` can be
found locally ;)

Ward

Re: [easybuild] (Recursive) github checkout with easybuild

[Ward Poelmans]

Hi Oliver,

We don't really have support for that at the moment but you try this:
Download all git repo's you need through the usual download link:
https://github.com/ekg/freebayes/archive/c15a28363.zip

And repeat this for all 3 sub repo's.

Than you still have to move the repo into the right subdirectory and
tell EB which directory is the 'start_dir' with the
start_dir = "freebayes"

The moving can be done with a preconfigopts like:
preconfigopts = "mv ../vcflib vcflib ; mv ... ; "

It's ugly but the clean way would propably be to use a custom easyblock.

There is an open PR for freebayes:
https://github.com/hpcugent/easybuild-easyconfigs/pull/1165
but that skips the download issue by requiring a manual download ;)

Ward

On Wed, Jun 10, 2015 at 10:36 AM, Stolpe, Oliver
<oliver.sto...@charite.de> wrote:
> Hello mailinglist,
>
> I wondered if there is a possibility to configure or write an easyblock to
> do a recursive github checkout instead of fetching files. I wanted to follow
> this instructions:
>
> https://github.com/ekg/freebayes#obtaining
>
> What possibilities do I have and what is the most straight-forward way?
>
> Thanks & cheers,
>   Oliver



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] EasyConfig repos?

[Ward Poelmans]

On Sat, Jun 6, 2015 at 7:13 PM, Jack Perdue  wrote:
>
> Any plans for adding web-based repos for easyconfig files?
>
> e.g.
> EASYBUILD_ROBOT=/http/www.siliconslick.com/easybuild/eb_repos_cleaned/ada
>
> which would resolve to (via EB framework):

Hi Jack,

I've thought about extending the github support within EB. Currently,
you can use --from-pr to use easyconfig from a PR. This could be
extended to use any fork of the easyconfigs on github. It simply has
to download the tree, extract it and add it to the robot path.

That is still somewhat different from what you want but it boils down
to the same: download, extract and add to robot path.
The major complication is of course when the easyblock also requires
changes. We would need a way to signal a minimum version.

Ward

Re: [easybuild] Re: [easybuidl] intel vs gnu toolchains, easyconfigs explosion (was: Intel toolchains and mpiicc/mpiicpc/mpiifort)

[Ward Poelmans]

On Thu, Jun 4, 2015 at 10:02 AM, Kenneth Hoste  wrote:

> Version numbers are very confusing with the Intel tools (and EB doesn't make
> it any better...).
>
> The 2015 versions of the Intel tools are currently only in the most recent
> intel/ictce toolchains, i.e. intel/2015.02 and ictce/7.x .
>
> intel/2015a was fixed end 2014, when the 2015 versions of the Intel
> compilers was not available yet.

No, intel/2015a matches the Parallel Studio 2015 versions. Which is
v15 of the compilers.

Ward

Re: [easybuild] EasyBuild newbie question

[Ward Poelmans]

Hi Christopher,

Welcome to EB :)

On Mon, May 25, 2015 at 8:44 AM, Christopher Samuel
<sam...@unimelb.edu.au> wrote:

> However, for something as mainstream as GROMACS there doesn't appear to
> be a way to build a modern version with the Intel and OpenMPI chain.

Actually, we have a intel + openmpi toolchain: iomkl
( see the complete list at
https://github.com/hpcugent/easybuild/wiki/Compiler-toolchains )
You can use iomkl-2015.02.eb for example? If you want different
version, just make an easyconfig
of your own in that toolchain.

> The simplest solution I can find is to modify the Intel toolchain
> definition thus (and so avoid duplication billions of config files
> unnecessarily):

We have --try-toolchain to make your life easy. If you want GROMACS,
simplify pick the intel
toolchain easyconfig and use --try-toolchain=iomkl,2015.02
This will do exactly what the name suggest: use the iomkl toolchain
instead of the intel toolchain. The created easyconfig are saved in
your repository path.

For example:
$ eb --dry-run --try-toolchain=iomkl,2015.02 GROMACS-5.0.4-intel-2015a-mt.eb
 * [ ] $CFG/g/GCC/GCC-4.9.2.eb (module: GCC/4.9.2)
 * [ ] $CFG/i/icc/icc-2015.2.164-GCC-4.9.2.eb (module: icc/2015.2.164-GCC-4.9.2)
 * [ ] $CFG/i/ifort/ifort-2015.2.164-GCC-4.9.2.eb (module:
ifort/2015.2.164-GCC-4.9.2)
 * [ ] $CFG/i/iccifort/iccifort-2015.2.164-GCC-4.9.2.eb (module:
iccifort/2015.2.164-GCC-4.9.2)
 * [ ] $CFG/h/hwloc/hwloc-1.10.0-iccifort-2015.2.164-GCC-4.9.2.eb
(module: hwloc/1.10.0-iccifort-2015.2.164-GCC-4.9.2)
 * [ ] $CFG/o/OpenMPI/OpenMPI-1.8.4-iccifort-2015.2.164-GCC-4.9.2.eb
(module: OpenMPI/1.8.4-iccifort-2015.2.164-GCC-4.9.2)
 * [ ] $CFG/i/iompi/iompi-2015.02.eb (module: iompi/2015.02)
 * [ ] $CFG/i/imkl/imkl-11.2.2.164-iompi-2015.02.eb (module:
imkl/11.2.2.164-iompi-2015.02)
 * [ ] $CFG/i/iomkl/iomkl-2015.02.eb (module: iomkl/2015.02)
 * [ ] $TMP/tweaked_easyconfigs/bzip2-1.0.6-iomkl-2015.02.eb (module:
bzip2/1.0.6-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/zlib-1.2.8-iomkl-2015.02.eb (module:
zlib/1.2.8-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/ncurses-5.9-iomkl-2015.02.eb (module:
ncurses/5.9-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/libreadline-6.3-iomkl-2015.02.eb
(module: libreadline/6.3-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/Tcl-8.6.3-iomkl-2015.02.eb (module:
Tcl/8.6.3-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/CMake-3.2.2-iomkl-2015.02.eb (module:
CMake/3.2.2-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/libxml2-2.9.2-iomkl-2015.02.eb
(module: libxml2/2.9.2-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/SQLite-3.8.8.1-iomkl-2015.02.eb
(module: SQLite/3.8.8.1-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/Python-2.7.9-iomkl-2015.02.eb (module:
Python/2.7.9-iomkl-2015.02)
 * [ ] $TMP/tweaked_easyconfigs/Boost-1.58.0-iomkl-2015.02-Python-2.7.9.eb
(module: Boost/1.58.0-iomkl-2015.02-Python-2.7.9)
 * [ ] $TMP/tweaked_easyconfigs/GROMACS-5.0.4-iomkl-2015.02-mt.eb
(module: GROMACS/5.0.4-iomkl-2015.02-mt)



> The other thing I'm puzzled about is how do we make it more
> relaxed about version numbers of packages.  For instance it'd
> be really nice for it to not care whether it's 1.8.4 (which is
> older) or 1.8.5 (which is current but not yet present) or 1.8.6
> (which will be out soon).
>
> Similarly for GCC, if there is a GCC 4.7.x already installed and
> a package wants a slightly older version of 4.7 it would be nice
> if it didn't try and rebuild an entire toolchain because that
> precise version isn't there.

Alas, that is currently not possible. It is on our wish/todo list.
Kenneth can correct me but I don't think there is currently somebody
working on this.

> Am I on the right track with all this?

You sure are! Good luck with the new system.

Ward



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] Adding files to the repository

[Ward Poelmans]

On Thu, Apr 30, 2015 at 10:06 AM, Backeljauw Franky
 wrote:
> The main.filetools module seems to have a problem with the fact that the
> file build.conf does not exist at first…

Yes because it cannot derive the patch level for a non-existing file.
You will have to specify it manually:
patches = [('kwant-1.0.3-build-conf.patch', 1)]

Ward

Re: [easybuild] Adding files to the repository

[Ward Poelmans]

On Wed, Apr 29, 2015 at 5:15 PM, Backeljauw Franky
 wrote:
> I tried with a patch as in “diff -ru -NB” but that tells me it cannot 
> determine the patch level…

That should do the trick. Does the patch work if you try to apply it manually?

Autodetecting the patch level usually works but you can force it by
using a tuple.

Ward

Re: [easybuild] MKLROOT in modulefiles

[Ward Poelmans]

On Wed, Apr 1, 2015 at 4:28 PM, Inigo Aldazabal Mensa
 wrote:
>
> I noticed this as it already caused me a couple of conflicts with the
> canonical MKLROOT definition. Or am I missing something?

Hi Inigo,

You're correct. The fix is already made but not yet merged:
https://github.com/hpcugent/easybuild-easyblocks/pull/576
This should be part of Easybuild 2.1.

Ward

Re: [easybuild] Re:

[Ward Poelmans]

On Mon, Mar 9, 2015 at 5:07 PM, Don Moore  wrote:
> eb gmpolf-1.4.8.eb -r
> Fail:
> from openBlas ..
> ideas?
> -/Don

Can you send the --debug logs again?

Ward

Re: [easybuild] Issue with v2.0.0? (can not import _bdist_rpm)

[Ward Poelmans]

On Sun, Mar 8, 2015 at 5:20 PM, Timothy Brown
 wrote:

> == 2015-03-08 10:08:53,646 bootstrap_eb.run ERROR EasyBuild crashed with an 
> error (at easybuild/tools/run.py:384 in parse_cmd_output): cmd " python 
> setup.py install --prefix=/curc/tools/x86_64/rh6/software/EasyBuild/2.0.0 " 
> exited with exitcode 1 and output:
> Traceback (most recent call last):
>   File "setup.py", line 39, in 
> import vsc.install.shared_setup as shared_setup
>   File 
> "/tmp/tmpPK2YGE/EasyBuild/2.0.0/dummy-dummy/vsc-base-2.0.3/lib/vsc/install/shared_setup.py",
>  line 112, in 
> from distutils.command.bdist_rpm import bdist_rpm, _bdist_rpm
> ImportError: cannot import name _bdist_rpm
>
> Thought I'd ask here before digging into the code.

I think you're hitting this: https://github.com/hpcugent/vsc-base/issues/129

But it should be fixed. Which version of python and setuptools are you using?

Ward

Re: [easybuild] Rebuild modulefiles

[Ward Poelmans]

On Fri, Mar 6, 2015 at 1:55 PM,   wrote:
> Can one only rebuild a modulefile without going through a whole build?
> I've already built quite a lot of software and not keen on rebuilding
> everything just to recreate the modulefiles.

Not yet, but it's on the todo list:
https://github.com/hpcugent/easybuild-framework/pull/1018

Ward

Re: [easybuild] easybuild new user experience

[Ward Poelmans]

On Wed, Feb 11, 2015 at 2:32 AM, Stuart Barkley  wrote:
> I've been looking at easybuild for a little while now and it appears
> to be a suitable framework for a good portion of our needs.

Welcome to EB!

> My cluster does not have direct access to the Internet so I can't use
> any of the automated download procedures.  I actually consider this a
> feature in that I eventually know exactly what software is running and
> that nothing was downloaded and installed that wasn't intended.

What is stopping your users from copying other software to your
cluster and running it? Or using a ssh portforward/socks proxy to get
internet on your cluster?

> Dependencies/source package downloads:
>
> I think I eventually used "--dry-run --robot" and grepped the output
> to determine the full dependency lists and create individual 'eb
> --stop=fetch --force' commands for every package dependency.  I also
> used something like this to determine an internal build order so I
> didn't need to --robot 50 packages which would all churn and fail on
> the same dependency.

We could create an `--fetch-only` option that doesn't stop on a failed download.


> Some questions:
>
> How do people deal with software with frequent updates (java) or
> security issues?  Do you rebuild and remove old packages?

For the security sensitive package, we use the OS provides ones
(openSSL, glibc, ...) and let the OS update them.



> Is there any planned support (or something I'm missing) for allowing
> someone else to use an existing easybuild installation (including
> toolchains and other packages) to build a test/prototype package?  I
> would like to have other staff be able to build local test packages
> off of the production toolchains and be able to give me a set of
> proposed/working .eb files for final build.  This is similar to the
> above beta question, except I don't expect others to be able to
> maintain their own easybuild installation (and don't want them writing
> into the production installation).

You can use a different `-installpath=` for the test packages? If the
production toolchains are in the module path, they will be used.


> Wish list (maybe for hack-a-thon):
>
> Don't automatically create $HOME/.local/easybuild (or any other top
> level directory).  Too many times I forgot to specify my configuration
> file name or fumble fingered something and ended up with a mess of
> files where I didn't intend.  Requiring the top level directory to
> already exist prevents a misconfigured easybuild from writing a bunch
> of stuff for later cleanup.  The error message should be clear about
> the directory name so that a simple copy/paste can be used to create
> the missing directory when it is truly missing.

I don't agree here. Almost all Linux programs will create their own
directory under $HOME without asking. It's the proper thing to do. If
you have troubles forgetting the specify certain options, add them to
the config.cg file? Or export them as bash variables if the change
from location to location.


> Process for removing a package.  It is probably fairly simple, mostly
> deleting the installation tree and the modules files.  Dependencies
> might be tricky (indicating all the dependencies and refuse to remove
> would be fine).

This is on the wish list for quite some time:
https://github.com/hpcugent/easybuild-framework/issues/1000

Ward

Re: [easybuild] no sqlite3 in python builds

[Ward Poelmans]

On Thu, Jan 22, 2015 at 9:57 PM, Heywood, Todd  wrote:
> I have built Python2 and Python3 using standard easyconfigs and the goolf
> toolchain. But they don¹t include the sqlite3 module:

Hi Todd,

We're aware of the issue [1]. If you want to fix it, it should be as
simple as adding sqlite as a dependencies in the easyconfig of python.

Ward

[1] https://github.com/hpcugent/easybuild-easyconfigs/issues/1304

Re: [easybuild] Install extensions to already installed package

[Ward Poelmans]

On Tue, Dec 2, 2014 at 3:04 PM, Backeljauw Franky
 wrote:
>
> Just a small question: is it possible to install extensions to a package 
> instead of using a separate easyconfig-file?
> E.g., I want to install some extra python modules to an already install 
> Python installation where these modules did not appear in the extensions list 
> when Python was installed using EasyBuild…

Just add them to the existing Python easyconfig and use the --skip
option. This will only install the newly added extensions.

Ward

Re: [easybuild] CMake-3.3.0-intel-2014b : problem building

[Ward Poelmans]

On Wed, Oct 29, 2014 at 3:40 PM, Backeljauw Franky
 wrote:

>> Well, you have to tell it about icc/icpc. By default, CMake will look
>> for gcc and if it finds gcc, it will use it. EB sets several env
>> variables to point CMake in the right direction (CC, CXX, CFLAGS,
>> CXXFLAGS, ...). You also have to set these variables for CMake to use
>> the intel compilers. Just setting CC=icc CXX=icpc will probably do the
>> trick. If you build something with easybuild, this is all taken care
>> of.
>
>
> In that case, then what’s the benefit of having multiple CMake modules, e.g. 
> CMake-3.0.0-intel-2014b, CMake-3.0.0-foss-2014b, CMake-3.0.0-GCC-4.8.3? I 
> would think that, depending on the version you load, CMake would also know it 
> has to take the compiler with which it was built…

There is no benefit of having multiple CMake with different
toolchains. We already discussed multiple times what to do with
toolchain independent modules (stuff like autotools, cmake, bison,
...) and haven't come up with the perfect solution. They way EB
currently works, we need to build CMake for each toolchain. What
CMake-3.0.0-intel-2014b means is: CMake 3.0 build with intel 2014b
toolchain, not CMake 3.0 which always uses intel 2014b toolchain to
build stuff.

> If easy build takes care of this, then what does it do ‘extra’?

It sets a bunch of environmental variables and gives some arguments to
CMake where to look for includes, libraries, etc. Run easybuild with
the -l option, and you will see what it does (or just look into the
cmakemake class).

The fastest way to build something with the intel toolchain using a
CMake build system is: CC=icc CXX=icpc cmake .

Ward

Re: [easybuild] CMake-3.3.0-intel-2014b : problem building

[Ward Poelmans]

On Wed, Oct 29, 2014 at 12:05 PM, Backeljauw Franky
 wrote:

> We’ve managed to get CMake to build on our local filesystem, but when I load 
> it, it falls back to the system installed gcc instead of the intel toolchain:

> /tmp/hello_world> cmake .
> -- The C compiler identification is GNU 4.4.7
> -- The CXX compiler identification is GNU 4.4.7
> -- Check for working C compiler: /usr/bin/cc
> -- Check for working C compiler: /usr/bin/cc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working CXX compiler: /usr/bin/c++
> -- Check for working CXX compiler: /usr/bin/c++ -- works
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /tmp/hello_world


Well, you have to tell it about icc/icpc. By default, CMake will look
for gcc and if it finds gcc, it will use it. EB sets several env
variables to point CMake in the right direction (CC, CXX, CFLAGS,
CXXFLAGS, ...). You also have to set these variables for CMake to use
the intel compilers. Just setting CC=icc CXX=icpc will probably do the
trick. If you build something with easybuild, this is all taken care
of.

Ward

Re: [easybuild] CMake-3.3.0-intel-2014b : problem building

[Ward Poelmans]

On Tue, Oct 28, 2014 at 12:04 PM, Pablo Escobar Lopez
 wrote:
> I verified it when I tried the workaround and it seems to work fine, at
> least in this case.

That's because you are using a gcc toolchain and it is in the path. I
suspect this would fall back to system gcc with an ictce toolchain.

Ward

Re: [easybuild] CMake-3.3.0-intel-2014b : problem building

[Ward Poelmans]

On Tue, Oct 28, 2014 at 11:10 AM, Pablo Escobar  wrote:

> recently I wrote a FreeBayes easyconfig and I had to add this to the
> easyconfig to avoid that error
>
> # workaround to avoid the error "The C compiler identification is unknown"
> prebuildopts = "unset CC && unset CXX && "

Don't you fall back to system gcc than?

Ward

Re: [easybuild] easyconfig list

[Ward Poelmans]

Not really, the scripts to generate that list are part of the
framework, I think?

You can use the --search option, as Pable said. With
--list-easyblocks, you can get a list of all easyblocks available.

Ward

On Mon, Jul 14, 2014 at 1:18 PM,  <ahmed.sa...@stfc.ac.uk> wrote:
> Yup.
>
> Is there option to pass to eb?
>
> Ahmed.
>
> -Original Message-
> From: Ward Poelmans [mailto:ward.poelm...@ugent.be]
> Sent: 14 July 2014 12:18
> To: easybuild@lists.ugent.be
> Subject: Re: [easybuild] easyconfig list
>
> Hi Ahmed,
>
> Do you mean this list?
> https://github.com/hpcugent/easybuild/wiki/List-of-supported-software-packages
>
> Ward
>
> On Mon, Jul 14, 2014 at 1:15 PM,  <ahmed.sa...@stfc.ac.uk> wrote:
>> Hi,
>>
>> I'm sure this has been answered before and is documented somewhere but
>> I cant find how to list the easyconfig file list.
>>
>> Regards,
>> Ahmed.
>
>
>
> --
> ir. Ward Poelmans
> Center for Molecular Modeling
> Ghent University
> Technologiepark 903,
> B-9052 Zwijnaarde
> Belgium
> Tel: +32 9 264 65 76
> E-mail: ward.poelm...@ugent.be
> http://molmod.UGent.be/



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] easyconfig list

[Ward Poelmans]

Hi Ahmed,

Do you mean this list?
https://github.com/hpcugent/easybuild/wiki/List-of-supported-software-packages

Ward

On Mon, Jul 14, 2014 at 1:15 PM,  <ahmed.sa...@stfc.ac.uk> wrote:
> Hi,
>
> I'm sure this has been answered before and is documented somewhere but I cant
> find how to list the easyconfig file list.
>
> Regards,
> Ahmed.



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] Request pull for 1000+ easyconfigs or --try-toolchain???

[Ward Poelmans]

On Sat, Jul 12, 2014 at 4:45 PM, Jack Perdue <j-per...@tamu.edu> wrote:
> Is there any reason for me to issue a pull
> request(s) for the attached?  I tried to
> break it down into little pieces, but even
> then, it seems to be a lot.

I don't think so. If all these easyconfigs are just old ones with a
newer toolchain version, then people can better use --try-toolchain.

> Can we get a bot to do easyconfig toolchain
> bumps for us?

No, but the easybuild format v2 should handle this. There we can
specify ranges for versions. I think instead of trying to process this
huge PR, it would be better to redirect our efforts to ECv2.

Ward


-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] A patch containing a new file

[Ward Poelmans]

On Fri, Apr 4, 2014 at 3:03 PM, Kenneth Hoste  wrote:
> prebuildopts = ["cd hmm && ", "cd phmm && "]

s/prebuildopts/premakeopts/ everywhere ;-)

We should rename that one.

Ward

Re: [easybuild] Install problem

[Ward Poelmans]

On Mon, Mar 31, 2014 at 11:36 PM, Bart Verleye  wrote:
>
> Is this something in the eb file that is wrong, or a small error in EB?

Hi,

I don't see any errors in your ebconfig file. Can you send (or
pastebin/github gist) the full debug output of the build?

Ward

Re: [easybuild] GCC

[Ward Poelmans]

Hi Keith,

Easybuild has all the easyconfigs (and easyblocks) needed to build
everything for you. All you have to do is:
eb -r -l GCC-4.8.2.eb
And watch how gcc gets build for you ;-)

However, easybuild requires a full toolchain (include mpi, blas/lapack,
...). With `eb --list-toolchains` you get a list of all supported
toolchains and what's in them. Pick one, and then look for the easyconfig
file for it and run it.

Ward



On Fri, Mar 7, 2014 at 5:32 PM, <keith.r.hostetler...@nd.edu> wrote:

> So I am trying to build a toolchain for GCC 1.7.3 and have run into an
> error I
> have yet to see.
>
> ebFile:
> name = 'gcc'
> version = '4.7.3'
>
> homepage = 'http://gcc.gnu.org/'
> description = "abc"
>
> toolchain = {'name': 'dummy', 'version': 'dummy'}
> toolchainopts = {'shared': True, 'static': True}
>
> sources = ['gcc-4.7.3.tar.gz']
>
> moduleclass = 'compiler'
>
>
> Console Results:
> == temporary log file in case of crash /tmp/easybuild-0D9sGY/
> easybuild-3vIaW6.log
> == resolving dependencies ...
> == processing EasyBuild easyconfig /afs/crc.nd.edu/x86_64_linux/easybuild-
> soft/src/gcc/4.7.3/gcc-4.7.3.eb
> == ERROR: Failed to get application instance for gcc (easyblock: None):
> EasyBuild crashed with an error (at easybuild/framework/easyblock.py:1928
> in
> get_class): Failed to obtain class for None easyblock (not available?):
> 'EasyBuild crashed with an error (at easybuild/framework/easyblock.py:1897
> in
> get_class): Failed to import easyblock for EB_gcc because of module issue:
> '
>



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
Fax: +32 9 264 66 97
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/

Re: [easybuild] CMake built, but does not run

[Ward Poelmans]

On Mon, Feb 24, 2014 at 5:24 PM, Heywood, Todd  wrote:
> From: , Todd Heywood >
> Reply-To: "easybuild@lists.ugent.be" 
> >
> Date: Monday, February 24, 2014 at 11:08 AM
> To: "easybuild@lists.ugent.be" 
> >
> Subject: [easybuild] CMake built, but does not run
>
> Hi,
>
> I build CMake with the easyconfig CMake-2.8.12-goalf-1.5.12-no-OFED.eb 
> (unchanged).
>
> It build fine, but when users try to use it, we get this:
>
> CMake Error at 
> /opt/eb/software/CMake/2.8.12-goalf-1.5.12-no-OFED/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:108
>  (message):
>   Could NOT find LibConfig++ (missing: LIBCONFIG++_INCLUDE_DIR
>   LIBCONFIG++_LIBRARY)
> Call Stack (most recent call first):
>   
> /opt/eb/software/CMake/2.8.12-goalf-1.5.12-no-OFED/share/cmake-2.8/Modules/FindPackageHandleStandardArgs.cmake:315
>  (_FPHSA_FAILURE_MESSAGE)
>   cmake/Modules/FindLibConfig++.cmake:14 (find_package_handle_standard_args)
>   CMakeLists.txt:24 (find_package)

CMake needs some help in finding all libraries. Normally, with eb 1.11
this should happen automatically. Are you trying to build something
with easybuild and CMake or use CMake outside easybuild? In that
latter case, make sure CMAKE_INCLUDE_PATH and CMAKE_LIBRARY_PATH point
to all libraries you want to use. You can use the EBROOT* env vars to
do that.

Ward

Re: [easybuild] More than 1 admin

[Ward Poelmans]

A small addition:

If you use easybuild with multiple users on the same machine, you
might run into problems as the temporary storage is shared between all
easybuild processes. This is fixed in EB v1.10. Every easybuild
process now uses a unique directory in the temporary storage.

Ward

On Fri, Dec 20, 2013 at 1:43 AM, Bart Verleye <b.verl...@auckland.ac.nz> wrote:
> Hi All,
>
> Just before I go and enjoy a hot Christmas holiday, I would like to share
> how I set up easybuild here to use with more than 1 admin and different
> hardware. Perhaps people will pick up some ideas, or, perhaps I can learn
> from comments.
>
> We have following directories:
>
> /share/easybuild/installation: with the release version of EB installed.
> /share/easybuild/src: where the source files will be kept.
> /share/easybuild/"OS"/"Hardware": for the software and the module files
> /share/easybuild/git: a local respo of the develop (not an installation).
>
> When an admin wants to install software, (s)he goes to the correct build
> node, and sources the attached init file. This will set the environment.
>
> Easy!
>
> When the release version does not provide yet what we need, the
> easyblocks/easyconfigs in the git directory can be used, by setting the
> pythonpath.
>
> When local changes are made, e.g. paths to licences, this is done in the git
> directory, and pushed to a branch 'Pan', never to be PR'd to the main repo.
>
> When new easyconfig/blocks have to be created, that is done in a fork of the
> repo in the home directory of the admin. By contributing that work to the
> develop, it will also be available in /share/easybuild/git and eventually in
> the release version of EB.
>
> Once the problem with group write settings will be solved, I think this
> approach should work fine.
>
> There is also documentation is progress for this:
> https://wiki.auckland.ac.nz/display/CER/Copy+of+Installing+software+with+EB+on+Pan
>
> All comments on this approach are more than welcome!
>
> Einen guten Rutsch ins neue Jahr! As they say in German,
> Bart
>
>
>
> --
> Dr. Bart Verleye
> Centre for e-Research
> Level G, Room 409-G21
> 24 SYMONDS ST
> Auckland 1010
> New Zealand
> +64 (0) 9 923 9740 ext 89740



-- 
ir. Ward Poelmans
Center for Molecular Modeling
Ghent University
Technologiepark 903,
B-9052 Zwijnaarde
Belgium
Tel: +32 9 264 65 76
Fax: +32 9 264 66 97
E-mail: ward.poelm...@ugent.be
http://molmod.UGent.be/