Hi Stuart,
On 04/06/15 05:27, Stuart Barkley wrote:
We have a user desiring to use the Intel compilers and Intel MPI for a
performance bump in his (OpenFOAM based) code.
What has been peoples experience with these versus GCC/OpenMPI? Do
you see significant performance improvements?
In general, using Intel tools should yield better performance.
But, it highly depends on the application/libraries being used, the
system archictecture, etc.
I don't have any number for OpenFOAM, but we consistently build it with
Intel tools on our systems.
On the Intel web site, I see "Intel Parallel Studio XE 2015 (Cluster
edition)". Is this the software supported in the intel-2015a
toolchain?
Version numbers are very confusing with the Intel tools (and EB doesn't
make it any better...).
The 2015 versions of the Intel tools are currently only in the most
recent intel/ictce toolchains, i.e. intel/2015.02 and ictce/7.x .
intel/2015a was fixed end 2014, when the 2015 versions of the Intel
compilers was not available yet.
I'm curious about the differences between intel-2015a, intel-2015-02
and ictce-7.3.5 toolchains? Are these just for different Intel
releases (it seem to be mostly version numbers). All seem to be
referenced by recent software, but intel-2015a has the most referrers.
The intel-2015a toolchain seems to also depend upon GCC-4.9.3 (just
from reading the .eb files). Is there some reason for this?
For intel/2015a vs intel/2015.02, the only difference is versions of the
components.
For intel vs ictce, the difference lies in the fact that the intel
toolchains also include a dependency on GCC, as you point out.
The reason for this is to be independent from the GCC provided by the
system; this is particularly important for recent Intel versions w.r.t.
C++11 support, for example.
But it also helps with the robustness of the toolchains, since different
sites using the same version of intel toolchain are for sure using the
same version of GCC...
In the near future, we're going to take that one step further, and also
include binutils in the foss/intel toolchains, underneath GCC.
There are important use cases here as well, e.g. the standard binutils
on CentOS 6 is too old to support AVX2 (unless you use EPEL as an extra
repo), and so is an issue on Haswell systems.
The foss/2015a and intel/2015a toolchains are 'common' toolchains, i.e.
they are intended to be used by multiple sites, to collaborate more
easily on figuring out build issues, etc. That also explains why those
are significantly more common.
They are refreshed every 6 months. So, very soon, I'll open the
discussion on the next iteration foss/2015b and intel/2015b, which are
intended to be used during July-Dec 2015 (for the sites interested in
joining the 'common' toolchain effort).
As a side note: I've been converting my things to use the foss-2015a
toolchain (from goolf-1.4.10) and this appears to be working out. I
initially did a --try-toolchain build and most things seemed to build.
So far, I'm just cribbing off of existing .eb files and changing the
toolchain in these one-by-one and adding them to my easyconfig
directory. Is there really any other better way? I'm not looking
forward to future easybuild releases where I need to compare renamed
.eb files against the original ones to pick up significant changes.
Can you be more specific, e.g. give an example? I'm not sure what you
mean, since you say that --try-toolchain works out well?
When foss/2015b and intel/2015b, we will be including easyconfigs for
recent software versions, that are basically a copy of their 2015a
counterpart, but with updated versions of dependencies/extensions, etc.
Note that this can only work out well if multiple sites contribute back
their easyconfigs by making pull requests when they've looked into 2015b
easyconfigs.
If I should add the intel-2015a toolchain, I also really don't want to
create another couple dozen .eb files with just the toolchain change
(although this time it would be a little simpler rename/sed commands).
If you use --try-toolchain, there's no reason to rename/edit by hand (or
sed)?
Just make sure you use Intel-based easyconfigs, since they may contain
things specific to the Intel compilers to make the build work properly
(e.g. patch files, etc.).
And if you do, consider contributing them back, since then other people
can benefit from your efforts...
You're right that the explosion of easyconfigs w.r.t. toolchains and
versions is a major issue though.
Last year, there was an ongoing effort to come up with a better
easyconfig format (dubbed 'format v2') which intends to collapse all
easyconfig files for a particular software package in a single file,
including checks on version/toolchain to do specific things.
That effort has died since last summer though, since nobody has time to
work on it...
Having a single 'fat' easyconfig file, e.g. OpenFOAM.eb, also opens up
other issues, since you need a good way to specify which toolchain you
want to build with, which software version you want, and maybe also
which dependency versions you want to use, etc...
I have some ideas for that, and we can probably steal/borrow/use a great
deal of what Spack does for that, but I'm not sure I have a satisfying
answer yet (I'm sure Todd-the-Spack-guy will disagree ;-)).
This is a very good reason why joining forces between EB and Spack would
make a lot of sense actually...
I hope the effort is revived soon, since we're rapidly heading to 5k
easyconfig files included in EasyBuild, and many people are sitting on a
heap of easyconfigs themselves they need to deal with. The major issue
is time here...
regards,
Kenneth
