Hi Jack,
First of all: welcome to the wonderful world of EasyBuild! :-)
On 03/06/14 20:13, Jack Perdue wrote:
Howdy EBers,
Is there a formula/heuristic that y'all
use to determine the version numbers for
the toolchains?
Short answer: formula: no, heuristics: yes.
This isn't the first time this question has come up.
I remember explaining our approach once somewhere in detail, but I can't
seem to find it either,
so I took the opportunity to document this on the EasyBuild wiki, see
https://github.com/hpcugent/easybuild/wiki/Compiler-toolchains#note-on-toolchain-versions
.
Let me know if that clarifies things.
For example, where did
"6.2.5" come from for the latest ictce
toolchain?
It grew historically. The first ictce toolchain we used was ictce v3.x,
which features icc/ifort v11.x.
Don't ask me why '3', that was decided before 'my time'.
When the major version of the Intel compilers was bumped, we made the
ictce version follow:
ictce 4.x => icc/ifort 12.x
ictce 5.x => icc/ifort 13.x
ictce 6.x => icc/ifort 14.x
The '2' in '6.2.5' is just because this is the 3rd installment of ictce
v6.x.
The '5' is chosen such that downgrades of e.g. imkl/impi are possible if
needed/desired.
For other examples, what would be the
version numbers for the three toolchains
defined below?
ictce/A.B.C
icc - 2013_sp1.3.174 (new)
ifort - 2013_sp1.3.174 (new)
impi - 4.1.3.049 (same as ictce/6.2.5)
imkl - 11.1.3.174 (new)
Presuming icc/ifort 2013_sp1.3.174 provide v14.x of the compilers, it
should be the next installment of ictce 6.x, so probably v6.3.5.
gompi/X.Y.Z
GCC - 4.9.0 (new)
hwloc - 1.9 (new)
OpenMPI - 1.8.1 (new)
goolf/Q.R.S (maybe X.Y.Z???)
gompi - X.Y.Z (from above)
OpenBLAS - 0.2.8 (same as goolf/1.5.14)
FFTW - 3.3.4 (same as goolf/1.5.14)
ScaLAPACK - 2.0.2 (same as goolf/1.5.14)
Usually, we align the gompi and goolf versions (since that's convenient).
There's already a v1.5.14 (GCC 4.8.2, OpenMPI 1.6.5, ...) and v1.6.10
(GCC 4.8.2, OpenMPI 1.7.3, ...) so this being the next installment, it
should probably be v1.7.x .
One worry here might be that both GCC and OpenMPI are being bumped, so
v1.8.x or even v1.9.x would make sense too.
Especially the latter makes sense, since it gives the freedom to combine
GCC 4.8.x with OpenMPI v1.8 (gompi/goolf v1.7.x) and to combine GCC
4.9.x with OpenMPI v1.7 (gompi/goolf v1.8.x)...
So, I'd say, go with something like v1.9.6 (lower tiny number for
GCC/OpenMPI, so lower tiny version for the toolchain).
Is there any deterministic way to determine
what the version numbers above should be, or
is one supposed to just create an arbitrary
new one and then hope it doesn't conflict with
future EB releases?
That's probably clear by now: no, it's very ad hoc.
It basically boils down to the first person who opens a pull request for
an update of a particular toolchain 'wins' (provided the toolchain
version 'makes sense').
See https://github.com/hpcugent/easybuild/wiki/Contributing-back for
more information and
https://github.com/hpcugent/easybuild-easyconfigs/pull/764 for a recent
example (note that the toolchain version was changed to v1.5.14, since
that made more sense according to the heuristics we try to stick to).
We spent some time trying to come up with a deterministic scheme, but
gave up fairly quickly after noticing that coming up with a sensible
scheme that we would be able to stick to was very difficult...
I searched the wiki and mailing lists for clues,
but never found a good answer. So any hints
appreciated.
Thanks for taking the time to do that. More information on this
should've been available on the wiki already. And now it is! :-)
Thanks,
jack (learning EB in order to facilitate software
deployment on TAMU's new clusters)
Sounds great, let us know how that turns out!
One very good way to get fast answers to any questions you might have is
to pass by in our IRC channel.
I know that sounds so 1990's, but it actually (still) works really well.
See https://github.com/hpcugent/easybuild/wiki/Contact#irc for more info.
regards,
Kenneth
