Hi Trey,
On 23/08/14 01:20, Trey Dockendorf wrote:
Kenneth,
Thanks for the response, see inline.
Do you have a full list of application, and which
compilers/MPI/BLAS/LAPACK you're aiming for?
Compilers:
- gcc-4.8.x
- intel-2013_SP1.3
MPI:
- MVAPICH2-2.0
- OpenMPI-1.8.1
BLAS/LAPACK:
- ACML 6.0.x
- OpenBLAS-0.2.10
- LAPACK-3.5.x
I don't know what category these go under as it was my colleague, who
understands these programs better than I, who built them:
* hdf5
* netcdf
* suitesparse-4.3.1
* octave-3.8.1
* petsc-3.5.1
EasyBuild supports all of these (although maybe not these recent
versions, cfr. ACML), except for Octave which is in the works (and has
been for a while), see
https://github.com/hpcugent/easybuild-easyconfigs/pull/196 and
https://github.com/hpcugent/easybuild-easyconfigs/pull/418.
Full list of what is currently in our Lmod instance:
http://www.brazos.tamu.edu/docs/apps.php
I recognize a lot of these as being supported by EasyBuild already,
while some others are new (e.g. towhee, tesseract, ior).
Are you using helper function to avoid having to copy-paste-adjust
largely similar blocks across Lua module files, or because some
functionality is missing in Lmod that you need?
What I'm really asking is: which helper functions would you still
need
if the (Lua) module files are generated automagically for you (e.g.
by
EasyBuild)?
It's both avoiding copy&paste as well as "missing" functionality. The functionality
isn't so much missing just that it's part of the "contrib" portion of Lmod and not part of
the core functionality.
Here's my SitePackage.lua -
https://gist.github.com/treydock/ca9db395f604f59c1ea5
All the functions could likely be replaced by EB's automagic except the
following:
logUsage & load_hook - Not yet used but on TODO list , As long as Lua files
were used and SitePackage loaded, these should work even with EB (if EB used Lua).
But that's my guess.
Yes, these are triggered by Lmod on some subcommands, the module files
are oblivious to them afaik.
setenv_ifunset - Useful as we set runtime variables like OMP_NUM_THREADS=1
unless a user has already set them, and warn user about the default being set
(so they can override if they wish).
That's a nice one. EasyBuild could be enhanced easily to support
something like this as well, by basically checking a condition before
doing a setenv.
That would be nice to have as a feature imho, and it's not specific to
Lua module files either.
EasyBuild won't allow you to install stuff as root, since it's
generally
a bad idea (who knows what those crazy scientists have come up with
in
their code ;-)).
Yes, that is too true and a precaution I need to adopt.
The Lua module files are a blocker for now, yes, but we're on that,
as
I've already mentioned to you. I'll bug Petar some more to get our
hands
on his work w.r.t. adding support for generating Lua module files...
Depending on how heavily you would still need your custom helper
functions, we may need to provide hooks that you can tap into to get
your site-specific tweaks included as well, but that shouldn't be too
hard (we already support stuff like --modules-footer that
'modtclfooter'
easyconfig parameter).
Excellent. By no means should you or Petar feel rushed to push anything
through on my account.
The likelihood I'll be able to keep pace and make use of any new features is
very low, which is no way a fault of EB.
Well, we want support for Lua modules files anyway, because of other
goodies (properties, families, etc.). Although they are strictly
speaking not tied to Lua module files, it makes a lot of sense to only
use those with Lua modules, since only Lmod knows about them.
We've been using a dedicated user for building/installing software,
which we share with multiple people. Which is a bad idea, for
practical
purposes. :-)
The proper way to do it is what you're suggesting: give users the
required permissions, and maybe also make them act under a common
'easybuild' (POSIX) group.
Shared accounts I can handle, but our University has strict policies against it. Of
course things like "root" are technically shared and their are rules we have to
follow for that too. I like the idea of groups, but seems a lot of 'make install'
scripts may set the permissions and override umask or I simply did not use the EB umask
feature correctly.
Hmm, if you notice unexpected behavior here, open an issue for it. I
don't think many people are actively using these settings, so maybe we
got something wrong there.
Consider using `/dev/shm` here.
That is a good idea, and something I overlooked when translating "old" ways
into EB.
There be dragons here with EasyBuild v1.14.0 (see
https://github.com/hpcugent/easybuild-framework/issues/1011). The
open
issues should hopefully be fixed by EasyBuild v1.15.0 (it's looking
promising, PRs are open for most of the issues, especially the most
important/tricky ones).
Excellent, I'll keep an eye out on that issue.
Check out --set-gid-bit, --sticky-bit and --group too.
Is there a way to make it so that everything I install is read/write by myself and my
group? I tried setting --group and that removed all permissions from "other".
I have --set-gid-bit and --sticky-bit also set with --umask=002 and everything is still
755 or 644.
If you use --group, you're specifying that the software should *only* be
available to people in that group, so it'll remove permissions from
'other', yes.
The functionality required to make everything read/write for user/group
is there, but there's no command line option yet to control that.
Right now, EasyBuild generally removes write permissions for
group/other, to avoid that nasty mistakes result in disasters (e.g.,
when you install software for a group of users, you don't want to give
them write permissions on the install dirs), see
https://github.com/hpcugent/easybuild-framework/blob/master/easybuild/framework/easyblock.py#L1458
.
I think you'd want something like --add-permissions=770, or something
along those lines?
regards,
Kenneth
