Hi Bernd,
On 03/06/16 18:32, Bernd Mohr wrote:
Kenneth,
I had a symlink /opt/local/bin/lmod --> /opt/local/lmod/lmod/libexec/lmod
(in the same way as for lua).
After removing the symlink, the installation works now.
What is weird, is that this worked before: I just checked my other SuSE
installation with EB 2.0, 2.1 and 2.2 and there it was no problem.
I only had to make sure
that /opt/local/lmod/lmod/libexec was in $PATH before /opt/local/bin
Glad to hear we have it figured out, but I'm still puzzled...
i) Why does Lmod crash like that when it's being called via a symlink?
That must be some kind of regression with a recent Lmod version, which
may explain why you weren't seeing it before.
ii) Why does EasyBuild suddenly pick up 'lmod' via $PATH instead of
using $LMOD_CMD like it did before?
I checked the code, and the checking for 'lmod' is indeed done twice.
That is, the ModulesTool instance is creating twice
Once via main, where it is being used to check whether modules are
available, and then once for every software installation being performed.
For some reason unclear to me, the 1st time the 'lmod' found via $PATH
is the one in /opt/local/lmod/lmod/libexec, which is the same as the
path indicated by $LMOD_CMD, and things work fine.
The 2nd time however, it finds the 'lmod' in /opt/local/bin via $PATH,
making it ignore the path in $LMOD_CMD.
(the latter is mostly for backward compatibility, picking up $LMOD_CMD
if it's there and ignoring $PATH would be better, we can change that in
EasyBuild v3.0).
It seems that a 'module use' command being run throws out
/opt/local/lmod/lmod/libexec from $PATH for some reason (or, doesn't
include it again)...
How are you included /opt/local/lmod/lmod/libexec in $PATH exactly?
Any ideas why it would no longer be there after a 'module use'?
Anyway, seems like the real bug is in Lmod, I'll pick this up with
Robert. ;-)
regards,
Kenneth
Anyhow
Thanks!
Bernd
On Fri, Jun 03, 2016 at 06:21:06PM +0200, Kenneth Hoste wrote:
Hi Bernd,
Seems like there's a problem with your Lmod setup/installation:
== 2016-06-03 17:55:47,165 modules.py:613 DEBUG Output of module
command '/opt/local/bin/lmod python load EasyBuild/2.8.1': stdout: ;
stderr: /opt/local/lua/lua/bin/lua: /opt/local/bin/lmod:84: module
'myGlobals' not found:
no field package.preload['myGlobals']
no file '/opt/local/bin/myGlobals.lua'
no file '/opt/local/bin/../tools/myGlobals.lua'
no file '/opt/local/bin/../shells/myGlobals.lua'
no file '/opt/local/bin/myGlobals/init.lua'
no file './myGlobals.lua'
no file '/opt/local/lua/5.1.4.8/share/lua/5.1/myGlobals.lua'
no file '/opt/local/lua/5.1.4.8/share/lua/5.1/myGlobals/init.lua'
no file '/opt/local/lua/5.1.4.8/lib/lua/5.1/myGlobals.lua'
no file '/opt/local/lua/5.1.4.8/lib/lua/5.1/myGlobals/init.lua'
no file '/opt/local/bin/../lib/myGlobals.so'
no file './myGlobals.so'
no file '/opt/local/lua/5.1.4.8/lib/lua/5.1/myGlobals.so'
no file '/opt/local/lua/5.1.4.8/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'require'
/opt/local/bin/lmod:84: in main chunk
[C]: ?
The $PYTHONPATH setting I was grabbing from the log earlier isn't
the value that is active when the 'import easybuild' is being run.
The problem with the load is not detected by EasyBuild because Lmod
is exiting normally; usually it includes a 'false' in the stdout to
make the evaluation of the environment changes it spits out fail.
Some commands are running fine:
"/opt/local/lmod/lmod/libexec/lmod python --version" spits out
"Modules based on Lua: Version 6.4 2016-06-01 17:42"
"/opt/local/lmod/lmod/libexec/lmod python --terse list"
"/opt/local/lmod/lmod/libexec/lmod python --ters avail"
"/opt/local/lmod/lmod/libexec/lmod python show Core/EasyBuild/2.8.1"
=> all work fine (i.e. they don't crash)
But then suddenly "/opt/local/bin/lmod python --terse list" goes
down hard like above.
Note the change in location for 'lmod' itself...
Shortly before, the log is showing this:
== 2016-06-03 17:55:15,958 filetools.py:217 INFO Command lmod found
at /opt/local/bin/lmod
== 2016-06-03 17:55:15,958 filetools.py:217 INFO Command lmod found
at /opt/local/bin/lmod
== 2016-06-03 17:55:15,958 modules.py:179 DEBUG Different paths
found for module command 'lmod' via which/$PATH and $LMOD_CMD: lmod
vs /opt/local/lmod/lmod/libexec/lmod
== 2016-06-03 17:55:15,959 modules.py:185 DEBUG Using command lmod
== 2016-06-03 17:55:15,959 filetools.py:217 INFO Command lmod found
at /opt/local/bin/lmod
== 2016-06-03 17:55:15,959 modules.py:253 INFO Full path for module
command is /opt/local/bin/lmod, so using it
At some point EasyBuild switches from
/opt/local/lmod/lmod/libexec/lmod ($LMOD_CMD) to opt/local/bin/lmod
(via $PATH).
That's very stupid/wrong, I'll check on that, but it seems to
explain what's going on...
regards,
Kenneth
On 03/06/16 18:00, Bernd Mohr wrote:
debug log attached.
Executing after the failed installation:
% setenv PYTHONPATH
/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages
% python -c "import easybuild"
Also
% setenv PYTHONPATH
/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages:/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib64/python2.7/site-packages
% python -c "import easybuild"
no error / output
Bernd
On Fri, Jun 03, 2016 at 05:47:57PM +0200, Kenneth Hoste wrote:
On 03/06/16 17:41, Bernd Mohr wrote:
OK, so everything *is* there, but for some reason Python isn't
picking up on it. Strange...
You do have this, right?
ls -l
/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_framework-2.8.1-py2.7.egg/easybuild/__init__.py
Yes
Sorry, I meant:
cat
/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easy-install.pth
import sys; sys.__plen = len(sys.path)
./distribute-0.7.3-py2.7.egg
./setuptools-22.0.2-py2.7.egg
./vsc_install-0.10.6-py2.7.egg
./vsc_base-2.5.1-py2.7.egg
./easybuild_framework-2.8.1-py2.7.egg
./easybuild_easyblocks-2.8.1-py2.7.egg
./easybuild_easyconfigs-2.8.1-py2.7.egg
import sys; new = sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p =
getattr(sys, '__egginsert', 0); sys.path[p:p] = new; sys.__egginsert = p +
len(new)
Looks fine...
and let's throw in this too:
cat
/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/setuptools.pth
./setuptools-22.0.2-py2.7.egg
This is fine too.
Any PYTHONPATH issue?
I tried unsetenv PYTHONPATH but doid not change anything
From the log you provided, right before the "/usr/bin/python -c
'import easybuid' " that fails.
== 2016-06-03 16:53:19,262 environment.py:97 INFO Environment
variable PYTHONPATH set to
/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages:/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib64/python2.7/site-packages
(previous value: ...)
So, looks fine. I have no idea why the import fails...
OK, let's dig a little deeper.
Can you rebootstrap with debugging enabled, and send back the full log?
export EASYBUILD_BOOTSTRAP_DEBUG=1
python bootstrap_eb.py /opt/local/easybuild-2.8.1 2>&1 | tee
eb-debug.log
Also, let's try this (after the failing bootstrap):
PYTHONPATH=/opt/local/easybuild-2.8.1/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages
python -c "import easybuild"
regards,
Kenneth
Bernd
regards,
Kenneth
;-)
Bernd
regards,
Kenneth
On 03/06/16 17:00, Bernd Mohr wrote:
Fresh install with
% ./bootstrap_eb.py /opt/local/easybuild-2.8.1
on a fresh (empty) SuSE LEAP 42.1 system.
% /usr/bin/python --version
Python 2.7.9
Failure in Stage2:
+++ STAGE 2: installing EasyBuild in /opt/local/easybuild-2.8.1 with EasyBuild
from stage 1...
== temporary log file in case of crash /tmp/eb-ZufCws/easybuild-bC72PS.log
== processing EasyBuild easyconfig /tmp/tmpnlyQGk/EasyBuild-2.8.1.eb
== building and installing Core/EasyBuild/2.8.1...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== patching...
== preparing...
== configuring...
== building...
== testing...
== installing...
== taking care of extensions...
== postprocessing...
== sanity checking...
== FAILED: Installation ended unsuccessfully (build directory:
/tmp/tmpnlyQGk/EasyBuild/2.8.1/dummy-dummy): build failed (first 300 chars): Sanity check
failed: EasyBuild failed to install, cmd '/usr/bin/python -c "import
easybuild"' (stdin: None) output: Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named easybuild
, sanity check command eb --version exited with code 1 (output: /usr/bi
Any ideas?
Full installation log attached.
Bernd
--
Dr.-Ing. Bernd Mohr
Juelich Supercomputing Centre
Institute for Advanced Simulation
E-Mail: [email protected]
WWW:
http://www.fz-juelich.de/SharedDocs/Personen/IAS/JSC/EN/staff/mohr_b.html
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
