On 08 Jun 2016, at 10:57, Kenneth Hoste <[email protected]
<mailto:[email protected]>> wrote:
Hi Joachim,
On 08/06/16 10:44, Joachim Hein wrote:
Hi Kenneth,
A bit more feedback - I am not sure action is actually needed.
After upgrading to easybuild 2.8.1 I needed to run the vtune build
with a force flag since the existence of the corrupt vtune module of
the earlier build attempts made easybuild believe it was there.
That is to be expected I think.
Yes, if EasyBuild sees the module, it will not reinstall by default,
that's exactly what --force is for.
When running with force, I figured easybuild invokes the installer
supplied by intel, which would not overwrite the old install from
the earlier build. Easybuild figures that the installer doesn’t
quite succeed and does not progress to the next steps (e.g.
checking, module file). I sorted myself with an rm -rf on the “old"
installation.
If you use --force, EasyBuild should wipe the installation directory
before getting to the install script supplied by Intel...
Are you saying that didn't work as expected? If so, can you provide
more details on what went wrong exactly?
So far I assumed, it just runs the installer again. So didn’t expect
it a break. Below is a cut-n-past of a log of a failed building attempt:
more /tmp/eb-4Q2fHI/easybuild-VTune-2016_update3-20160607.155340.dvTAA.log
== 2016-06-07 15:53:40,058 easyblock.py:255 INFO This is EasyBuild
2.8.1 (framework: 2.8.1, easyblocks: 2.8.1) on host aurora1.
== 2016-06-07 15:53:40,058 easyblock.py:261 INFO This is easyblock
EB_VTune from module easybuild.easyblocks.vtune
(/sw/easybuild/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_easyblocks-2
.8.1-py2.7.egg/easybuild/easyblocks/v/vtune.pyo)
== 2016-06-07 15:53:40,059 easyblock.py:746 INFO Build dir set to
/local/easybuild/build/VTune/2016_update3/dummy-dummy
== 2016-06-07 15:53:40,059 easyblock.py:780 INFO Software install dir
set to /sw/easybuild/software/Core/VTune/2016_update3
== 2016-06-07 15:53:40,059 easyblock.py:785 INFO Module install dir
set to /sw/easybuild/modules/all
== 2016-06-07 15:53:40,060 easyblock.py:240 INFO Init completed for
application name VTune version 2016_update3
== 2016-06-07 15:53:40,060 easyblock.py:2336 INFO Obtained application
instance of for VTune (easyblock: None)
== 2016-06-07 15:53:40,060 build_log.py:227 INFO building and
installing Core/VTune/2016_update3...
== 2016-06-07 15:53:40,061 build_log.py:227 INFO fetching files...
== 2016-06-07 15:53:40,061 easyblock.py:2130 INFO Starting fetch step
== 2016-06-07 15:53:40,063 easyblock.py:2133 INFO Running method
fetch_step part of step fetch
== 2016-06-07 15:53:40,063 easyblock.py:1408 WARNING Easyconfig does
not specify an EasyBuild-version (key 'easybuild_version')! Assuming
the latest version
== 2016-06-07 15:53:40,064 filetools.py:217 INFO Command eb found at
/sw/easybuild/software/Core/EasyBuild/2.8.1/bin/eb
== 2016-06-07 15:53:40,066 easyblock.py:584 INFO Found file
vtune_amplifier_xe_2016_update3.tar.gz at
/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz
== 2016-06-07 15:53:40,066 easyblock.py:344 INFO Added sources:
[{'path':
'/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz',
'cmd': None, 'finalpath': '/local/easybuild/bui
ld/VTune/2016_update3/dummy-dummy', 'name':
'vtune_amplifier_xe_2016_update3.tar.gz', 'checksum': None}]
== 2016-06-07 15:53:40,066 easyblock.py:1449 INFO no patches provided
== 2016-06-07 15:53:44,174 easyblock.py:1458 INFO md5 checksum for
/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz:
ccd9125bcb0ef3a99147acf3f785e8c5
== 2016-06-07 15:53:44,174 easyblock.py:1475 INFO Checking dirs that
need to be created: ['/sw/easybuild/software/Core/VTune/2016_update3',
'/sw/easybuild/modules/all/Core/VTune']
== 2016-06-07 15:53:44,175 build_log.py:227 INFO creating build dir,
resetting environment...
== 2016-06-07 15:53:44,176 easyblock.py:2130 INFO Starting ready step
== 2016-06-07 15:53:44,179 easyblock.py:2133 INFO Running method
check_readiness_step part of step ready
== 2016-06-07 15:53:44,191 easyblock.py:1372 INFO Setting parallelism: 40
== 2016-06-07 15:53:44,282 easyblock.py:1377 WARNING Loaded modules
detected: ['lmod/6.0.24', 'EasyBuild/2.8.1']
== 2016-06-07 15:53:44,283 easyblock.py:2133 INFO Running method
make_builddir part of step ready
== 2016-06-07 15:53:44,284 filetools.py:905 INFO Creating directory
/local/easybuild/build/VTune/2016_update3/dummy-dummy (parents: True,
set_gid: False, sticky: False)
== 2016-06-07 15:53:44,284 easyblock.py:2133 INFO Running method
env_reset_changes part of step ready
== 2016-06-07 15:53:44,284 easyblock.py:2133 INFO Running method
handle_iterate_opts part of step ready
== 2016-06-07 15:53:44,285 build_log.py:227 INFO unpacking...
== 2016-06-07 15:53:44,285 easyblock.py:2130 INFO Starting source step
== 2016-06-07 15:53:44,287 easyblock.py:2133 INFO Running method
checksum_step part of step source
== 2016-06-07 15:53:44,288 easyblock.py:1491 INFO Checksum
verification for
/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz
using None passed.
== 2016-06-07 15:53:44,288 easyblock.py:2133 INFO Running method
extract_step part of step source
== 2016-06-07 15:53:44,288 easyblock.py:1498 INFO Unpacking source
vtune_amplifier_xe_2016_update3.tar.gz
== 2016-06-07 15:53:48,842 build_log.py:227 INFO patching...
== 2016-06-07 15:53:48,842 easyblock.py:2130 INFO Starting patch step
== 2016-06-07 15:53:48,845 easyblock.py:2133 INFO Running method
patch_step part of step patch
== 2016-06-07 15:53:48,846 build_log.py:227 INFO preparing...
== 2016-06-07 15:53:48,846 easyblock.py:2130 INFO Starting prepare step
== 2016-06-07 15:53:48,849 easyblock.py:2133 INFO Running method
prepare_step part of step prepare
== 2016-06-07 15:53:48,849 toolchain.py:537 INFO prepare: toolchain
dummy mode, dummy version; not loading dependencies
== 2016-06-07 15:53:48,849 easyblock.py:1309 INFO Using
/local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/
as start dir
== 2016-06-07 15:53:48,850 build_log.py:227 INFO configuring...
== 2016-06-07 15:53:48,850 easyblock.py:2130 INFO Starting configure step
== 2016-06-07 15:53:48,853 easyblock.py:2133 INFO Running method
configure_step part of step configure
== 2016-06-07 15:53:48,854 filetools.py:1276 INFO Considering
INTEL_LICENSE_FILE to find FlexLM license specs:
['[email protected] <mailto:[email protected]>']
== 2016-06-07 15:53:48,855 filetools.py:1312 INFO Found valid license
specs via $INTEL_LICENSE_FILE: ['[email protected]
<mailto:[email protected]>']
== 2016-06-07 15:53:48,855 intelbase.py:220 INFO Using Intel license
specifications from $INTEL_LICENSE_FILE: ['[email protected]
<mailto:[email protected]>']
== 2016-06-07 15:53:48,856 environment.py:97 INFO Environment variable
INTEL_LICENSE_FILE set to [email protected]
<mailto:[email protected]> (previous value:
'[email protected] <mailto:[email protected]>')
== 2016-06-07 15:53:48,856 build_log.py:227 INFO building...
== 2016-06-07 15:53:48,856 easyblock.py:2130 INFO Starting build step
== 2016-06-07 15:53:48,859 easyblock.py:2133 INFO Running method
build_step part of step build
== 2016-06-07 15:53:48,859 build_log.py:227 INFO testing...
== 2016-06-07 15:53:48,859 easyblock.py:2130 INFO Starting test step
== 2016-06-07 15:53:48,862 easyblock.py:2133 INFO Running method
test_step part of step test
== 2016-06-07 15:53:48,862 build_log.py:227 INFO installing...
== 2016-06-07 15:53:48,863 easyblock.py:2130 INFO Starting install step
== 2016-06-07 15:53:48,865 easyblock.py:2133 INFO Running method
stage_install_step part of step install
== 2016-06-07 15:53:48,865 easyblock.py:2133 INFO Running method
make_installdir part of step install
== 2016-06-07 15:53:48,866 easyblock.py:2133 INFO Running method
install_step part of step install
== 2016-06-07 15:53:49,275 environment.py:97 INFO Environment variable
LOCAL_INSTALL_VERBOSE set to 1 (previously undefined)
== 2016-06-07 15:53:49,275 environment.py:97 INFO Environment variable
VERBOSE_MODE set to 1 (previously undefined)
== 2016-06-07 15:53:49,275 environment.py:97 INFO Environment variable
INSTALL_PATH set to /sw/easybuild/software/Core/VTune/2016_update3
(previously undefined)
== 2016-06-07 15:54:02,444 build_log.py:152 ERROR cmd "./install.sh
-s
/local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg"
exited with exitcode 2 and output:
WARNING: Destination directory already exists.
Installing to this directory could overwrite existing files and
folders within
this directory.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
(at
easybuild/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_framework-2.8.1-py2.7.egg/easybuild/tools/run.py:397
in parse_cmd_output)
== 2016-06-07 15:54:02,444 easyblock.py:2362 WARNING build failed
(first 300 chars): cmd "./install.sh -s
/local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg"
exite
d with exitcode 2 and output:
WARNING: Destination directory already exists.
Installing to this directory could overwrite existing files and
folders within
this directory.
--
== 2016-06-07 15:54:02,444 easyblock.py:270 INFO Closing log for
application name VTune version 2016_update3
Using
rm -rf /sw/easybuild/software/Core/VTune/2016_update3
which is the old and new installation directory holding the binaries,
libs, etc. followed by
eb VTune-2016_update3.eb --robot --force
produced a new installation, incl module file.
The user reported that there are issues, but I am not sure whether
this is an EB issue or something else. If I think it is EB, I will let
you know.
Best wishes
Joachim
As far as I am concerned that is fine with me. I just assumed that
the “force” option is not heavily used. I think users of EB need to
be aware, that depending on the features of the underlying install
mechanism, the EB “force” option may not work. Whether it is worth
figuring, I severely doubt.
There are situations in which --force is not reliable, but they
should be rare...
I used it quite often when testing pull requests, for example.
if you want it to work better: Is there EB option for removing a
piece of software (module and install), which I have so far not
figured? That might be a way around any potential issues. If you
issue force: EB will delete the old install and create a new one.
Though that might create another can of worms that installs suddenly
lack prerequisites.
We don't have a --uninstall option, exactly because we don't have a
mechanism in place to make sure no dependencies are broken, i.e. we
don't have reverse dependency tracking. Yet, see also
https://github.com/hpcugent/easybuild-framework/issues/590 .
regards,
Kenneth
As I said, I am personally happy with the current state of affairs.
Best wishes
Joachim
On 07 Jun 2016, at 14:26, Joachim Hein <[email protected]> wrote:
Hi Kenneth,
Thanks for the quick reply. I had looked at the 2.8.1 release
notes and must have overlooked the vtune update in there. So I was
still on 2.8.0 (turning slightly read now) - thinking it was just a
minor update ...
EB 2.8.1 is currently testing.
Best wishes
Joachim
On 07 Jun 2016, at 12:50, Kenneth Hoste <[email protected]
<mailto:[email protected]>> wrote:
Hi Joachim,
The VTune easyblock was updated recently to be made aware of this,
see https://github.com/hpcugent/easybuild-easyblocks/pull/935 .
We also already provide an easyconfig file for VTune 2016 update 3
(in EasyBuild v2.8.1), cfr.
https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/v/VTune/VTune-2016_update3.eb
.
So, if you're using EasyBuild v2.8.1, you shouldn't be seeing this?
regards,
Kenneth
On 07/06/16 11:09, Joachim Hein wrote:
Hi,
I am trying to build vtune 2016 update 3 using the easyconfig
2013 update 10. There is this complication that there is now an
additional intermediate layer in the installation. For example
using a simple tweak of the 2013_update_10 config, binaries are
expected in (full path of our settings):
/sw/easybuild/software/Core/VTune/2016_update3/bin64
but they are actually installed in
/sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe_2016.3.0.463186/bin64/
with soft-links:
/sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe/bin64/
/sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe_2016/bin64/
So by modifying the config’s sanity_chack_paths arguments from:
18c18
< 'files': ["bin64/amplxe-runss", "bin64/amplxe-cl"],
---
> 'files': ["vtune_amplifier_xe_2016/bin64/amplxe-runss",
"vtune_amplifier_xe_2016/bin64/amplxe-cl"],
I should make it check ok, but that change is not communicated to
the module creation. As a results the PATH, LD_LIBARARY_PATH,
CPATH, etc are not created in the module file.
Looking into the easy block, there seems a lot of “auto-guessing”
going on. Is there an easy way (like the sanity_chack_path) to
make it guess in a different base directory? Is that documented
in the read the docs?
Thanks and best wishes
Joachim
