On 08 Jun 2016, at 10:57, Kenneth Hoste <[email protected]>
wrote:
Hi Joachim,
On 08/06/16 10:44, Joachim Hein wrote:
Hi Kenneth,
A bit more feedback - I am not sure action is actually needed.
After upgrading to easybuild 2.8.1 I needed to run the vtune
build with a force flag since the existence of the corrupt vtune
module of the earlier build attempts made easybuild believe it
was there. That is to be expected I think.
Yes, if EasyBuild sees the module, it will not reinstall by
default, that's exactly what --force is for.
When running with force, I figured easybuild invokes the
installer supplied by intel, which would not overwrite the old
install from the earlier build. Easybuild figures that the
installer doesn’t quite succeed and does not progress to the
next steps (e.g. checking, module file). I sorted myself with
an rm -rf on the “old" installation.
If you use --force, EasyBuild should wipe the installation
directory before getting to the install script supplied by Intel...
Are you saying that didn't work as expected? If so, can you
provide more details on what went wrong exactly?
So far I assumed, it just runs the installer again. So didn’t
expect it a break. Below is a cut-n-past of a log of a failed
building attempt:
more
/tmp/eb-4Q2fHI/easybuild-VTune-2016_update3-20160607.155340.dvTAA.log
== 2016-06-07 15:53:40,058 easyblock.py:255 INFO This is EasyBuild
2.8.1 (framework: 2.8.1, easyblocks: 2.8.1) on host aurora1.
== 2016-06-07 15:53:40,058 easyblock.py:261 INFO This is easyblock
EB_VTune from module easybuild.easyblocks.vtune
(/sw/easybuild/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_easyblocks-2
.8.1-py2.7.egg/easybuild/easyblocks/v/vtune.pyo)
== 2016-06-07 15:53:40,059 easyblock.py:746 INFO Build dir set to
/local/easybuild/build/VTune/2016_update3/dummy-dummy
== 2016-06-07 15:53:40,059 easyblock.py:780 INFO Software install
dir set to /sw/easybuild/software/Core/VTune/2016_update3
== 2016-06-07 15:53:40,059 easyblock.py:785 INFO Module install
dir set to /sw/easybuild/modules/all
== 2016-06-07 15:53:40,060 easyblock.py:240 INFO Init completed
for application name VTune version 2016_update3
== 2016-06-07 15:53:40,060 easyblock.py:2336 INFO Obtained
application instance of for VTune (easyblock: None)
== 2016-06-07 15:53:40,060 build_log.py:227 INFO building and
installing Core/VTune/2016_update3...
== 2016-06-07 15:53:40,061 build_log.py:227 INFO fetching files...
== 2016-06-07 15:53:40,061 easyblock.py:2130 INFO Starting fetch step
== 2016-06-07 15:53:40,063 easyblock.py:2133 INFO Running method
fetch_step part of step fetch
== 2016-06-07 15:53:40,063 easyblock.py:1408 WARNING Easyconfig
does not specify an EasyBuild-version (key 'easybuild_version')!
Assuming the latest version
== 2016-06-07 15:53:40,064 filetools.py:217 INFO Command eb found
at /sw/easybuild/software/Core/EasyBuild/2.8.1/bin/eb
== 2016-06-07 15:53:40,066 easyblock.py:584 INFO Found file
vtune_amplifier_xe_2016_update3.tar.gz at
/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz
== 2016-06-07 15:53:40,066 easyblock.py:344 INFO Added sources:
[{'path':
'/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz',
'cmd': None, 'finalpath': '/local/easybuild/bui
ld/VTune/2016_update3/dummy-dummy', 'name':
'vtune_amplifier_xe_2016_update3.tar.gz', 'checksum': None}]
== 2016-06-07 15:53:40,066 easyblock.py:1449 INFO no patches provided
== 2016-06-07 15:53:44,174 easyblock.py:1458 INFO md5 checksum for
/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz:
ccd9125bcb0ef3a99147acf3f785e8c5
== 2016-06-07 15:53:44,174 easyblock.py:1475 INFO Checking dirs
that need to be created:
['/sw/easybuild/software/Core/VTune/2016_update3',
'/sw/easybuild/modules/all/Core/VTune']
== 2016-06-07 15:53:44,175 build_log.py:227 INFO creating build
dir, resetting environment...
== 2016-06-07 15:53:44,176 easyblock.py:2130 INFO Starting ready step
== 2016-06-07 15:53:44,179 easyblock.py:2133 INFO Running method
check_readiness_step part of step ready
== 2016-06-07 15:53:44,191 easyblock.py:1372 INFO Setting
parallelism: 40
== 2016-06-07 15:53:44,282 easyblock.py:1377 WARNING Loaded
modules detected: ['lmod/6.0.24', 'EasyBuild/2.8.1']
== 2016-06-07 15:53:44,283 easyblock.py:2133 INFO Running method
make_builddir part of step ready
== 2016-06-07 15:53:44,284 filetools.py:905 INFO Creating
directory /local/easybuild/build/VTune/2016_update3/dummy-dummy
(parents: True, set_gid: False, sticky: False)
== 2016-06-07 15:53:44,284 easyblock.py:2133 INFO Running method
env_reset_changes part of step ready
== 2016-06-07 15:53:44,284 easyblock.py:2133 INFO Running method
handle_iterate_opts part of step ready
== 2016-06-07 15:53:44,285 build_log.py:227 INFO unpacking...
== 2016-06-07 15:53:44,285 easyblock.py:2130 INFO Starting source step
== 2016-06-07 15:53:44,287 easyblock.py:2133 INFO Running method
checksum_step part of step source
== 2016-06-07 15:53:44,288 easyblock.py:1491 INFO Checksum
verification for
/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz
using None passed.
== 2016-06-07 15:53:44,288 easyblock.py:2133 INFO Running method
extract_step part of step source
== 2016-06-07 15:53:44,288 easyblock.py:1498 INFO Unpacking source
vtune_amplifier_xe_2016_update3.tar.gz
== 2016-06-07 15:53:48,842 build_log.py:227 INFO patching...
== 2016-06-07 15:53:48,842 easyblock.py:2130 INFO Starting patch step
== 2016-06-07 15:53:48,845 easyblock.py:2133 INFO Running method
patch_step part of step patch
== 2016-06-07 15:53:48,846 build_log.py:227 INFO preparing...
== 2016-06-07 15:53:48,846 easyblock.py:2130 INFO Starting prepare
step
== 2016-06-07 15:53:48,849 easyblock.py:2133 INFO Running method
prepare_step part of step prepare
== 2016-06-07 15:53:48,849 toolchain.py:537 INFO prepare:
toolchain dummy mode, dummy version; not loading dependencies
== 2016-06-07 15:53:48,849 easyblock.py:1309 INFO Using
/local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/
as start dir
== 2016-06-07 15:53:48,850 build_log.py:227 INFO configuring...
== 2016-06-07 15:53:48,850 easyblock.py:2130 INFO Starting
configure step
== 2016-06-07 15:53:48,853 easyblock.py:2133 INFO Running method
configure_step part of step configure
== 2016-06-07 15:53:48,854 filetools.py:1276 INFO Considering
INTEL_LICENSE_FILE to find FlexLM license specs:
['[email protected] <mailto:[email protected]>']
== 2016-06-07 15:53:48,855 filetools.py:1312 INFO Found valid
license specs via $INTEL_LICENSE_FILE: ['[email protected]']
== 2016-06-07 15:53:48,855 intelbase.py:220 INFO Using Intel
license specifications from $INTEL_LICENSE_FILE:
['[email protected] <mailto:[email protected]>']
== 2016-06-07 15:53:48,856 environment.py:97 INFO Environment
variable INTEL_LICENSE_FILE set to [email protected]
<mailto:[email protected]> (previous value:
'[email protected] <mailto:[email protected]>')
== 2016-06-07 15:53:48,856 build_log.py:227 INFO building...
== 2016-06-07 15:53:48,856 easyblock.py:2130 INFO Starting build step
== 2016-06-07 15:53:48,859 easyblock.py:2133 INFO Running method
build_step part of step build
== 2016-06-07 15:53:48,859 build_log.py:227 INFO testing...
== 2016-06-07 15:53:48,859 easyblock.py:2130 INFO Starting test step
== 2016-06-07 15:53:48,862 easyblock.py:2133 INFO Running method
test_step part of step test
== 2016-06-07 15:53:48,862 build_log.py:227 INFO installing...
== 2016-06-07 15:53:48,863 easyblock.py:2130 INFO Starting install
step
== 2016-06-07 15:53:48,865 easyblock.py:2133 INFO Running method
stage_install_step part of step install
== 2016-06-07 15:53:48,865 easyblock.py:2133 INFO Running method
make_installdir part of step install
== 2016-06-07 15:53:48,866 easyblock.py:2133 INFO Running method
install_step part of step install
== 2016-06-07 15:53:49,275 environment.py:97 INFO Environment
variable LOCAL_INSTALL_VERBOSE set to 1 (previously undefined)
== 2016-06-07 15:53:49,275 environment.py:97 INFO Environment
variable VERBOSE_MODE set to 1 (previously undefined)
== 2016-06-07 15:53:49,275 environment.py:97 INFO Environment
variable INSTALL_PATH set to
/sw/easybuild/software/Core/VTune/2016_update3 (previously undefined)
== 2016-06-07 15:54:02,444 build_log.py:152 ERROR cmd
"./install.sh -s
/local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg"
exited with exitcode 2 and output:
WARNING: Destination directory already exists.
Installing to this directory could overwrite existing files and
folders within
this directory.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
(at
easybuild/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_framework-2.8.1-py2.7.egg/easybuild/tools/run.py:397
in parse_cmd_output)
== 2016-06-07 15:54:02,444 easyblock.py:2362 WARNING build failed
(first 300 chars): cmd "./install.sh -s
/local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg"
exite
d with exitcode 2 and output:
WARNING: Destination directory already exists.
Installing to this directory could overwrite existing files and
folders within
this directory.
--
== 2016-06-07 15:54:02,444 easyblock.py:270 INFO Closing log for
application name VTune version 2016_update3
Using
rm -rf /sw/easybuild/software/Core/VTune/2016_update3
which is the old and new installation directory holding the
binaries, libs, etc. followed by
eb VTune-2016_update3.eb --robot --force
produced a new installation, incl module file.
The user reported that there are issues, but I am not sure whether
this is an EB issue or something else. If I think it is EB, I will
let you know.
Best wishes
Joachim
As far as I am concerned that is fine with me. I just assumed
that the “force” option is not heavily used. I think users of
EB need to be aware, that depending on the features of the
underlying install mechanism, the EB “force” option may not
work. Whether it is worth figuring, I severely doubt.
There are situations in which --force is not reliable, but they
should be rare...
I used it quite often when testing pull requests, for example.
if you want it to work better: Is there EB option for removing a
piece of software (module and install), which I have so far not
figured? That might be a way around any potential issues. If
you issue force: EB will delete the old install and create a new
one. Though that might create another can of worms that installs
suddenly lack prerequisites.
We don't have a --uninstall option, exactly because we don't have
a mechanism in place to make sure no dependencies are broken,
i.e. we don't have reverse dependency tracking. Yet, see also
https://github.com/hpcugent/easybuild-framework/issues/590 .
regards,
Kenneth
As I said, I am personally happy with the current state of affairs.
Best wishes
Joachim
On 07 Jun 2016, at 14:26, Joachim Hein
<[email protected]> wrote:
Hi Kenneth,
Thanks for the quick reply. I had looked at the 2.8.1 release
notes and must have overlooked the vtune update in there. So I
was still on 2.8.0 (turning slightly read now) - thinking it
was just a minor update ...
EB 2.8.1 is currently testing.
Best wishes
Joachim
On 07 Jun 2016, at 12:50, Kenneth Hoste
<[email protected]> wrote:
Hi Joachim,
The VTune easyblock was updated recently to be made aware of
this, see
https://github.com/hpcugent/easybuild-easyblocks/pull/935 .
We also already provide an easyconfig file for VTune 2016
update 3 (in EasyBuild v2.8.1), cfr.
https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/v/VTune/VTune-2016_update3.eb
.
So, if you're using EasyBuild v2.8.1, you shouldn't be seeing
this?
regards,
Kenneth
On 07/06/16 11:09, Joachim Hein wrote:
Hi,
I am trying to build vtune 2016 update 3 using the easyconfig
2013 update 10. There is this complication that there is now
an additional intermediate layer in the installation. For
example using a simple tweak of the 2013_update_10 config,
binaries are expected in (full path of our settings):
/sw/easybuild/software/Core/VTune/2016_update3/bin64
but they are actually installed in
/sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe_2016.3.0.463186/bin64/
with soft-links:
/sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe/bin64/
/sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe_2016/bin64/
So by modifying the config’s sanity_chack_paths arguments from:
18c18
< 'files': ["bin64/amplxe-runss", "bin64/amplxe-cl"],
---
> 'files': ["vtune_amplifier_xe_2016/bin64/amplxe-runss",
"vtune_amplifier_xe_2016/bin64/amplxe-cl"],
I should make it check ok, but that change is not
communicated to the module creation. As a results the PATH,
LD_LIBARARY_PATH, CPATH, etc are not created in the module file.
Looking into the easy block, there seems a lot of
“auto-guessing” going on. Is there an easy way (like the
sanity_chack_path) to make it guess in a different base
directory? Is that documented in the read the docs?
Thanks and best wishes
Joachim
