Hi Kenneth, Didn’t expect much out of this. So below is a cutting from the log:
== 2016-06-08 11:40:31,546 templates.py:173 DEBUG version found in easyconfig is 2016_update3 == 2016-06-08 11:40:31,546 templates.py:173 DEBUG version found in easyconfig is 2016_update3 == 2016-06-08 11:40:31,546 templates.py:210 DEBUG name: name, config: VTune == 2016-06-08 11:40:31,546 templates.py:210 DEBUG name: version, config: 2016_update3 == 2016-06-08 11:40:31,547 templates.py:210 DEBUG name: versionsuffix, config: == 2016-06-08 11:40:31,547 templates.py:210 DEBUG name: versionprefix, config: == 2016-06-08 11:40:31,547 easyblock.py:2133 INFO Running method stage_install_step part of step install == 2016-06-08 11:40:31,547 easyblock.py:2133 INFO Running method make_installdir part of step install == 2016-06-08 11:40:31,547 easyblock.py:2133 INFO Running method install_step part of step install == 2016-06-08 11:40:31,548 intelbase.py:315 DEBUG Contents of /local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg: ACTIVATION_TYPE=license_server [email protected]<mailto:[email protected]> PSET_INSTALL_DIR=/sw/easybuild/software/Core/VTune/2016_update3 ACCEPT_EULA=accept INSTALL_MODE=NONRPM CONTINUE_WITH_OPTIONAL_ERROR=yes == 2016-06-08 11:40:31,548 environment.py:97 INFO Environment variable LOCAL_INSTALL_VERBOSE set to 1 (previously undefined) == 2016-06-08 11:40:31,548 environment.py:97 INFO Environment variable VERBOSE_MODE set to 1 (previously undefined) == 2016-06-08 11:40:31,549 environment.py:97 INFO Environment variable INSTALL_PATH set to /sw/easybuild/software/Core/VTune/2016_update3 (previously undefined) == 2016-06-08 11:40:31,549 run.py:99 DEBUG run_cmd: running cmd ./install.sh -s /local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg (in /local/easybuild/build /VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3) == 2016-06-08 11:40:31,549 run.py:107 DEBUG run_cmd: Command output will be logged to /tmp/eb-c_vKEE/easybuild-run_cmd-N5ooC7.log == 2016-06-08 11:40:42,289 run.py:392 DEBUG cmd "./install.sh -s /local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg" exited with exitcode 2 and output: WARNING: Destination directory already exists. Installing to this directory could overwrite existing files and folders within this directory. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- That seems to be the relevant portion of the log file you are interested in (generated with eb VTune-2016_update3.eb --robot --force —debug) The disk is nfs mounted. In the past I had often good success when putting a UNIX ‘ls’ into the script, when it failed due to outdated cache. Perhaps it is an idea to put an ‘ls' on the just deleted directory or the one below to force cache updates. Best wishes Joachim On 08 Jun 2016, at 11:25, Kenneth Hoste <[email protected]<mailto:[email protected]>> wrote: Hi Joachim, Can you rerun this with debugging enabled (eb --debug ...)? Shortly after the log line "Running method make_installdir part of step install", you should see that the install dir is effectively cleaned up by EasyBuild. What may be happening though is that the directory is still being seeing by the Intel install script even though it's no longer there due to a caching effect... Are you installing things to an NFS mount? regards, Kenneth On 08/06/16 11:12, Joachim Hein wrote: On 08 Jun 2016, at 10:57, Kenneth Hoste <<mailto:[email protected]>[email protected]<mailto:[email protected]>> wrote: Hi Joachim, On 08/06/16 10:44, Joachim Hein wrote: Hi Kenneth, A bit more feedback - I am not sure action is actually needed. After upgrading to easybuild 2.8.1 I needed to run the vtune build with a force flag since the existence of the corrupt vtune module of the earlier build attempts made easybuild believe it was there. That is to be expected I think. Yes, if EasyBuild sees the module, it will not reinstall by default, that's exactly what --force is for. When running with force, I figured easybuild invokes the installer supplied by intel, which would not overwrite the old install from the earlier build. Easybuild figures that the installer doesn’t quite succeed and does not progress to the next steps (e.g. checking, module file). I sorted myself with an rm -rf on the “old" installation. If you use --force, EasyBuild should wipe the installation directory before getting to the install script supplied by Intel... Are you saying that didn't work as expected? If so, can you provide more details on what went wrong exactly? So far I assumed, it just runs the installer again. So didn’t expect it a break. Below is a cut-n-past of a log of a failed building attempt: more /tmp/eb-4Q2fHI/easybuild-VTune-2016_update3-20160607.155340.dvTAA.log == 2016-06-07 15:53:40,058 easyblock.py:255 INFO This is EasyBuild 2.8.1 (framework: 2.8.1, easyblocks: 2.8.1) on host aurora1. == 2016-06-07 15:53:40,058 easyblock.py:261 INFO This is easyblock EB_VTune from module easybuild.easyblocks.vtune (/sw/easybuild/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_easyblocks-2 .8.1-py2.7.egg/easybuild/easyblocks/v/vtune.pyo) == 2016-06-07 15:53:40,059 easyblock.py:746 INFO Build dir set to /local/easybuild/build/VTune/2016_update3/dummy-dummy == 2016-06-07 15:53:40,059 easyblock.py:780 INFO Software install dir set to /sw/easybuild/software/Core/VTune/2016_update3 == 2016-06-07 15:53:40,059 easyblock.py:785 INFO Module install dir set to /sw/easybuild/modules/all == 2016-06-07 15:53:40,060 easyblock.py:240 INFO Init completed for application name VTune version 2016_update3 == 2016-06-07 15:53:40,060 easyblock.py:2336 INFO Obtained application instance of for VTune (easyblock: None) == 2016-06-07 15:53:40,060 build_log.py:227 INFO building and installing Core/VTune/2016_update3... == 2016-06-07 15:53:40,061 build_log.py:227 INFO fetching files... == 2016-06-07 15:53:40,061 easyblock.py:2130 INFO Starting fetch step == 2016-06-07 15:53:40,063 easyblock.py:2133 INFO Running method fetch_step part of step fetch == 2016-06-07 15:53:40,063 easyblock.py:1408 WARNING Easyconfig does not specify an EasyBuild-version (key 'easybuild_version')! Assuming the latest version == 2016-06-07 15:53:40,064 filetools.py:217 INFO Command eb found at /sw/easybuild/software/Core/EasyBuild/2.8.1/bin/eb == 2016-06-07 15:53:40,066 easyblock.py:584 INFO Found file vtune_amplifier_xe_2016_update3.tar.gz at /home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz == 2016-06-07 15:53:40,066 easyblock.py:344 INFO Added sources: [{'path': '/home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz', 'cmd': None, 'finalpath': '/local/easybuild/bui ld/VTune/2016_update3/dummy-dummy', 'name': 'vtune_amplifier_xe_2016_update3.tar.gz', 'checksum': None}] == 2016-06-07 15:53:40,066 easyblock.py:1449 INFO no patches provided == 2016-06-07 15:53:44,174 easyblock.py:1458 INFO md5 checksum for /home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz: ccd9125bcb0ef3a99147acf3f785e8c5 == 2016-06-07 15:53:44,174 easyblock.py:1475 INFO Checking dirs that need to be created: ['/sw/easybuild/software/Core/VTune/2016_update3', '/sw/easybuild/modules/all/Core/VTune'] == 2016-06-07 15:53:44,175 build_log.py:227 INFO creating build dir, resetting environment... == 2016-06-07 15:53:44,176 easyblock.py:2130 INFO Starting ready step == 2016-06-07 15:53:44,179 easyblock.py:2133 INFO Running method check_readiness_step part of step ready == 2016-06-07 15:53:44,191 easyblock.py:1372 INFO Setting parallelism: 40 == 2016-06-07 15:53:44,282 easyblock.py:1377 WARNING Loaded modules detected: ['lmod/6.0.24', 'EasyBuild/2.8.1'] == 2016-06-07 15:53:44,283 easyblock.py:2133 INFO Running method make_builddir part of step ready == 2016-06-07 15:53:44,284 filetools.py:905 INFO Creating directory /local/easybuild/build/VTune/2016_update3/dummy-dummy (parents: True, set_gid: False, sticky: False) == 2016-06-07 15:53:44,284 easyblock.py:2133 INFO Running method env_reset_changes part of step ready == 2016-06-07 15:53:44,284 easyblock.py:2133 INFO Running method handle_iterate_opts part of step ready == 2016-06-07 15:53:44,285 build_log.py:227 INFO unpacking... == 2016-06-07 15:53:44,285 easyblock.py:2130 INFO Starting source step == 2016-06-07 15:53:44,287 easyblock.py:2133 INFO Running method checksum_step part of step source == 2016-06-07 15:53:44,288 easyblock.py:1491 INFO Checksum verification for /home/swadmin/.local/easybuild/sources/v/VTune/vtune_amplifier_xe_2016_update3.tar.gz using None passed. == 2016-06-07 15:53:44,288 easyblock.py:2133 INFO Running method extract_step part of step source == 2016-06-07 15:53:44,288 easyblock.py:1498 INFO Unpacking source vtune_amplifier_xe_2016_update3.tar.gz == 2016-06-07 15:53:48,842 build_log.py:227 INFO patching... == 2016-06-07 15:53:48,842 easyblock.py:2130 INFO Starting patch step == 2016-06-07 15:53:48,845 easyblock.py:2133 INFO Running method patch_step part of step patch == 2016-06-07 15:53:48,846 build_log.py:227 INFO preparing... == 2016-06-07 15:53:48,846 easyblock.py:2130 INFO Starting prepare step == 2016-06-07 15:53:48,849 easyblock.py:2133 INFO Running method prepare_step part of step prepare == 2016-06-07 15:53:48,849 toolchain.py:537 INFO prepare: toolchain dummy mode, dummy version; not loading dependencies == 2016-06-07 15:53:48,849 easyblock.py:1309 INFO Using /local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/ as start dir == 2016-06-07 15:53:48,850 build_log.py:227 INFO configuring... == 2016-06-07 15:53:48,850 easyblock.py:2130 INFO Starting configure step == 2016-06-07 15:53:48,853 easyblock.py:2133 INFO Running method configure_step part of step configure == 2016-06-07 15:53:48,854 filetools.py:1276 INFO Considering INTEL_LICENSE_FILE to find FlexLM license specs: ['[email protected]<mailto:[email protected]>'] == 2016-06-07 15:53:48,855 filetools.py:1312 INFO Found valid license specs via $INTEL_LICENSE_FILE: ['<mailto:[email protected]>[email protected]<mailto:[email protected]>'] == 2016-06-07 15:53:48,855 intelbase.py:220 INFO Using Intel license specifications from $INTEL_LICENSE_FILE: ['[email protected]<mailto:[email protected]>'] == 2016-06-07 15:53:48,856 environment.py:97 INFO Environment variable INTEL_LICENSE_FILE set to [email protected]<mailto:[email protected]> (previous value: '[email protected]<mailto:[email protected]>') == 2016-06-07 15:53:48,856 build_log.py:227 INFO building... == 2016-06-07 15:53:48,856 easyblock.py:2130 INFO Starting build step == 2016-06-07 15:53:48,859 easyblock.py:2133 INFO Running method build_step part of step build == 2016-06-07 15:53:48,859 build_log.py:227 INFO testing... == 2016-06-07 15:53:48,859 easyblock.py:2130 INFO Starting test step == 2016-06-07 15:53:48,862 easyblock.py:2133 INFO Running method test_step part of step test == 2016-06-07 15:53:48,862 build_log.py:227 INFO installing... == 2016-06-07 15:53:48,863 easyblock.py:2130 INFO Starting install step == 2016-06-07 15:53:48,865 easyblock.py:2133 INFO Running method stage_install_step part of step install == 2016-06-07 15:53:48,865 easyblock.py:2133 INFO Running method make_installdir part of step install == 2016-06-07 15:53:48,866 easyblock.py:2133 INFO Running method install_step part of step install == 2016-06-07 15:53:49,275 environment.py:97 INFO Environment variable LOCAL_INSTALL_VERBOSE set to 1 (previously undefined) == 2016-06-07 15:53:49,275 environment.py:97 INFO Environment variable VERBOSE_MODE set to 1 (previously undefined) == 2016-06-07 15:53:49,275 environment.py:97 INFO Environment variable INSTALL_PATH set to /sw/easybuild/software/Core/VTune/2016_update3 (previously undefined) == 2016-06-07 15:54:02,444 build_log.py:152 ERROR cmd "./install.sh -s /local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg" exited with exitcode 2 and output: WARNING: Destination directory already exists. Installing to this directory could overwrite existing files and folders within this directory. -------------------------------------------------------------------------------- -------------------------------------------------------------------------------- (at easybuild/software/Core/EasyBuild/2.8.1/lib/python2.7/site-packages/easybuild_framework-2.8.1-py2.7.egg/easybuild/tools/run.py:397 in parse_cmd_output) == 2016-06-07 15:54:02,444 easyblock.py:2362 WARNING build failed (first 300 chars): cmd "./install.sh -s /local/easybuild/build/VTune/2016_update3/dummy-dummy/vtune_amplifier_xe_2016_update3/silent.cfg" exite d with exitcode 2 and output: WARNING: Destination directory already exists. Installing to this directory could overwrite existing files and folders within this directory. -- == 2016-06-07 15:54:02,444 easyblock.py:270 INFO Closing log for application name VTune version 2016_update3 Using rm -rf /sw/easybuild/software/Core/VTune/2016_update3 which is the old and new installation directory holding the binaries, libs, etc. followed by eb VTune-2016_update3.eb --robot --force produced a new installation, incl module file. The user reported that there are issues, but I am not sure whether this is an EB issue or something else. If I think it is EB, I will let you know. Best wishes Joachim As far as I am concerned that is fine with me. I just assumed that the “force” option is not heavily used. I think users of EB need to be aware, that depending on the features of the underlying install mechanism, the EB “force” option may not work. Whether it is worth figuring, I severely doubt. There are situations in which --force is not reliable, but they should be rare... I used it quite often when testing pull requests, for example. if you want it to work better: Is there EB option for removing a piece of software (module and install), which I have so far not figured? That might be a way around any potential issues. If you issue force: EB will delete the old install and create a new one. Though that might create another can of worms that installs suddenly lack prerequisites. We don't have a --uninstall option, exactly because we don't have a mechanism in place to make sure no dependencies are broken, i.e. we don't have reverse dependency tracking. Yet, see also https://github.com/hpcugent/easybuild-framework/issues/590 . regards, Kenneth As I said, I am personally happy with the current state of affairs. Best wishes Joachim On 07 Jun 2016, at 14:26, Joachim Hein <[email protected]<mailto:[email protected]>> wrote: Hi Kenneth, Thanks for the quick reply. I had looked at the 2.8.1 release notes and must have overlooked the vtune update in there. So I was still on 2.8.0 (turning slightly read now) - thinking it was just a minor update ... EB 2.8.1 is currently testing. Best wishes Joachim On 07 Jun 2016, at 12:50, Kenneth Hoste <<mailto:[email protected]>[email protected]<mailto:[email protected]>> wrote: Hi Joachim, The VTune easyblock was updated recently to be made aware of this, see <https://github.com/hpcugent/easybuild-easyblocks/pull/935> https://github.com/hpcugent/easybuild-easyblocks/pull/935 . We also already provide an easyconfig file for VTune 2016 update 3 (in EasyBuild v2.8.1), cfr. https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/v/VTune/VTune-2016_update3.eb . So, if you're using EasyBuild v2.8.1, you shouldn't be seeing this? regards, Kenneth On 07/06/16 11:09, Joachim Hein wrote: Hi, I am trying to build vtune 2016 update 3 using the easyconfig 2013 update 10. There is this complication that there is now an additional intermediate layer in the installation. For example using a simple tweak of the 2013_update_10 config, binaries are expected in (full path of our settings): /sw/easybuild/software/Core/VTune/2016_update3/bin64 but they are actually installed in /sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe_2016.3.0.463186/bin64/ with soft-links: /sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe/bin64/ /sw/easybuild/software/Core/VTune/2016_update3/vtune_amplifier_xe_2016/bin64/ So by modifying the config’s sanity_chack_paths arguments from: 18c18 < 'files': ["bin64/amplxe-runss", "bin64/amplxe-cl"], --- > 'files': ["vtune_amplifier_xe_2016/bin64/amplxe-runss", > "vtune_amplifier_xe_2016/bin64/amplxe-cl"], I should make it check ok, but that change is not communicated to the module creation. As a results the PATH, LD_LIBARARY_PATH, CPATH, etc are not created in the module file. Looking into the easy block, there seems a lot of “auto-guessing” going on. Is there an easy way (like the sanity_chack_path) to make it guess in a different base directory? Is that documented in the read the docs? Thanks and best wishes Joachim
