Hi Kenneth,
That is what I mean by source path as well. I have the tar and the eb file in
the same folder and for some reason when I use the try-amend=parallel=1, it
makes easybuild look in my local directory
(/afs/crc.nd.edu/user/k/khostetl/.local/easybuild/sources/i/impi/l_mpi_pu_4.0.0.028.tgz).
As for how I provided the license, I tried both the default way [license_file =
os.path.join(os.getenv('HOME'), "licenses", "intel", "license.lic")], as well
as directly specifying the path myself. Both of these ways worked fine for icc
and ifort when I made them.
I have not been the one to install any of the intel packages, so I don't know
if there was ever an issue with the license before.
________________________________________
From: [email protected] [[email protected]] On
Behalf Of Kenneth Hoste [[email protected]]
Sent: Monday, March 17, 2014 2:28 AM
To: [email protected]
Subject: Re: [easybuild] Postprocessing Error
Hi Keith,
On 14/03/14 20:21, Keith Hostetler wrote:
> Thanks for the replies Kenneth and Fotis.
>
> I first tried your suggestion Fotis, but it changed my source path that I set
> in the eb file for some reason.
What do you mean exactly by "changed my source path"? To us, source path
is the location where EasyBuild tries to find the install files (i.e.
the 'source' tarball)...
> Here was the script I used:
>
> #!/bin/bash
>
> export EASYBUILD_BUILDPATH=/opt/crc/easybuild/build
> installPath=/opt/crc/easybuild-soft
> ebFile=$1
> eb --try-amend=parallel=1 --installpath=$installPath $ebFile --robot
>
> I just run this script with impi-4.0.0.028.eb as a parameter.
The "parallel=1" part is meaningless when installing impi, because
nothing is being built (only installed), so it's never used (this is
intended to result in something like 'make -j 1" in the build step).
>
> After this, I instead tried logging as Kenneth suggested. One thing I noticed
> a little bit before it errors out is:
>
> No valid FLEXlm license keys were found for this product.
> To get FLEXlm license please contact the Intel(R) support team at
> https://registrationcenter.intel.com/support.
> Exiting...
>
> So this makes it seem like a license problem. But why does it let me make icc
> and ifort just fine? The only difference I see is that icc and ifort were new
> versions and impi was an older version, but that shouldn't be an issue I
> don't think.
I think this part is very relevant.
What's basically happening is that nothing is being installed, and it
seems like not providing a valid license is the problem...
How did you provide a license for EasyBuild to use? Is it just picking
it up via $INTEL_LICENSE_FILE or $LM_LICENSE_FILE which is possibly
already set in your environment? Or are you providing a license file
(e.g. via $HOME/licenses/intel/license.lic, which is the default path)?
Have you ever been able to install old versions of Intel MPI with this
license?
regards,
Kenneth
>
> Here is most of the log (I dont know how to log it to a file):
>
> s: value mpi
> == 2014-03-14 15:19:56,349 main.easyconfig.easyconfig WARNING Unable to
> resolve template value l_mpi_pu_%(version)s.tgz with dict {'versionprefix':
> '', 'versionsuffix': '', 'toolchain_name': 'dummy', 'toolchain_version':
> 'dummy'}
> == 2014-03-14 15:19:56,350 main.EasyConfig INFO setting config option
> sources: value ['l_mpi_pu_%(version)s.tgz']
> == 2014-03-14 15:19:56,350 main.EasyConfig INFO setting config option
> version: value 4.0.0.028
> == 2014-03-14 15:19:56,351 main.EasyConfig INFO setting config option
> dontcreateinstalldir: value True
> == 2014-03-14 15:19:56,351 main.EasyConfig INFO setting config option
> license_file: value /afs/crc.nd.edu/user/k/khostetl/licenses/intel/license.lic
> == 2014-03-14 15:19:56,351 main.EasyConfig INFO setting config option
> homepage: value http://software.intel.com/en-us/intel-mpi-library/
> == 2014-03-14 15:19:56,352 main.EasyConfig INFO setting config option
> description: value The Intel(R) MPI Library for Linux* OS is a multi-fabric
> message
> passing library based on ANL MPICH2 and OSU MVAPICH2. The Intel MPI Library
> for
> Linux OS implements the Message Passing Interface, version 2 (MPI-2)
> specification.
> == 2014-03-14 15:19:56,357 main.EasyConfig INFO Validating easy block
> == 2014-03-14 15:19:56,357 main.EasyConfig INFO Checking OS dependencies
> == 2014-03-14 15:19:56,358 main.EasyConfig INFO OS dependencies ok: []
> == 2014-03-14 15:19:56,358 main.EasyConfig INFO Checking skipsteps
> == 2014-03-14 15:19:56,358 main.EasyConfig INFO Checking build option lists
> == 2014-03-14 15:19:56,359 main.EasyConfig INFO Checking licenses
> == 2014-03-14 15:19:56,359 main.EasyConfig WARNING Deprecated functionality,
> will no longer work in v2.0: Mandatory license not enforced
> == 2014-03-14 15:19:56,364 main.EB_impi INFO This is EasyBuild 1.10.0
> (framework: 1.10.0, easyblocks: 1.10.0) on host crcfe01.crc.nd.edu.
> == 2014-03-14 15:19:56,365 main.EB_impi INFO Init completed for application
> name impi version 4.0.0.028
> == 2014-03-14 15:19:56,366 main INFO Obtained application instance of for
> impi (easyblock: None)
> == 2014-03-14 15:19:56,368 main.EB_impi INFO building and installing
> impi/4.0.0.028...
> == building and installing impi/4.0.0.028...
> == 2014-03-14 15:19:56,369 main.EB_impi INFO fetching files...
> == fetching files...
> == 2014-03-14 15:19:56,369 main.EB_impi INFO Starting fetch step
> == 2014-03-14 15:19:56,375 main.EB_impi INFO Running method fetch_step part
> of step fetch
> == 2014-03-14 15:19:56,375 main.EB_impi WARNING Easyconfig does not specify
> an EasyBuild-version (key 'easybuild_version')! Assuming the latest version
> == 2014-03-14 15:19:56,408 main.EB_impi INFO Found file
> l_mpi_pu_4.0.0.028.tgz at
> /afs/crc.nd.edu/x86_64_linux/easybuild/test/l_mpi_pu_4.0.0.028.tgz
> == 2014-03-14 15:19:56,409 main.EB_impi INFO Added sources: [{'path':
> '/afs/crc.nd.edu/x86_64_linux/easybuild/test/l_mpi_pu_4.0.0.028.tgz', 'cmd':
> None, 'finalpath': None, 'name': 'l_mpi_pu_4.0.0.028.tgz', 'checksum': None}]
> == 2014-03-14 15:19:56,410 main.EB_impi INFO no patches provided
> == 2014-03-14 15:19:56,996 main.EB_impi INFO MD5 checksum for
> /afs/crc.nd.edu/x86_64_linux/easybuild/test/l_mpi_pu_4.0.0.028.tgz:
> c82734bfdbfe00536ba4fd8390df9637
> == 2014-03-14 15:19:57,032 main.EB_impi INFO Setting parallelism: 64
> == 2014-03-14 15:19:57,033 main.EB_impi INFO Checking dirs that need to be
> created: ['/opt/crc/easybuild-soft/software/impi',
> '/opt/crc/easybuild-soft/modules/all/impi',
> '/opt/crc/easybuild-soft/modules/mpi/impi']
> == 2014-03-14 15:19:57,034 main.EB_impi INFO creating build dir, resetting
> environment...
> == creating build dir, resetting environment...
> == 2014-03-14 15:19:57,035 main.EB_impi INFO Starting ready step
> == 2014-03-14 15:19:57,040 main.EB_impi INFO Running method
> check_readiness_step part of step ready
> == 2014-03-14 15:19:57,058 main.EB_impi WARNING Loaded modules detected:
> ['opt_local/1.0', '/afs/crc.nd.edu/x86_64_linux/easybuild/ebModule-1.10.0',
> 'intel/12.1']
> == 2014-03-14 15:19:57,075 main.EnvironmentModulesC INFO Prepended list of
> module paths with path used by EasyBuild: /opt/crc/easybuild-soft/modules/all
> == 2014-03-14 15:19:57,076 main.EnvironmentModulesC INFO $MODULEPATH set
> based on list of module paths:
> /opt/crc/easybuild-soft/modules/all:/afs/crc.nd.edu/x86_64_linux/Modules/3.2.8/modules/development_tools_and_libraries:/afs/crc.nd.edu/x86_64_linux/Modules/3.2.8/modules/general_software:/afs/crc.nd.edu/x86_64_linux/Modules/3.2.8/modules/restricted_software:/afs/crc.nd.edu/x86_64_linux/Modules/3.2.8/modules/deprecated_software:/afs/crc.nd.edu/x86_64_linux/Modules/3.2.8/modules/system_modules
> == 2014-03-14 15:19:57,076 main.filetools INFO Command modulecmd found at
> /opt/crc/Modules/current/bin/modulecmd
> == 2014-03-14 15:19:57,077 main.EnvironmentModulesC INFO Full path for module
> command is /opt/crc/Modules/current/bin/modulecmd, so using it
> == 2014-03-14 15:19:57,201 main.EB_impi INFO Running method gen_builddir part
> of step ready
> == 2014-03-14 15:19:57,202 main.EB_impi INFO Build dir set to
> /afs/crc.nd.edu/user/k/khostetl/.local/easybuild/build/impi/4.0.0.028/dummy-dummy.2
> == 2014-03-14 15:19:57,203 main.EB_impi INFO Running method gen_installdir
> part of step ready
> == 2014-03-14 15:19:57,204 main.EB_impi INFO Running method make_builddir
> part of step ready
> == 2014-03-14 15:19:57,205 main.EB_impi INFO Running method env_reset_changes
> part of step ready
> == 2014-03-14 15:19:57,206 main.EB_impi INFO Running method
> handle_iterate_opts part of step ready
> == 2014-03-14 15:19:57,207 main.EB_impi INFO unpacking...
> == unpacking...
> == 2014-03-14 15:19:57,207 main.EB_impi INFO Starting source step
> == 2014-03-14 15:19:57,212 main.EB_impi INFO Running method checksum_step
> part of step source
> == 2014-03-14 15:19:57,213 main.EB_impi INFO Checksum verification for
> /afs/crc.nd.edu/x86_64_linux/easybuild/test/l_mpi_pu_4.0.0.028.tgz using None
> passed.
> == 2014-03-14 15:19:57,213 main.EB_impi INFO Running method extract_step part
> of step source
> == 2014-03-14 15:19:57,214 main.EB_impi INFO Unpacking source
> l_mpi_pu_4.0.0.028.tgz
> == 2014-03-14 15:20:01,300 main.EB_impi INFO patching...
> == patching...
> == 2014-03-14 15:20:01,301 main.EB_impi INFO Starting patch step
> == 2014-03-14 15:20:01,305 main.EB_impi INFO Running method patch_step part
> of step patch
> == 2014-03-14 15:20:01,305 main.EB_impi INFO preparing...
> == preparing...
> == 2014-03-14 15:20:01,306 main.EB_impi INFO Starting prepare step
> == 2014-03-14 15:20:01,310 main.EB_impi INFO Running method prepare_step part
> of step prepare
> == 2014-03-14 15:20:01,310 main.Dummy INFO prepare: toolchain dummy mode,
> dummy version; not loading dependencies
> == 2014-03-14 15:20:01,311 main.EB_impi INFO configuring...
> == configuring...
> == 2014-03-14 15:20:01,311 main.EB_impi INFO Starting configure step
> == 2014-03-14 15:20:01,315 main.EB_impi INFO Running method configure_step
> part of step configure
> == 2014-03-14 15:20:01,316 main.EB_impi INFO Picked the first *.lic file from
> $INTEL_LICENSE_FILE: /afs/crc.nd.edu/x86_64_linux/intel/licenses/license.lic
> == 2014-03-14 15:20:01,317 main.environment INFO Environment variable
> INTEL_LICENSE_FILE set to
> /afs/crc.nd.edu/x86_64_linux/intel/licenses/license.lic
> == 2014-03-14 15:20:01,317 main.EB_impi INFO Using Intel license
> specifications from $INTEL_LICENSE_FILE:
> /afs/crc.nd.edu/x86_64_linux/intel/licenses/license.lic
> == 2014-03-14 15:20:01,318 main.EB_impi INFO building...
> == building...
> == 2014-03-14 15:20:01,318 main.EB_impi INFO Starting build step
> == 2014-03-14 15:20:01,322 main.EB_impi INFO Running method build_step part
> of step build
> == 2014-03-14 15:20:01,322 main.EB_impi INFO testing...
> == testing...
> == 2014-03-14 15:20:01,322 main.EB_impi INFO Starting test step
> == 2014-03-14 15:20:01,328 main.EB_impi INFO Running method test_step part of
> step test
> == 2014-03-14 15:20:01,328 main.EB_impi INFO installing...
> == installing...
> == 2014-03-14 15:20:01,329 main.EB_impi INFO Starting install step
> == 2014-03-14 15:20:01,334 main.EB_impi INFO Running method
> stage_install_step part of step install
> == 2014-03-14 15:20:01,334 main.EB_impi INFO Running method make_installdir
> part of step install
> == 2014-03-14 15:20:01,335 main.EB_impi INFO Cleaning only, no actual
> creation of /opt/crc/easybuild-soft/software/impi/4.0.0.028, only
> verification/creation of dirname /opt/crc/easybuild-soft/software/impi
> == 2014-03-14 15:20:01,336 main.EB_impi INFO Running method install_step part
> of step install
> == 2014-03-14 15:20:01,947 main.filetools INFO cmd "./install.sh
> --tmp-dir=/afs/crc.nd.edu/user/k/khostetl/.local/easybuild/build/impi/4.0.0.028/dummy-dummy.2/l_mpi_pu_4.0.0.028/4.0.0.028/mytmpdir
>
> --silent=/afs/crc.nd.edu/user/k/khostetl/.local/easybuild/build/impi/4.0.0.028/dummy-dummy.2/l_mpi_pu_4.0.0.028/silent.cfg"
> exited with exitcode 0 and output:
> No valid FLEXlm license keys were found for this product.
> To get FLEXlm license please contact the Intel(R) support team at
> https://registrationcenter.intel.com/support.
> Exiting...
>
> == 2014-03-14 15:20:01,949 main.EB_impi INFO taking care of extensions...
> == taking care of extensions...
> == 2014-03-14 15:20:01,949 main.EB_impi INFO Starting extensions step
> == 2014-03-14 15:20:01,955 main.EB_impi INFO Running method extensions_step
> part of step extensions
> == 2014-03-14 15:20:01,955 main.EB_impi INFO packaging...
> == packaging...
> == 2014-03-14 15:20:01,956 main.EB_impi INFO Starting package step
> == 2014-03-14 15:20:01,961 main.EB_impi INFO Running method package_step part
> of step package
> == 2014-03-14 15:20:01,961 main.EB_impi INFO postprocessing...
> == postprocessing...
> == 2014-03-14 15:20:01,962 main.EB_impi INFO Starting postproc step
> == 2014-03-14 15:20:01,967 main.EB_impi INFO Running method post_install_step
> part of step postproc
> == 2014-03-14 15:20:01,967 main.filetools INFO Adjusting permissions
> recursively for /opt/crc/easybuild-soft/software/impi/4.0.0.028
> == 2014-03-14 15:20:01,968 main.filetools INFO Failed to chmod/chown
> /opt/crc/easybuild-soft/software/impi/4.0.0.028 (but ignoring it): [Errno 2]
> No such file or directory: '/opt/crc/easybuild-soft/software/impi/4.0.0.028'
> == 2014-03-14 15:20:01,969 main.filetools ERROR EasyBuild crashed with an
> error (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/filetools.py:962
> in adjust_permissions): 100.00% of permissions/owner operations failed (more
> than 50.00%), something must be wrong...
> == 2014-03-14 15:20:01,970 main ERROR EasyBuild crashed with an error (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/build_log.py:96
> in exception): EasyBuild encountered an exception (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/main.py:809
> in build_and_install_software): autoBuild Failed (last 300 chars): EasyBuild
> crashed with an error (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/filetools.py:962
> in adjust_permissions): 100.00% of permissions/owner operations failed (more
> than 50.00%), something must be wrong...
> Traceback (most recent call last):
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/main.py",
> line 805, in build_and_install_software
> regtest_online=options.regtest_online)
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/framework/easyblock.py",
> line 1817, in run_all_steps
> self.run_step(stop_name, step_methods, skippable=skippable)
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/framework/easyblock.py",
> line 1699, in run_step
> m(self)
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/framework/easyblock.py",
> line 1784, in <lambda>
> ('postproc', 'postprocessing', [lambda x: x.post_install_step()], True),
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/framework/easyblock.py",
> line 1474, in post_install_step
> adjust_permissions(self.installdir, perms, add=False, recursive=True,
> relative=True, ignore_errors=True)
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/filetools.py",
> line 962, in adjust_permissions
> (100*fail_ratio, 100*max_fail_ratio))
> File
> "/opt/crc/easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/build_log.py",
> line 88, in error
> raise EasyBuildError(newMsg)
> EasyBuildError: 'EasyBuild crashed with an error (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/filetools.py:962
> in adjust_permissions): 100.00% of permissions/owner operations failed (more
> than 50.00%), something must be wrong...'
> ERROR: EasyBuild encountered an exception (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/main.py:809
> in build_and_install_software): autoBuild Failed (last 300 chars): EasyBuild
> crashed with an error (at
> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/filetools.py:962
> in adjust_permissions): 100.00% of permissions/owner operations failed (more
> than 50.00%), something must be wrong...
> [khostetl@crcfe01 test]$
>
>
> ________________________________________
> From: [email protected] [[email protected]] On
> Behalf Of Fotis Georgatos [[email protected]]
> Sent: Friday, March 14, 2014 1:01 PM
> To: [email protected]
> Subject: Re: [easybuild] Postprocessing Error
>
> Hi Keith,
>
> I recall I have seen this error, as well.
>
> If I remember correctly, it corresponded to (parallel) filesystem issues
> (metadata synchronization or something of that type).
>
> I suggest the following strategy:
> * export EASYBUILD_BUILDPATH=/dev/shm # or /tmp
> * export EASYBUILD_INSTALLPATH=/tmp
> * eb --try-amend=parallel=1 <foobar> # get rid of parallelization for a while
>
> If the build succeeds, you pretty much can corner the cause(s) now.
>
> On Mar 14, 2014, at 6:39 PM, Keith Hostetler wrote:
>> So I am finally getting the hang of easy build and have started making intel
>> toolchains. I built icc and ifort with no problems (though I haven't tested
>> them yet). However, I am having issues with impi. I am building
>> impi-4.0.0.028.eb. Everything looks fine until postprocessing where I get
>> this error:
>>
>> == postprocessing...
>> ERROR: EasyBuild encountered an exception (at
>> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/main.py:809
>> in build_and_install_software): autoBuild Failed (last 300 chars):
>> EasyBuild crashed with an error (at
>> easybuild/lib/python2.7/site-packages/easybuild_framework-1.10.0-py2.7.egg/easybuild/tools/filetools.py:962
>> in adjust_permissions): 100.00% of permissions/owner operations failed
>> (more than 50.00%), something must be wrong...
> good luck,
> Fotis
>
>
> --
> echo "sysadmin know better bash than english" | sed s/min/mins/ \
> | sed 's/better bash/bash better/' # Yelling in a CERN forum
>
>
>
>
