Dear William,
It's indeed a little bit too aggressive their with using the %(version)s
template in the generated easyconfig file...
Whenever %(version)s is used when installing an extension, it indeed
uses the version of the extension, rather than the "parent", which is
where this goes horribly wrong.
This is a bug, do you mind opening an issue for this at
https://github.com/easybuilders/easybuild-framework/issues, so we don't
lose track of it?
regards,
Kenneth
On 20/11/2019 17:05, William Brown wrote:
I have hit a problem building with EasyBuild 4.0.0
The issue looks to be the reverse substitution of %(version)s in the
source_urls, where the version picked up is not that of the main package
but of an extension from exts_list.
$ eb --buildpath=/ichec/home/users/brownwdl --job-cores=12
--optarch=march=skylake-avx512 --try-toolchain "foss,2019b" --dry-run
R-bundle-Bioconductor-3.9-foss-2019a-R-3.6.0.eb -r
I am trying to use an updated toolchain (because I have it already
built) so a temporary easyconfig has been created on-the-fly. This
re-orders the items in the easyconfig and seems to have
reverse-substituted the specified version ‘3.9’ with %(version) .
The original easyconfig for
R-bundle-Bioconductor-3.9-foss-2019a-R-3.6.0.eb has this:
easyblock = 'Bundle'
name = 'R-bundle-Bioconductor'
version = '3.9'
versionsuffix = '-R-%(rver)s'
.
.
exts_default_options = {
'source_urls': [
'https://bioconductor.org/packages/3.9/bioc/src/contrib/',
'https://bioconductor.org/packages/3.9/bioc/src/contrib/Archive/%(name)s',
'https://bioconductor.org/packages/3.9/data/annotation/src/contrib/',
'https://bioconductor.org/packages/3.9/data/experiment/src/contrib/',
'https://cran.r-project.org/src/contrib/Archive/%(name)s', #
package archive
'https://cran.r-project.org/src/contrib/', # current version
of packages
'https://cran.freestatistics.org/src/contrib', # mirror
alternative for current packages
],
'source_tmpl': '%(name)s_%(version)s.tar.gz'
}
The version is explicitly written as ‘3.9’. The ‘tweaked’ easyconfig
that is created for --try-toolchain "foss,2019b" has this:
easyblock = 'Bundle'
name = 'R-bundle-Bioconductor'
version = '3.9'
versionsuffix = '-R-%(rver)s'
.
.
exts_default_options = {
'source_tmpl': '%(name)s_%(version)s.tar.gz',
'source_urls':
['https://bioconductor.org/packages/%(version)s/bioc/src/contrib/',
'https://bioconductor.org/packages/%(version)s/bioc/src/contrib/Archive/%(name)s',
'https://bioconductor.org/packages/%(version)s/data/annotation/src/contrib/',
'https://bioconductor.org/packages/%(version)s/data/experiment/src/contrib/',
'https://cran.r-project.org/src/contrib/Archive/%(name)s',
'https://cran.r-project.org/src/contrib/',
'https://cran.freestatistics.org/src/contrib'],
}
When in the build process it gets to the BiocGenerics extension:
exts_list = [
('readr', '1.3.1', {
'checksums':
['33f94de39bb7f2a342fbb2bd4e5afcfec08798eac39672ee18042ac0b349e4f3'],
}),
('BiocGenerics', '0.30.0', {
'checksums':
['2e6d2fa6ed389cfae6fd75713379ba7dcd164a8161f37f1bbed596b70c3207d9'],
}),
It fails. It works for the first extension ‘readr’ but I think that it
had successfully downloaded that source on an earlier date (quite
possibly with an earlier version of EasyBuild).
== 2019-11-20 14:31:14,022 filetools.py:404 INFO Command eb found at
/ichec/work/rcsi01/packages/easybuild/software/EasyBuild/4.0.0/bin/eb
== 2019-11-20 14:31:14,044 easyblock.py:668 INFO Found file
readr_1.3.1.tar.gz at
/ichec/work/rcsi01/packages/easybuild/sources/r/R-bundle-Bioconductor/extensions/readr_1.3.1.tar.gz
== 2019-11-20 14:31:14,101 easyblock.py:528 INFO md5 checksum for
/ichec/work/rcsi01/packages/easybuild/sources/r/R-bundle-Bioconductor/extensions/readr_1.3.1.tar.gz:
b41b0e9251c19b1c54c032e1f55be9d0
== 2019-11-20 14:31:14,104 easyblock.py:528 INFO sha256 checksum for
/ichec/work/rcsi01/packages/easybuild/sources/r/R-bundle-Bioconductor/extensions/readr_1.3.1.tar.gz:
33f94de39bb7f2a342fbb2bd4e5afcfec08798eac39672ee18042ac0b349e4f3
== 2019-11-20 14:31:14,108 easyblock.py:534 INFO Checksum for extension
source readr_1.3.1.tar.gz verified
== 2019-11-20 14:31:14,108 easyblock.py:456 INFO Fetched extension
patches: []
== 2019-11-20 14:31:14,109 filetools.py:404 INFO Command eb found at
/ichec/work/rcsi01/packages/easybuild/software/EasyBuild/4.0.0/bin/eb
== 2019-11-20 14:31:14,555 filetools.py:559 WARNING URL
https://bioconductor.org/packages/0.30.0/bioc/src/contrib/BiocGenerics_0.30.0.tar.gz
was not found (HTTP response code 404), not trying again
== 2019-11-20 14:31:14,555 filetools.py:585 WARNING Download of
https://bioconductor.org/packages/0.30.0/bioc/src/contrib/BiocGenerics_0.30.0.tar.gz
to
/ichec/work/rcsi01/packages/easybuild/sources/r/R-bundle-Bioconductor/extensions/BiocGenerics_0.30.0.tar.gz
failed, done trying
It has substituted the %(version) from the version of the **extension**,
not the main package, and that cannot work.
I am not sure whether I have tried to do something incorrectly but I
would not expect the ‘source_urls’ to be edited in this way.
William Brown
*William Brown*
Systems Administrator
*RCSI* Information Technology
Royal College of Surgeons in Ireland
Building 121 St Stephens Green, IT dept
*T: *015720012
*E: *[email protected] *W: *www.rcsi.com <http://www.rcsi.com/>
/Transforming Healthcare Education, Research and Service: RCSI Strategic
Plan 2018-2022 <http://www.rcsi.ie/strategy2018>/
