Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete
On 15/05/2019 11:34, Loris Bennett wrote: Kenneth Hoste writes: On 15/05/2019 10:31, Loris Bennett wrote: Hi Lars, Thanks for the info. I hadn't realised that the package list for a specific version-toolchain combination could change. However, if I do '--force --skip' I get the following error: [build@admin ~]$ eb R-3.5.1-foss-2018b.eb --robot --force --skip == temporary log file in case of crash /tmp/eb-yj4XcQ/easybuild-x7nh9l.log == resolving dependencies ... == processing EasyBuild easyconfig /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb == building and installing R/3.5.1-foss-2018b... == fetching files... == FAILED: Installation ended unsuccessfully (build directory: /trinity/shared/easybuild/build/R/3.5.1/foss-2018b): build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed == Results of the build can be found in the log file(s) /tmp/eb-yj4XcQ/easybuild-R-3.5.1-20190515.102021.kwjjO.log ERROR: Build of /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb failed (err: 'build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed') I assume this means that the CRAN nlme package has been surreptitiously swapped without the version number being changed. Is '--inject-checksums' the best way dealing with this? In this case, you should probably re-download the nlme source tarball (or let EB re-download it for you). The checksum for the upstream source tarball indeed changed, so we fixed the checksum (see https://github.com/easybuilders/easybuild-easyconfigs/pull/8052). There were no actual code changes though (see PR description), so a re-download should be safe to do. We went through a similar issue for 'boot' recently (see https://github.com/easybuilders/easybuild-easyconfigs/pull/8054), after which they restored the original source tarball, so we had to revert that (see https://github.com/easybuilders/easybuild-easyconfigs/pull/8271). Yes, I have just bumped into the 'boot' problem. So I guess I have to get the GitHub integration set up so that I can make use of the PRs, right? "eb --from-pr" should just work, no setup needed at all I think... Has anyone reached out to the R people to point out that switching a package but keeping the version the same is a poke in the eye for anyone who gives a fig about reproducibility, not just for easybuilders? I've brought it up, yes, see for example https://github.com/easybuilders/easybuild-easyconfigs/pull/6446#issuecomment-396621378 and https://github.com/easybuilders/easybuild-easyconfigs/pull/7122#issuecomment-436573707 . Cheers, Loris [snip (63 lines)]
Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete
Kenneth Hoste writes: > On 15/05/2019 10:31, Loris Bennett wrote: >> Hi Lars, >> >> Thanks for the info. I hadn't realised that the package list for a >> specific version-toolchain combination could change. >> >> However, if I do '--force --skip' I get the following error: >> >>[build@admin ~]$ eb R-3.5.1-foss-2018b.eb --robot --force --skip >>== temporary log file in case of crash /tmp/eb-yj4XcQ/easybuild-x7nh9l.log >>== resolving dependencies ... >>== processing EasyBuild easyconfig >> /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb >>== building and installing R/3.5.1-foss-2018b... >>== fetching files... >>== FAILED: Installation ended unsuccessfully (build directory: >> /trinity/shared/easybuild/build/R/3.5.1/foss-2018b): build failed (first 300 >> chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed >>== Results of the build can be found in the log file(s) >> /tmp/eb-yj4XcQ/easybuild-R-3.5.1-20190515.102021.kwjjO.log >>ERROR: Build of >> /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb >> failed (err: 'build failed (first 300 chars): Checksum verification for >> extension source nlme_3.1-137.tar.gz failed') >> >> I assume this means that the CRAN nlme package has been surreptitiously >> swapped without the version number being changed. >> >> Is '--inject-checksums' the best way dealing with this? > > In this case, you should probably re-download the nlme source tarball (or let > EB > re-download it for you). > > The checksum for the upstream source tarball indeed changed, so we fixed the > checksum (see > https://github.com/easybuilders/easybuild-easyconfigs/pull/8052). > > There were no actual code changes though (see PR description), so a > re-download > should be safe to do. > > We went through a similar issue for 'boot' recently (see > https://github.com/easybuilders/easybuild-easyconfigs/pull/8054), after which > they restored the original source tarball, so we had to revert that (see > https://github.com/easybuilders/easybuild-easyconfigs/pull/8271). Yes, I have just bumped into the 'boot' problem. So I guess I have to get the GitHub integration set up so that I can make use of the PRs, right? Has anyone reached out to the R people to point out that switching a package but keeping the version the same is a poke in the eye for anyone who gives a fig about reproducibility, not just for easybuilders? Cheers, Loris [snip (63 lines)] -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete
On 15/05/2019 10:31, Loris Bennett wrote: Hi Lars, Thanks for the info. I hadn't realised that the package list for a specific version-toolchain combination could change. However, if I do '--force --skip' I get the following error: [build@admin ~]$ eb R-3.5.1-foss-2018b.eb --robot --force --skip == temporary log file in case of crash /tmp/eb-yj4XcQ/easybuild-x7nh9l.log == resolving dependencies ... == processing EasyBuild easyconfig /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb == building and installing R/3.5.1-foss-2018b... == fetching files... == FAILED: Installation ended unsuccessfully (build directory: /trinity/shared/easybuild/build/R/3.5.1/foss-2018b): build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed == Results of the build can be found in the log file(s) /tmp/eb-yj4XcQ/easybuild-R-3.5.1-20190515.102021.kwjjO.log ERROR: Build of /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb failed (err: 'build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed') I assume this means that the CRAN nlme package has been surreptitiously swapped without the version number being changed. Is '--inject-checksums' the best way dealing with this? In this case, you should probably re-download the nlme source tarball (or let EB re-download it for you). The checksum for the upstream source tarball indeed changed, so we fixed the checksum (see https://github.com/easybuilders/easybuild-easyconfigs/pull/8052). There were no actual code changes though (see PR description), so a re-download should be safe to do. We went through a similar issue for 'boot' recently (see https://github.com/easybuilders/easybuild-easyconfigs/pull/8054), after which they restored the original source tarball, so we had to revert that (see https://github.com/easybuilders/easybuild-easyconfigs/pull/8271). regards, Kenneth Cheers, Loris Lars Viklund writes: R-3.5.1-foss-2018.eb was refreshed adding some extensions, including the `shinythemes` that Bioconductor is lacking. I ran into this just last week. https://github.com/easybuilders/easybuild-easyconfigs/pull/7420 You should be able to address this by an `eb --force --skip` to install the missing extensions. This is apparently something that happens occasionally judging by the list of pull requests in the easyconfigs repo. The PR for Bioconductor/3.7 now mentions that it depends on the refreshed R/3.5.1 PR. // Lars From: easybuild-requ...@lists.ugent.be on behalf of Loris Bennett Sent: Tuesday, May 14, 2019 16:58 To: easybuild Subject: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete Hi, I just tried to do eb R-bundle-Bioconductor-3.7-foss-2018b-R-3.5.1.eb --robot This failed on the last of 134 packages with ERROR: dependency ‘shinythemes’ is not available for package ‘GSVA’ The package 'shinythemes' is one which should be installed by R-3.5.1-foss-2018b.eb but as it turns out, of the 668 packages which should be installed, only 648 in fact are. If some of the packages had failed to install, I would have expected the whole thing to fail and the module file not to be generated, but in fact it was. Am I missing something here? How could I debug the issue? Cheers, Loris -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete
I had faith in the current checksums in the recipe and moved the old files away from my sources/ tree and had EB redownload them and thus validate against the new checksums. For me, there were two files that had checksums change over time: mgcv_1.8-24.tar.gz and nlme_3.1-137.tar.gz From: Loris Bennett Sent: Wednesday, May 15, 2019 10:31 To: Lars Viklund Cc: easybuild@lists.ugent.be Subject: Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete Hi Lars, Thanks for the info. I hadn't realised that the package list for a specific version-toolchain combination could change. However, if I do '--force --skip' I get the following error: [build@admin ~]$ eb R-3.5.1-foss-2018b.eb --robot --force --skip == temporary log file in case of crash /tmp/eb-yj4XcQ/easybuild-x7nh9l.log == resolving dependencies ... == processing EasyBuild easyconfig /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb == building and installing R/3.5.1-foss-2018b... == fetching files... == FAILED: Installation ended unsuccessfully (build directory: /trinity/shared/easybuild/build/R/3.5.1/foss-2018b): build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed == Results of the build can be found in the log file(s) /tmp/eb-yj4XcQ/easybuild-R-3.5.1-20190515.102021.kwjjO.log ERROR: Build of /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb failed (err: 'build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed') I assume this means that the CRAN nlme package has been surreptitiously swapped without the version number being changed. Is '--inject-checksums' the best way dealing with this? Cheers, Loris Lars Viklund writes: > R-3.5.1-foss-2018.eb was refreshed adding some extensions, including the > `shinythemes` that Bioconductor is lacking. > I ran into this just last week. > > https://github.com/easybuilders/easybuild-easyconfigs/pull/7420 > > You should be able to address this by an `eb --force --skip` to install the > missing extensions. > > This is apparently something that happens occasionally judging by the list of > pull requests in the easyconfigs repo. > > The PR for Bioconductor/3.7 now mentions that it depends on the refreshed > R/3.5.1 PR. > > // Lars > > From: easybuild-requ...@lists.ugent.be on > behalf of Loris Bennett > Sent: Tuesday, May 14, 2019 16:58 > To: easybuild > Subject: [easybuild] Installation of R/3.5.1-foss-2018b successful but > incomplete > > Hi, > > I just tried to do > > eb R-bundle-Bioconductor-3.7-foss-2018b-R-3.5.1.eb --robot > > This failed on the last of 134 packages with > > ERROR: dependency ‘shinythemes’ is not available for package ‘GSVA’ > > The package 'shinythemes' is one which should be installed by > > R-3.5.1-foss-2018b.eb > > but as it turns out, of the 668 packages which should be installed, only > 648 in fact are. > > If some of the packages had failed to install, I would have expected the > whole thing to fail and the module file not to be generated, but in fact > it was. > > Am I missing something here? How could I debug the issue? > > Cheers, > > Loris > > -- > Dr. Loris Bennett (Mr.) > ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete
Hi Lars, Thanks for the info. I hadn't realised that the package list for a specific version-toolchain combination could change. However, if I do '--force --skip' I get the following error: [build@admin ~]$ eb R-3.5.1-foss-2018b.eb --robot --force --skip == temporary log file in case of crash /tmp/eb-yj4XcQ/easybuild-x7nh9l.log == resolving dependencies ... == processing EasyBuild easyconfig /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb == building and installing R/3.5.1-foss-2018b... == fetching files... == FAILED: Installation ended unsuccessfully (build directory: /trinity/shared/easybuild/build/R/3.5.1/foss-2018b): build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed == Results of the build can be found in the log file(s) /tmp/eb-yj4XcQ/easybuild-R-3.5.1-20190515.102021.kwjjO.log ERROR: Build of /trinity/shared/easybuild/software/EasyBuild/3.9.0/lib/python2.7/site-packages/easybuild_easyconfigs-3.9.0-py2.7.egg/easybuild/easyconfigs/r/R/R-3.5.1-foss-2018b.eb failed (err: 'build failed (first 300 chars): Checksum verification for extension source nlme_3.1-137.tar.gz failed') I assume this means that the CRAN nlme package has been surreptitiously swapped without the version number being changed. Is '--inject-checksums' the best way dealing with this? Cheers, Loris Lars Viklund writes: > R-3.5.1-foss-2018.eb was refreshed adding some extensions, including the > `shinythemes` that Bioconductor is lacking. > I ran into this just last week. > > https://github.com/easybuilders/easybuild-easyconfigs/pull/7420 > > You should be able to address this by an `eb --force --skip` to install the > missing extensions. > > This is apparently something that happens occasionally judging by the list of > pull requests in the easyconfigs repo. > > The PR for Bioconductor/3.7 now mentions that it depends on the refreshed > R/3.5.1 PR. > > // Lars > > From: easybuild-requ...@lists.ugent.be on > behalf of Loris Bennett > Sent: Tuesday, May 14, 2019 16:58 > To: easybuild > Subject: [easybuild] Installation of R/3.5.1-foss-2018b successful but > incomplete > > Hi, > > I just tried to do > > eb R-bundle-Bioconductor-3.7-foss-2018b-R-3.5.1.eb --robot > > This failed on the last of 134 packages with > > ERROR: dependency ‘shinythemes’ is not available for package ‘GSVA’ > > The package 'shinythemes' is one which should be installed by > > R-3.5.1-foss-2018b.eb > > but as it turns out, of the 668 packages which should be installed, only > 648 in fact are. > > If some of the packages had failed to install, I would have expected the > whole thing to fail and the module file not to be generated, but in fact > it was. > > Am I missing something here? How could I debug the issue? > > Cheers, > > Loris > > -- > Dr. Loris Bennett (Mr.) > ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
Re: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete
R-3.5.1-foss-2018.eb was refreshed adding some extensions, including the `shinythemes` that Bioconductor is lacking. I ran into this just last week. https://github.com/easybuilders/easybuild-easyconfigs/pull/7420 You should be able to address this by an `eb --force --skip` to install the missing extensions. This is apparently something that happens occasionally judging by the list of pull requests in the easyconfigs repo. The PR for Bioconductor/3.7 now mentions that it depends on the refreshed R/3.5.1 PR. // Lars From: easybuild-requ...@lists.ugent.be on behalf of Loris Bennett Sent: Tuesday, May 14, 2019 16:58 To: easybuild Subject: [easybuild] Installation of R/3.5.1-foss-2018b successful but incomplete Hi, I just tried to do eb R-bundle-Bioconductor-3.7-foss-2018b-R-3.5.1.eb --robot This failed on the last of 134 packages with ERROR: dependency ‘shinythemes’ is not available for package ‘GSVA’ The package 'shinythemes' is one which should be installed by R-3.5.1-foss-2018b.eb but as it turns out, of the 668 packages which should be installed, only 648 in fact are. If some of the packages had failed to install, I would have expected the whole thing to fail and the module file not to be generated, but in fact it was. Am I missing something here? How could I debug the issue? Cheers, Loris -- Dr. Loris Bennett (Mr.) ZEDAT, Freie Universität Berlin Email loris.benn...@fu-berlin.de
