+1. I was editing the PyTorch module of Lars earlier, and it took me a while to realize what was going on with the submodules. A git clone --recursive would be way - dare I say it? - easier :)
Don't know if it's just lack of time, or whether there is a good reason why this hasn't been implemented (yet) in EasyBuild. More good topics for the EasyBuild user meeting? Caspar From: "Maxime Boissonneault" <maxime.boissonnea...@calculquebec.ca> To: "easybuild" <easybuild@lists.ugent.be>, "Espen Tangen" <espen.tan...@uit.no> Cc: "charles coulombe" <charles.coulo...@calculquebec.ca> Sent: Wednesday, 17 January, 2018 15:06:17 Subject: Re: [easybuild] Re: EB support for git submodules That's a long standing wish of one of my colleague as well. Maxime On 18-01-17 08:56, Espen Tangen wrote: No; I was more in line with Sources = "git clone --recursive --checkout=""commit-SHA""" Cmake Make But I get the feeling that world is not so easy yet? Et. Fra: [ mailto:easybuild-requ...@lists.ugent.be | <easybuild-requ...@lists.ugent.be> ] på vegne av Lars Viklund [ mailto:l...@hpc2n.umu.se | <l...@hpc2n.umu.se> ] Svar til: [ mailto:easybuild@lists.ugent.be | "easybuild@lists.ugent.be" ] [ mailto:easybuild@lists.ugent.be | <easybuild@lists.ugent.be> ] Dato: onsdag 17. januar 2018 14:15 Til: [ mailto:easybuild@lists.ugent.be | "easybuild@lists.ugent.be" ] [ mailto:easybuild@lists.ugent.be | <easybuild@lists.ugent.be> ] Emne: [easybuild] Re: EB support for git submodules Hi all, As I'm the one that made that config, it was indeed obtained by hand. If memory serves me right, I did a: git clone -b v0.3.0 --recursive [ https://github.com/pytorch/pytorch.git%60 | https://github.com/pytorch/pytorch.git ] and grabbed the hashes, repos, and paths from the output of the git command, manually transforming them into the shape you see in the easyconfig. I did this after seeing the horror of the mxnet.py easyblock which relied on unpacking all the tarballs into a common directory and moving them into the approximate places in the source tree. // Lars Viklund, HPC2N From: [ mailto:easybuild-requ...@lists.ugent.be | easybuild-requ...@lists.ugent.be ] [ mailto:easybuild-requ...@lists.ugent.be | <easybuild-requ...@lists.ugent.be> ] on behalf of Caspar van Leeuwen [ mailto:caspar.vanleeu...@surfsara.nl | <caspar.vanleeu...@surfsara.nl> ] Sent: Wednesday, January 17, 2018 11:12 To: easybuild Subject: Re: [easybuild] EB support for git submodules Hi Espen, Do you mean something like what's happening for the submodules of PyTorch (gloo, googletest, etc) in this EasyConfig: [ https://github.com/easybuilders/easybuild-easyconfigs/pull/5530/files#diff-60532b1c315a237f3b42ee9fafeef614 | https://github.com/easybuilders/easybuild-easyconfigs/pull/5530/files#diff-60532b1c315a237f3b42ee9fafeef614 ] ? (this uses the syntax [ https://github.com/ | https://github.com/ ] <github_account>/<repo_name>/archive/<hash>.tar.gz) Seems the hashes were obtained by hand there, I'm not sure if there's also an easier / more automated way to do this. Regards, Caspar From: "Espen Tangen" [ mailto:espen.tan...@uit.no | <espen.tan...@uit.no> ] To: "easybuild" [ mailto:easybuild@lists.ugent.be | <easybuild@lists.ugent.be> ] Sent: Wednesday, 17 January, 2018 10:41:08 Subject: [easybuild] EB support for git submodules Hi. I am trying to figure out if EB is supporting source download via git recursive - given a specific hash. What I have seen so far indicates that there is none, but I want to be sure before I create an issue on this. Regards Et. -- --------------------------------- Maxime Boissonneault Analyste de calcul - Calcul Québec, Université Laval Président - Comité de coordination du soutien à la recherche de Calcul Québec Team lead - Research Support National Team, Compute Canada Instructeur Software Carpentry Ph. D. en physique
