Re: [easybuild] Python 3 in 2018b toolchains
I have pushed corrections to Libjpeg turbo 2.0 #6717 I am working on your recommendations for R-3.5.1 and will update my repo today John Dey On 8/16/18, 12:14 AM, "Kenneth Hoste" wrote: Dear John, On 15/08/2018 19:37, Dey, John F wrote: > I have R-3.5.1, Python-3.7.0 and Python-2.7.15 built for 2018b. I am in the process of getting these pushed. I am pushing the 2018b libraries first. > > Before I plunder ahead with a pull request they can be reviewed here: https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_FredHutch_easybuild-2Dlife-2Dsciences_blob_master_fh-5Feasyconfigs_=DwIDaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=EXBuwM9jpv51OScj00Tp2UjJwujbdSQ3O2EJl6pAfOU=THQBmGd-bN8q8bnDCXeFy-9r9Ow9h6xzRvmSocvJ-pc= We already have easyconfigs merged into develop for Python 2.7.15, see https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6479=DwIDaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=EXBuwM9jpv51OScj00Tp2UjJwujbdSQ3O2EJl6pAfOU=3EDLCyH2SeD1nbJZaRKIXFjjFaES9sJ3JHr-EoPpdJs= . For Python 3.x, we added easyconfigs for Python 3.7.0 as well, but we had to revisit that to go forward with Python 3.6.6 instead of the foss/2018b & intel/2018b toolchains because of too many problems with Python 3.7.0, see https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682=DwIDaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=EXBuwM9jpv51OScj00Tp2UjJwujbdSQ3O2EJl6pAfOU=QBo14giOHs1he0QnCmmeTGeK9_9s4yMcdiSDFen57lQ= for more information. The easyconfigs for Python 3.6.6 were merged into develop just yesterday. W.r.t. R 3.5.1: there are a couple of minor things I would like to see changed to your easyconfig, for example the hardcoding of 'GCCcore' for a couple of dependencies should be removed. There are also a couple of extensions missing that have been added recently, and it seems we could do another round of updating for the included extensions? Let me know if you need any help with that, but first, let's get your PR that adds a bunch of tools & libraries for foss/2018b merged, cfr. https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6704=DwIDaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=EXBuwM9jpv51OScj00Tp2UjJwujbdSQ3O2EJl6pAfOU=3Zq8xeRPYCoyaJrqLxlXJCw8J3_GJuVAPuBecuo9jws= (+ my PR to your branch to make your PR good to go). regards, Kenneth > > > John Dey > > On 8/15/18, 3:34 AM, "easybuild-requ...@lists.ugent.be on behalf of Joachim Hein" wrote: > > > > > On 15 Aug 2018, at 12:18, Kenneth Hoste wrote: > > > > Dear Joachim, > > > > On 15/08/2018 11:29, Joachim Hein wrote: > >> Hi, > >> I noticed https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=F_o0YzRqd_v2kIt6n3YbPE5TtaGQTY8_dRqXxX2Ftrw= and https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6703=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=Zk_4ID_w-2HttuYo5o-SHbAl3fJh451rV9XWaekA9K8= > >> I am presently working on moving python packages to foss and intel 2018b (h5py was merged, so did boost with python). Could there please be a discussion/statement how we want to handle this? > >> Also, we need to think how we want to share such important info in our community? Discussion in slack seems not to be working (the channel is to busy in my view). Stumbling about stuff on GitHub is in my view not the best way to share info like that. > >> Thanks for reading. > > > > I was planning to also share this on the mailing list, but you beat me to it. > > > > The plan is outlined at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682-23issuecomment-2D413003200=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=C1U2AeaGCPl-6qF6BV2EPABGwMWAuRvne8B7vq8fNxo=, in short: > > > > * get the easyconfigs for Python 3.6.6 with */2018b merged (see https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6703=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=Zk_4ID_w-2HttuYo5o-SHbAl3fJh451rV9XWaekA9K8=) >
Re: [easybuild] Python 3 in 2018b toolchains
> On 16 Aug 2018, at 09:16, Kenneth Hoste wrote: > > Dear Joachim, > > On 15/08/2018 12:34, Joachim Hein wrote: >>> On 15 Aug 2018, at 12:18, Kenneth Hoste wrote: >>> >>> Dear Joachim, >>> >>> On 15/08/2018 11:29, Joachim Hein wrote: Hi, I noticed https://github.com/easybuilders/easybuild-easyconfigs/issues/6682 and https://github.com/easybuilders/easybuild-easyconfigs/pull/6703 I am presently working on moving python packages to foss and intel 2018b (h5py was merged, so did boost with python). Could there please be a discussion/statement how we want to handle this? Also, we need to think how we want to share such important info in our community? Discussion in slack seems not to be working (the channel is to busy in my view). Stumbling about stuff on GitHub is in my view not the best way to share info like that. Thanks for reading. >>> >>> I was planning to also share this on the mailing list, but you beat me to >>> it. >>> >>> The plan is outlined at >>> https://github.com/easybuilders/easybuild-easyconfigs/issues/6682#issuecomment-413003200, >>> in short: >>> >>> * get the easyconfigs for Python 3.6.6 with */2018b merged (see >>> https://github.com/easybuilders/easybuild-easyconfigs/pull/6703) >>> >>> * rework easyconfigs already in develop to use Python 3.6.6 rather than >>> 3.7.0 >>> >>> >> Kenneth, >> Thanks. Are we withdrawing 3.7 or adding 3.6.6? There is something to be >> said for keeping the 3.7.0 but have the 3.6.6 as the preferred and better >> supported option. Similar to what we have in the GCC space. We often have >> a conservative GCC in the standard toolchains, but offer more modern >> versions (e.g. GCC 8.1) as an option with limited support. The question is, >> how one would communicate that to users … > > As mentioned in > https://github.com/easybuilders/easybuild-easyconfigs/issues/6682, we plan to > keep the easyconfigs for Python 3.7.0. > > The tests that are run for each pull request ensure that only one Python 3.x > version is used as dependency for the 2018b toolchain generation, so if > someone opens a PR using Python 3.7.0, the tests will make it clear 3.6.6 > should be used instead. Hi Kenneth, I read that: 3.6.6 is now the “preferred” option but the 3.7.0 remains available. Thanks, that is what I asked. I rolled out the 3.7.0 on a few clusters and withdrawing things is always tricky. > > >> Is it an option to have an EB quick release, that only releases the new >> Python 3.6.6 (and a few add ons)? Having that in a proper release makes >> developing python packages (like h5py) so much easier (at least for my >> workflow). > > I'm trying to work towards another release, but I still have a fair amount of > catching up to do after my summer leave, and there are a couple more (minor) > things I would like to see included in EasyBuild v3.6.3… > Fine. Since writing the last time, I decided to do it another way. Having the Python 3.6.6 released quickly is not a concern for me any longer. Thanks and best wishes Joachim > > regards, > > Kenneth > >> Best wishes >>joachim >>> regards, >>> >>> Kenneth
Re: [easybuild] Python 3 in 2018b toolchains
Dear John, On 15/08/2018 19:37, Dey, John F wrote: I have R-3.5.1, Python-3.7.0 and Python-2.7.15 built for 2018b. I am in the process of getting these pushed. I am pushing the 2018b libraries first. Before I plunder ahead with a pull request they can be reviewed here: https://github.com/FredHutch/easybuild-life-sciences/blob/master/fh_easyconfigs/ We already have easyconfigs merged into develop for Python 2.7.15, see https://github.com/easybuilders/easybuild-easyconfigs/pull/6479 . For Python 3.x, we added easyconfigs for Python 3.7.0 as well, but we had to revisit that to go forward with Python 3.6.6 instead of the foss/2018b & intel/2018b toolchains because of too many problems with Python 3.7.0, see https://github.com/easybuilders/easybuild-easyconfigs/issues/6682 for more information. The easyconfigs for Python 3.6.6 were merged into develop just yesterday. W.r.t. R 3.5.1: there are a couple of minor things I would like to see changed to your easyconfig, for example the hardcoding of 'GCCcore' for a couple of dependencies should be removed. There are also a couple of extensions missing that have been added recently, and it seems we could do another round of updating for the included extensions? Let me know if you need any help with that, but first, let's get your PR that adds a bunch of tools & libraries for foss/2018b merged, cfr. https://github.com/easybuilders/easybuild-easyconfigs/pull/6704 (+ my PR to your branch to make your PR good to go). regards, Kenneth John Dey On 8/15/18, 3:34 AM, "easybuild-requ...@lists.ugent.be on behalf of Joachim Hein" wrote: > On 15 Aug 2018, at 12:18, Kenneth Hoste wrote: > > Dear Joachim, > > On 15/08/2018 11:29, Joachim Hein wrote: >> Hi, >> I noticed https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=F_o0YzRqd_v2kIt6n3YbPE5TtaGQTY8_dRqXxX2Ftrw= and https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6703=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=Zk_4ID_w-2HttuYo5o-SHbAl3fJh451rV9XWaekA9K8= >> I am presently working on moving python packages to foss and intel 2018b (h5py was merged, so did boost with python). Could there please be a discussion/statement how we want to handle this? >> Also, we need to think how we want to share such important info in our community? Discussion in slack seems not to be working (the channel is to busy in my view). Stumbling about stuff on GitHub is in my view not the best way to share info like that. >> Thanks for reading. > > I was planning to also share this on the mailing list, but you beat me to it. > > The plan is outlined at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682-23issuecomment-2D413003200=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=C1U2AeaGCPl-6qF6BV2EPABGwMWAuRvne8B7vq8fNxo=, in short: > > * get the easyconfigs for Python 3.6.6 with */2018b merged (see https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6703=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=Zk_4ID_w-2HttuYo5o-SHbAl3fJh451rV9XWaekA9K8=) > > * rework easyconfigs already in develop to use Python 3.6.6 rather than 3.7.0 > > Kenneth, Thanks. Are we withdrawing 3.7 or adding 3.6.6? There is something to be said for keeping the 3.7.0 but have the 3.6.6 as the preferred and better supported option. Similar to what we have in the GCC space. We often have a conservative GCC in the standard toolchains, but offer more modern versions (e.g. GCC 8.1) as an option with limited support. The question is, how one would communicate that to users … Is it an option to have an EB quick release, that only releases the new Python 3.6.6 (and a few add ons)? Having that in a proper release makes developing python packages (like h5py) so much easier (at least for my workflow). Best wishes joachim > regards, > > Kenneth
Re: [easybuild] Python 3 in 2018b toolchains
I have R-3.5.1, Python-3.7.0 and Python-2.7.15 built for 2018b. I am in the process of getting these pushed. I am pushing the 2018b libraries first. Before I plunder ahead with a pull request they can be reviewed here: https://github.com/FredHutch/easybuild-life-sciences/blob/master/fh_easyconfigs/ John Dey On 8/15/18, 3:34 AM, "easybuild-requ...@lists.ugent.be on behalf of Joachim Hein" wrote: > On 15 Aug 2018, at 12:18, Kenneth Hoste wrote: > > Dear Joachim, > > On 15/08/2018 11:29, Joachim Hein wrote: >> Hi, >> I noticed https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=F_o0YzRqd_v2kIt6n3YbPE5TtaGQTY8_dRqXxX2Ftrw= and https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6703=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=Zk_4ID_w-2HttuYo5o-SHbAl3fJh451rV9XWaekA9K8= >> I am presently working on moving python packages to foss and intel 2018b (h5py was merged, so did boost with python). Could there please be a discussion/statement how we want to handle this? >> Also, we need to think how we want to share such important info in our community? Discussion in slack seems not to be working (the channel is to busy in my view). Stumbling about stuff on GitHub is in my view not the best way to share info like that. >> Thanks for reading. > > I was planning to also share this on the mailing list, but you beat me to it. > > The plan is outlined at https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_issues_6682-23issuecomment-2D413003200=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=C1U2AeaGCPl-6qF6BV2EPABGwMWAuRvne8B7vq8fNxo=, in short: > > * get the easyconfigs for Python 3.6.6 with */2018b merged (see https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_easybuilders_easybuild-2Deasyconfigs_pull_6703=DwIGaQ=eRAMFD45gAfqt84VtBcfhQ=3TR-iteG1SyRqQ5yubQg-_2KIAToz9bj5dZrRdW36Hc=xQJ-rOglYDVpYgP61bnwZCWIsOArG3zhZwq6npzh0HY=Zk_4ID_w-2HttuYo5o-SHbAl3fJh451rV9XWaekA9K8=) > > * rework easyconfigs already in develop to use Python 3.6.6 rather than 3.7.0 > > Kenneth, Thanks. Are we withdrawing 3.7 or adding 3.6.6? There is something to be said for keeping the 3.7.0 but have the 3.6.6 as the preferred and better supported option. Similar to what we have in the GCC space. We often have a conservative GCC in the standard toolchains, but offer more modern versions (e.g. GCC 8.1) as an option with limited support. The question is, how one would communicate that to users … Is it an option to have an EB quick release, that only releases the new Python 3.6.6 (and a few add ons)? Having that in a proper release makes developing python packages (like h5py) so much easier (at least for my workflow). Best wishes joachim > regards, > > Kenneth
Re: [easybuild] Python 3 in 2018b toolchains
> On 15 Aug 2018, at 12:18, Kenneth Hoste wrote: > > Dear Joachim, > > On 15/08/2018 11:29, Joachim Hein wrote: >> Hi, >> I noticed https://github.com/easybuilders/easybuild-easyconfigs/issues/6682 >> and https://github.com/easybuilders/easybuild-easyconfigs/pull/6703 >> I am presently working on moving python packages to foss and intel 2018b >> (h5py was merged, so did boost with python). Could there please be a >> discussion/statement how we want to handle this? >> Also, we need to think how we want to share such important info in our >> community? Discussion in slack seems not to be working (the channel is to >> busy in my view). Stumbling about stuff on GitHub is in my view not the >> best way to share info like that. >> Thanks for reading. > > I was planning to also share this on the mailing list, but you beat me to it. > > The plan is outlined at > https://github.com/easybuilders/easybuild-easyconfigs/issues/6682#issuecomment-413003200, > in short: > > * get the easyconfigs for Python 3.6.6 with */2018b merged (see > https://github.com/easybuilders/easybuild-easyconfigs/pull/6703) > > * rework easyconfigs already in develop to use Python 3.6.6 rather than 3.7.0 > > Kenneth, Thanks. Are we withdrawing 3.7 or adding 3.6.6? There is something to be said for keeping the 3.7.0 but have the 3.6.6 as the preferred and better supported option. Similar to what we have in the GCC space. We often have a conservative GCC in the standard toolchains, but offer more modern versions (e.g. GCC 8.1) as an option with limited support. The question is, how one would communicate that to users … Is it an option to have an EB quick release, that only releases the new Python 3.6.6 (and a few add ons)? Having that in a proper release makes developing python packages (like h5py) so much easier (at least for my workflow). Best wishes joachim > regards, > > Kenneth
Re: [easybuild] Python 3 in 2018b toolchains
Dear Joachim, On 15/08/2018 11:29, Joachim Hein wrote: Hi, I noticed https://github.com/easybuilders/easybuild-easyconfigs/issues/6682 and https://github.com/easybuilders/easybuild-easyconfigs/pull/6703 I am presently working on moving python packages to foss and intel 2018b (h5py was merged, so did boost with python). Could there please be a discussion/statement how we want to handle this? Also, we need to think how we want to share such important info in our community? Discussion in slack seems not to be working (the channel is to busy in my view). Stumbling about stuff on GitHub is in my view not the best way to share info like that. Thanks for reading. I was planning to also share this on the mailing list, but you beat me to it. The plan is outlined at https://github.com/easybuilders/easybuild-easyconfigs/issues/6682#issuecomment-413003200, in short: * get the easyconfigs for Python 3.6.6 with */2018b merged (see https://github.com/easybuilders/easybuild-easyconfigs/pull/6703) * rework easyconfigs already in develop to use Python 3.6.6 rather than 3.7.0 regards, Kenneth
[easybuild] Python 3 in 2018b toolchains
Hi, I noticed https://github.com/easybuilders/easybuild-easyconfigs/issues/6682 and https://github.com/easybuilders/easybuild-easyconfigs/pull/6703 I am presently working on moving python packages to foss and intel 2018b (h5py was merged, so did boost with python). Could there please be a discussion/statement how we want to handle this? Also, we need to think how we want to share such important info in our community? Discussion in slack seems not to be working (the channel is to busy in my view). Stumbling about stuff on GitHub is in my view not the best way to share info like that. Thanks for reading. Best wishes Joachim
