Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Sep 4, 2014, at 3:38 AM, Yuriy Taraday wrote: > On Wed, Sep 3, 2014 at 7:24 PM, Doug Hellmann wrote: > On Sep 3, 2014, at 5:27 AM, Yuriy Taraday wrote: >> On Tue, Sep 2, 2014 at 11:17 PM, Clark Boylan wrote: >> It has been pointed out to me that one case where it won't be so easy is >> oslo.messaging and its use of eventlet under python2. Messaging will >> almost certainly need python 2 and python 3 wheels to be separate. I >> think we should continue to use universal wheels where possible and only >> build python2 and python3 wheels in the special cases where necessary. >> >> We can make eventlet an optional dependency of oslo.messaging (through >> setuptools' extras). In fact I don't quite understand the need for eventlet >> as direct dependency there since we can just write code that uses threading >> library and it'll get monkeypatched if consumer app wants to use eventlet. > > There is code in the messaging library that makes calls directly into > eventlet now, IIRC. It sounds like that could be changed, but that’s > something to consider for a future version. > > Yes, I hope to see unified threading/eventlet executor there (futures-based, > I guess) some day. > > The last time I looked at setuptools extras they were a documented but > unimplemented specification. Has that changed? > > According to docs [1] it works in pip (and has been working in setuptools for > ages), and according to bug [2], it has been working for couple years. Wow, apparently it has been a while since I looked at it. :-) We should look into whether it would make sense to add optional dependencies for some of our drivers. I don’t know how the distros interpret the requirements.txt files and if they can filter out optional items or if it would end up confusing their tool chain. Doug > > [1] http://pip.readthedocs.org/en/latest/reference/pip_install.html#examples > (#6) > [2] https://github.com/pypa/pip/issues/7 > > -- > > Kind regards, Yuriy. > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Thu, Sep 4, 2014 at 4:47 AM, Jeremy Stanley wrote: > On 2014-09-03 13:27:55 +0400 (+0400), Yuriy Taraday wrote: > [...] > > May be we should drop 3.3 already? > > It's in progress. Search review.openstack.org for open changes in > all projects with the topic "py34". Shortly I'll also have some > infra config changes up to switch python33 jobs out for python34, > ready to drop once the j-3 milestone has been tagged and is finally > behind us. > Great! Looking forward to purging python 3.3 from my system. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Wed, Sep 3, 2014 at 8:21 PM, Doug Hellmann wrote: > > On Sep 3, 2014, at 11:57 AM, Clark Boylan wrote: > > On Wed, Sep 3, 2014, at 08:22 AM, Doug Hellmann wrote: > >> > >> On Sep 2, 2014, at 3:17 PM, Clark Boylan wrote: > >>> The setup.cfg classifiers should be able to do that for us, though PBR > >>> may need updating? We will also need to learn to upload potentially >1 > >> > >> How do you see that working? We want all of the Oslo libraries to, > >> eventually, support both python 2 and 3. How would we use the > classifiers > >> to tell when to build a universal wheel and when to build separate > >> wheels? > >> > > The classifiers provide info on the versions of python we support. By > > default we can build python2 wheel if only 2 is supported, build python3 > > wheel if only 3 is supported, build a universal wheel if both are > > supported. Then we can add a setup.cfg flag to override the universal > > wheel default to build both a python2 and python3 wheel instead. Dstufft > > and mordred should probably comment on this idea before we implement > > anything. > > OK. I’m not aware of any python-3-only projects, and the flag to override > the universal wheel is the piece I was missing. I think there’s already a > setuptools flag related to whether or not we should build universal wheels, > isn’t there? > I think we should rely on wheel.universal flag from setup.cfg if it's there. If it's set, we should always build universal wheels. If it's not set, we should look in specifiers and build wheels for Python versions that are mentioned there. -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Wed, Sep 3, 2014 at 7:24 PM, Doug Hellmann wrote: > > On Sep 3, 2014, at 5:27 AM, Yuriy Taraday wrote: > > On Tue, Sep 2, 2014 at 11:17 PM, Clark Boylan > wrote: > >> It has been pointed out to me that one case where it won't be so easy is >> oslo.messaging and its use of eventlet under python2. Messaging will >> almost certainly need python 2 and python 3 wheels to be separate. I >> think we should continue to use universal wheels where possible and only >> build python2 and python3 wheels in the special cases where necessary. >> > > We can make eventlet an optional dependency of oslo.messaging (through > setuptools' extras). In fact I don't quite understand the need for eventlet > as direct dependency there since we can just write code that uses threading > library and it'll get monkeypatched if consumer app wants to use eventlet. > > > There is code in the messaging library that makes calls directly into > eventlet now, IIRC. It sounds like that could be changed, but that’s > something to consider for a future version. > Yes, I hope to see unified threading/eventlet executor there (futures-based, I guess) some day. The last time I looked at setuptools extras they were a documented but > unimplemented specification. Has that changed? > According to docs [1] it works in pip (and has been working in setuptools for ages), and according to bug [2], it has been working for couple years. [1] http://pip.readthedocs.org/en/latest/reference/pip_install.html#examples (#6) [2] https://github.com/pypa/pip/issues/7 -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On 2014-09-03 13:27:55 +0400 (+0400), Yuriy Taraday wrote: [...] > May be we should drop 3.3 already? It's in progress. Search review.openstack.org for open changes in all projects with the topic "py34". Shortly I'll also have some infra config changes up to switch python33 jobs out for python34, ready to drop once the j-3 milestone has been tagged and is finally behind us. -- Jeremy Stanley ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Sep 3, 2014, at 11:57 AM, Clark Boylan wrote: > On Wed, Sep 3, 2014, at 08:22 AM, Doug Hellmann wrote: >> >> On Sep 2, 2014, at 3:17 PM, Clark Boylan wrote: >> >>> On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote: Hello. Currently for alpha releases of oslo libraries we generate either universal or Python 2.x-only wheels. This presents a problem: we can't adopt alpha releases in projects where Python 3.x is supported and verified in the gate. I've ran into this in change request [1] generated after global-requirements change [2]. There we have oslotest library that can't be built as a universal wheel because of different requirements (mox vs mox3 as I understand is the main difference). Because of that py33 job in [1] failed and we can't bump oslotest version in requirements. I propose to change infra scripts that generate and upload wheels to create py3 wheels as well as py2 wheels for projects that support Python 3.x (we can use setup.cfg classifiers to find that out) but don't support universal wheels. What do you think about that? [1] https://review.openstack.org/117940 [2] https://review.openstack.org/115643 -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> We may find that we will need to have py3k wheels in addition to the >>> existing wheels at some point, but I don't think this use case requires >>> it. If oslo.test needs to support python2 and python3 it should use mox3 >>> in both cases which claims to support python2.6, 2.7 and 3.2. Then you >>> can ship a universal wheel. This should solve the immediate problem. >> >> That sounds like a good solution to that specific issue. It may also >> require changes in the application test suites, but those changes can be >> made as we move them to use oslotest. >> >>> >>> It has been pointed out to me that one case where it won't be so easy is >>> oslo.messaging and its use of eventlet under python2. Messaging will >>> almost certainly need python 2 and python 3 wheels to be separate. I >>> think we should continue to use universal wheels where possible and only >>> build python2 and python3 wheels in the special cases where necessary. >>> >>> The setup.cfg classifiers should be able to do that for us, though PBR >>> may need updating? We will also need to learn to upload potentially >1 >> >> How do you see that working? We want all of the Oslo libraries to, >> eventually, support both python 2 and 3. How would we use the classifiers >> to tell when to build a universal wheel and when to build separate >> wheels? >> > The classifiers provide info on the versions of python we support. By > default we can build python2 wheel if only 2 is supported, build python3 > wheel if only 3 is supported, build a universal wheel if both are > supported. Then we can add a setup.cfg flag to override the universal > wheel default to build both a python2 and python3 wheel instead. Dstufft > and mordred should probably comment on this idea before we implement > anything. OK. I’m not aware of any python-3-only projects, and the flag to override the universal wheel is the piece I was missing. I think there’s already a setuptools flag related to whether or not we should build universal wheels, isn’t there? >> >>> wheel in our wheel jobs. That bit is likely straight foward. The last >>> thing that we need to make sure we do is that we have some testing in >>> place for the special wheels. We currently have the requirements >>> integration test which runs under python2 checking that we can actually >>> install all the things together. This ends up exercising our wheels and >>> checking that they actually work. We don't have a python3 equivalent for >> >> We only know the wheels can be installed. We don’t actually have a test >> that installs our code and runs it any more (devstack uses “develop” mode >> which bypasses some of the installation steps, as we found while fixing >> the recent neutron/pbr issue with a missing config file in their >> packaging instructions). >> > Yup, this would be a second level of testing that we should consider as > well. The post install can it do stuff test. We moved our unittesting > away from using sdist installs in preference for setup.py develop > equivalent which means that our unittests no longer cover some of this > stuff for us. Sorry if I hijacked this thread into a "how do we test > release artifacts?" thread. I wonder if we should have a flag in devstack to control whether we install in develop or “regular” mode? That would let us test real installations, but still have editable versions for local developer systems. >> >>> that job. It may be better to work out some explicit checking of the >>> wheel
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Wed, Sep 3, 2014, at 08:22 AM, Doug Hellmann wrote: > > On Sep 2, 2014, at 3:17 PM, Clark Boylan wrote: > > > On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote: > >> Hello. > >> > >> Currently for alpha releases of oslo libraries we generate either > >> universal > >> or Python 2.x-only wheels. This presents a problem: we can't adopt alpha > >> releases in projects where Python 3.x is supported and verified in the > >> gate. I've ran into this in change request [1] generated after > >> global-requirements change [2]. There we have oslotest library that can't > >> be built as a universal wheel because of different requirements (mox vs > >> mox3 as I understand is the main difference). Because of that py33 job in > >> [1] failed and we can't bump oslotest version in requirements. > >> > >> I propose to change infra scripts that generate and upload wheels to > >> create > >> py3 wheels as well as py2 wheels for projects that support Python 3.x (we > >> can use setup.cfg classifiers to find that out) but don't support > >> universal > >> wheels. What do you think about that? > >> > >> [1] https://review.openstack.org/117940 > >> [2] https://review.openstack.org/115643 > >> > >> -- > >> > >> Kind regards, Yuriy. > >> ___ > >> OpenStack-dev mailing list > >> OpenStack-dev@lists.openstack.org > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > We may find that we will need to have py3k wheels in addition to the > > existing wheels at some point, but I don't think this use case requires > > it. If oslo.test needs to support python2 and python3 it should use mox3 > > in both cases which claims to support python2.6, 2.7 and 3.2. Then you > > can ship a universal wheel. This should solve the immediate problem. > > That sounds like a good solution to that specific issue. It may also > require changes in the application test suites, but those changes can be > made as we move them to use oslotest. > > > > > It has been pointed out to me that one case where it won't be so easy is > > oslo.messaging and its use of eventlet under python2. Messaging will > > almost certainly need python 2 and python 3 wheels to be separate. I > > think we should continue to use universal wheels where possible and only > > build python2 and python3 wheels in the special cases where necessary. > > > > The setup.cfg classifiers should be able to do that for us, though PBR > > may need updating? We will also need to learn to upload potentially >1 > > How do you see that working? We want all of the Oslo libraries to, > eventually, support both python 2 and 3. How would we use the classifiers > to tell when to build a universal wheel and when to build separate > wheels? > The classifiers provide info on the versions of python we support. By default we can build python2 wheel if only 2 is supported, build python3 wheel if only 3 is supported, build a universal wheel if both are supported. Then we can add a setup.cfg flag to override the universal wheel default to build both a python2 and python3 wheel instead. Dstufft and mordred should probably comment on this idea before we implement anything. > > > wheel in our wheel jobs. That bit is likely straight foward. The last > > thing that we need to make sure we do is that we have some testing in > > place for the special wheels. We currently have the requirements > > integration test which runs under python2 checking that we can actually > > install all the things together. This ends up exercising our wheels and > > checking that they actually work. We don't have a python3 equivalent for > > We only know the wheels can be installed. We don’t actually have a test > that installs our code and runs it any more (devstack uses “develop” mode > which bypasses some of the installation steps, as we found while fixing > the recent neutron/pbr issue with a missing config file in their > packaging instructions). > Yup, this would be a second level of testing that we should consider as well. The post install can it do stuff test. We moved our unittesting away from using sdist installs in preference for setup.py develop equivalent which means that our unittests no longer cover some of this stuff for us. Sorry if I hijacked this thread into a "how do we test release artifacts?" thread. > > > that job. It may be better to work out some explicit checking of the > > wheels we produce that applies to both versions of python. I am not > > quite sure how we should approach that yet. > > To fix the asymmetric gating we have between pbr and everything else, > Robert suggested setting up some sort of job to install pbr and then > build and install a package for the project being tested. We already, as > you point out, have a job that does this for all of the projects to test > changes to pbr itself. Maybe we can run the same test under python 2 and > 3 as part of the same job? > And to get that extra level of tested we can run
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Sep 3, 2014, at 5:27 AM, Yuriy Taraday wrote: > On Tue, Sep 2, 2014 at 11:17 PM, Clark Boylan wrote: > On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote: > > Hello. > > > > Currently for alpha releases of oslo libraries we generate either > > universal > > or Python 2.x-only wheels. This presents a problem: we can't adopt alpha > > releases in projects where Python 3.x is supported and verified in the > > gate. I've ran into this in change request [1] generated after > > global-requirements change [2]. There we have oslotest library that can't > > be built as a universal wheel because of different requirements (mox vs > > mox3 as I understand is the main difference). Because of that py33 job in > > [1] failed and we can't bump oslotest version in requirements. > > > > I propose to change infra scripts that generate and upload wheels to > > create > > py3 wheels as well as py2 wheels for projects that support Python 3.x (we > > can use setup.cfg classifiers to find that out) but don't support > > universal > > wheels. What do you think about that? > > > > [1] https://review.openstack.org/117940 > > [2] https://review.openstack.org/115643 > > > > -- > > > > Kind regards, Yuriy. > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > We may find that we will need to have py3k wheels in addition to the > existing wheels at some point, but I don't think this use case requires > it. If oslo.test needs to support python2 and python3 it should use mox3 > in both cases which claims to support python2.6, 2.7 and 3.2. Then you > can ship a universal wheel. This should solve the immediate problem. > > Yes, I think, it's the way to go with oslotest specifically. Created a change > request for this: https://review.openstack.org/118551 > > It has been pointed out to me that one case where it won't be so easy is > oslo.messaging and its use of eventlet under python2. Messaging will > almost certainly need python 2 and python 3 wheels to be separate. I > think we should continue to use universal wheels where possible and only > build python2 and python3 wheels in the special cases where necessary. > > We can make eventlet an optional dependency of oslo.messaging (through > setuptools' extras). In fact I don't quite understand the need for eventlet > as direct dependency there since we can just write code that uses threading > library and it'll get monkeypatched if consumer app wants to use eventlet. There is code in the messaging library that makes calls directly into eventlet now, IIRC. It sounds like that could be changed, but that’s something to consider for a future version. The last time I looked at setuptools extras they were a documented but unimplemented specification. Has that changed? > > The setup.cfg classifiers should be able to do that for us, though PBR > may need updating? > > I don't think so - it loads all classifiers from setup.cfg, they should be > available through some distutils machinery. > > We will also need to learn to upload potentially >1 > wheel in our wheel jobs. That bit is likely straight foward. The last > thing that we need to make sure we do is that we have some testing in > place for the special wheels. We currently have the requirements > integration test which runs under python2 checking that we can actually > install all the things together. This ends up exercising our wheels and > checking that they actually work. We don't have a python3 equivalent for > that job. It may be better to work out some explicit checking of the > wheels we produce that applies to both versions of python. I am not > quite sure how we should approach that yet. > > I guess we can just repeat that check with Python 3.x. If I see it right, all > we need is to repeat loop in pbr/tools/integration.sh with different Python > version. The problem might occur that we'll be running this test with Python > 3.4 that is default on trusty but all our unittests jobs run on 3.3 instead. > May be we should drop 3.3 already? > > -- > > Kind regards, Yuriy. > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Sep 2, 2014, at 3:17 PM, Clark Boylan wrote: > On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote: >> Hello. >> >> Currently for alpha releases of oslo libraries we generate either >> universal >> or Python 2.x-only wheels. This presents a problem: we can't adopt alpha >> releases in projects where Python 3.x is supported and verified in the >> gate. I've ran into this in change request [1] generated after >> global-requirements change [2]. There we have oslotest library that can't >> be built as a universal wheel because of different requirements (mox vs >> mox3 as I understand is the main difference). Because of that py33 job in >> [1] failed and we can't bump oslotest version in requirements. >> >> I propose to change infra scripts that generate and upload wheels to >> create >> py3 wheels as well as py2 wheels for projects that support Python 3.x (we >> can use setup.cfg classifiers to find that out) but don't support >> universal >> wheels. What do you think about that? >> >> [1] https://review.openstack.org/117940 >> [2] https://review.openstack.org/115643 >> >> -- >> >> Kind regards, Yuriy. >> ___ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > We may find that we will need to have py3k wheels in addition to the > existing wheels at some point, but I don't think this use case requires > it. If oslo.test needs to support python2 and python3 it should use mox3 > in both cases which claims to support python2.6, 2.7 and 3.2. Then you > can ship a universal wheel. This should solve the immediate problem. That sounds like a good solution to that specific issue. It may also require changes in the application test suites, but those changes can be made as we move them to use oslotest. > > It has been pointed out to me that one case where it won't be so easy is > oslo.messaging and its use of eventlet under python2. Messaging will > almost certainly need python 2 and python 3 wheels to be separate. I > think we should continue to use universal wheels where possible and only > build python2 and python3 wheels in the special cases where necessary. > > The setup.cfg classifiers should be able to do that for us, though PBR > may need updating? We will also need to learn to upload potentially >1 How do you see that working? We want all of the Oslo libraries to, eventually, support both python 2 and 3. How would we use the classifiers to tell when to build a universal wheel and when to build separate wheels? > wheel in our wheel jobs. That bit is likely straight foward. The last > thing that we need to make sure we do is that we have some testing in > place for the special wheels. We currently have the requirements > integration test which runs under python2 checking that we can actually > install all the things together. This ends up exercising our wheels and > checking that they actually work. We don't have a python3 equivalent for We only know the wheels can be installed. We don’t actually have a test that installs our code and runs it any more (devstack uses “develop” mode which bypasses some of the installation steps, as we found while fixing the recent neutron/pbr issue with a missing config file in their packaging instructions). > that job. It may be better to work out some explicit checking of the > wheels we produce that applies to both versions of python. I am not > quite sure how we should approach that yet. To fix the asymmetric gating we have between pbr and everything else, Robert suggested setting up some sort of job to install pbr and then build and install a package for the project being tested. We already, as you point out, have a job that does this for all of the projects to test changes to pbr itself. Maybe we can run the same test under python 2 and 3 as part of the same job? Doug > > Clark > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Tue, Sep 2, 2014 at 11:17 PM, Clark Boylan wrote: > On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote: > > Hello. > > > > Currently for alpha releases of oslo libraries we generate either > > universal > > or Python 2.x-only wheels. This presents a problem: we can't adopt alpha > > releases in projects where Python 3.x is supported and verified in the > > gate. I've ran into this in change request [1] generated after > > global-requirements change [2]. There we have oslotest library that can't > > be built as a universal wheel because of different requirements (mox vs > > mox3 as I understand is the main difference). Because of that py33 job in > > [1] failed and we can't bump oslotest version in requirements. > > > > I propose to change infra scripts that generate and upload wheels to > > create > > py3 wheels as well as py2 wheels for projects that support Python 3.x (we > > can use setup.cfg classifiers to find that out) but don't support > > universal > > wheels. What do you think about that? > > > > [1] https://review.openstack.org/117940 > > [2] https://review.openstack.org/115643 > > > > -- > > > > Kind regards, Yuriy. > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > We may find that we will need to have py3k wheels in addition to the > existing wheels at some point, but I don't think this use case requires > it. If oslo.test needs to support python2 and python3 it should use mox3 > in both cases which claims to support python2.6, 2.7 and 3.2. Then you > can ship a universal wheel. This should solve the immediate problem. > Yes, I think, it's the way to go with oslotest specifically. Created a change request for this: https://review.openstack.org/118551 It has been pointed out to me that one case where it won't be so easy is > oslo.messaging and its use of eventlet under python2. Messaging will > almost certainly need python 2 and python 3 wheels to be separate. I > think we should continue to use universal wheels where possible and only > build python2 and python3 wheels in the special cases where necessary. > We can make eventlet an optional dependency of oslo.messaging (through setuptools' extras). In fact I don't quite understand the need for eventlet as direct dependency there since we can just write code that uses threading library and it'll get monkeypatched if consumer app wants to use eventlet. The setup.cfg classifiers should be able to do that for us, though PBR > may need updating? I don't think so - it loads all classifiers from setup.cfg, they should be available through some distutils machinery. We will also need to learn to upload potentially >1 > wheel in our wheel jobs. That bit is likely straight foward. The last > thing that we need to make sure we do is that we have some testing in > place for the special wheels. We currently have the requirements > integration test which runs under python2 checking that we can actually > install all the things together. This ends up exercising our wheels and > checking that they actually work. We don't have a python3 equivalent for > that job. It may be better to work out some explicit checking of the > wheels we produce that applies to both versions of python. I am not > quite sure how we should approach that yet. > I guess we can just repeat that check with Python 3.x. If I see it right, all we need is to repeat loop in pbr/tools/integration.sh with different Python version. The problem might occur that we'll be running this test with Python 3.4 that is default on trusty but all our unittests jobs run on 3.3 instead. May be we should drop 3.3 already? -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote: > Hello. > > Currently for alpha releases of oslo libraries we generate either > universal > or Python 2.x-only wheels. This presents a problem: we can't adopt alpha > releases in projects where Python 3.x is supported and verified in the > gate. I've ran into this in change request [1] generated after > global-requirements change [2]. There we have oslotest library that can't > be built as a universal wheel because of different requirements (mox vs > mox3 as I understand is the main difference). Because of that py33 job in > [1] failed and we can't bump oslotest version in requirements. > > I propose to change infra scripts that generate and upload wheels to > create > py3 wheels as well as py2 wheels for projects that support Python 3.x (we > can use setup.cfg classifiers to find that out) but don't support > universal > wheels. What do you think about that? > > [1] https://review.openstack.org/117940 > [2] https://review.openstack.org/115643 > > -- > > Kind regards, Yuriy. > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev We may find that we will need to have py3k wheels in addition to the existing wheels at some point, but I don't think this use case requires it. If oslo.test needs to support python2 and python3 it should use mox3 in both cases which claims to support python2.6, 2.7 and 3.2. Then you can ship a universal wheel. This should solve the immediate problem. It has been pointed out to me that one case where it won't be so easy is oslo.messaging and its use of eventlet under python2. Messaging will almost certainly need python 2 and python 3 wheels to be separate. I think we should continue to use universal wheels where possible and only build python2 and python3 wheels in the special cases where necessary. The setup.cfg classifiers should be able to do that for us, though PBR may need updating? We will also need to learn to upload potentially >1 wheel in our wheel jobs. That bit is likely straight foward. The last thing that we need to make sure we do is that we have some testing in place for the special wheels. We currently have the requirements integration test which runs under python2 checking that we can actually install all the things together. This ends up exercising our wheels and checking that they actually work. We don't have a python3 equivalent for that job. It may be better to work out some explicit checking of the wheels we produce that applies to both versions of python. I am not quite sure how we should approach that yet. Clark ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo] [infra] Alpha wheels for Python 3.x
Hello. Currently for alpha releases of oslo libraries we generate either universal or Python 2.x-only wheels. This presents a problem: we can't adopt alpha releases in projects where Python 3.x is supported and verified in the gate. I've ran into this in change request [1] generated after global-requirements change [2]. There we have oslotest library that can't be built as a universal wheel because of different requirements (mox vs mox3 as I understand is the main difference). Because of that py33 job in [1] failed and we can't bump oslotest version in requirements. I propose to change infra scripts that generate and upload wheels to create py3 wheels as well as py2 wheels for projects that support Python 3.x (we can use setup.cfg classifiers to find that out) but don't support universal wheels. What do you think about that? [1] https://review.openstack.org/117940 [2] https://review.openstack.org/115643 -- Kind regards, Yuriy. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
