Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
There're still keystone client conflict issues: https://review.openstack.org/#/c/36684/ It seems uncapping keystone client(remove upper bound) in some else project first is needed? On Fri, Jul 12, 2013 at 7:25 PM, Sean Dague s...@dague.net wrote: We are working towards uncapping all the clients, with the exception of neutron client, because they need to make some incompatible changes on their next major release. On 07/12/2013 12:12 AM, Gareth wrote: so, what's the final conclusion about this issue? On Fri, Jul 12, 2013 at 12:11 PM, Gareth academicgar...@gmail.com mailto:academicgareth@gmail.**com academicgar...@gmail.com wrote: Thanks, Monty but in my review https://review.openstack.org/#**/c/36684/https://review.openstack.org/#/c/36684/, Doug said we will go without upper bound with those python-*clients and in this one https://review.openstack.org/#**/c/36753/https://review.openstack.org/#/c/36753/, keystoneclient still keep '0.4' and requirements test doesn't fail in keystoneclient (https://jenkins.openstack.**org/job/gate-cinder-** requirements/96/consolehttps://jenkins.openstack.org/job/gate-cinder-requirements/96/console it failed on glanceclient) On Fri, Jul 12, 2013 at 11:54 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: On 07/11/2013 11:38 PM, Gareth wrote: I heard there's a talk about this issue in #openstack-infra last night (china standard time), what's the conclusion of that? BTW, how to find meeting log of #openstack-infra? I didn't find it in http://eavesdrop.openstack.**org/http://eavesdrop.openstack.org/ We don't log it currently. There is a wider conversation going on about which things we should log and which things we should not log ... but for the time being I've submitted this: https://review.openstack.org/**36773https://review.openstack.org/36773 to add -infra. I think we talk about enough things that have ramifications on everyone in there that we should really capture it. On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller d...@dmllr.de mailto:d...@dmllr.de mailto:d...@dmllr.de mailto:d...@dmllr.de wrote: See for example https://bugs.launchpad.net/**horizon/+bug/1196823https://bugs.launchpad.net/horizon/+bug/1196823 This is arguably a deficiency of mox, which (apparently?) doesn't let us mock properties automatically. I agree, but it is just one example. other test-only issues can happen as well. Similar problem: the *client packages are not self-contained, they have pretty strict dependencies on other packages. One case I already run into was a dependency on python-requests: newer python-*client packages (rightfully) require requests = 1.x. running those on a system that has OpenStack services from Grizzly or Folsom installed cause a conflict: there are one or two that require requests to be 1.0. When you run gating on this scenario, I think the same flipping would happen on e.g. requests as well, due to *client or the module being installed in varying order. Greetings, Dirk __**_ OpenStack-dev mailing list OpenStack-dev@lists.openstack.**orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org http://lists.openstack.org/**cgi-bin/mailman/listinfo/** openstack-devhttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth /Cloud Computing, OpenStack, Fitness, Basketball/ /OpenStack contributor/ /Company: UnitedStack http://www.ustack.com/ /My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me / /and I'll donate $1 or ￥1 to an open organization you specify./ __**_ OpenStack-dev mailing list OpenStack-dev@lists.openstack.**orgOpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.**openstack.orgOpenStack-dev@lists.openstack.org
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
On 07/11/2013 05:06 AM, Thierry Carrez wrote: Sean Dague wrote: I think we need to get strict on projects and prevent them from capping their client requirements. That will also put burden on clients that they don't break backwards compatibility (which I think was a goal regardless). Indeed. The whole idea behind a single release channel for python client libraries was that you should always be running the latest, as they should drastically enforce backward compatibility. Any reason why those caps were introduced in the first place ? Well global requirements specifies caps for most clients: python-cinderclient=1.0.4,2 python-ceilometerclient=1.0.1 python-heatclient=0.2.2 python-glanceclient=0.9.0,2 python-keystoneclient=0.2.1,0.4 python-memcached python-neutronclient=2.2.3,3.0.0 python-novaclient=2.12.0,3 python-quantumclient=2.2.0,3.0.0 python-swiftclient=1.2,2 I assume projects just copied those lines into their requirements. Then keystoneclient bumped release number, and got outside the boundary that was allowed by some project. I know a flury of python-keystoneclient patches went in after python-keystoneclient 0.3.0 released, but has a broken compatibility issue. So step one is purge from global requirements. Step two purge from projects. Step three enforce they don't come back. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
Let's submit a multi-project bug on launchpad, and be serious for changing these global requirements in following days https://bugs.launchpad.net/keystone/+bug/1200214 created. Greetings, Dirk ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
Hi Thierry, Indeed. The whole idea behind a single release channel for python client libraries was that you should always be running the latest, as they should drastically enforce backward compatibility. Well, backward compatibility can be tricky when it comes to test. We've for example recently had an issue where the newer keystoneclient broke mocking in tests. It is debateable if tests are part of the backward compatibility or not. See for example https://bugs.launchpad.net/horizon/+bug/1196823 This is currently also preventing me from being able to get a change on stable/grizzly past gating checks (which stumble on exactly this regression). Greetings, Dirk ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
On 07/11/2013 09:12 AM, Dirk Müller wrote: Let's submit a multi-project bug on launchpad, and be serious for changing these global requirements in following days https://bugs.launchpad.net/keystone/+bug/1200214 Great! This is the first review we need to land to make progress: https://review.openstack.org/#/c/36631/ -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
I heard there's a talk about this issue in #openstack-infra last night (china standard time), what's the conclusion of that? BTW, how to find meeting log of #openstack-infra? I didn't find it in http://eavesdrop.openstack.org/ On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller d...@dmllr.de wrote: See for example https://bugs.launchpad.net/horizon/+bug/1196823 This is arguably a deficiency of mox, which (apparently?) doesn't let us mock properties automatically. I agree, but it is just one example. other test-only issues can happen as well. Similar problem: the *client packages are not self-contained, they have pretty strict dependencies on other packages. One case I already run into was a dependency on python-requests: newer python-*client packages (rightfully) require requests = 1.x. running those on a system that has OpenStack services from Grizzly or Folsom installed cause a conflict: there are one or two that require requests to be 1.0. When you run gating on this scenario, I think the same flipping would happen on e.g. requests as well, due to *client or the module being installed in varying order. Greetings, Dirk ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth *Cloud Computing, OpenStack, Fitness, Basketball* *OpenStack contributor* *Company: UnitedStack http://www.ustack.com* *My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me * *and I'll donate $1 or ￥1 to an open organization you specify.* ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
On 07/11/2013 11:38 PM, Gareth wrote: I heard there's a talk about this issue in #openstack-infra last night (china standard time), what's the conclusion of that? BTW, how to find meeting log of #openstack-infra? I didn't find it in http://eavesdrop.openstack.org/ We don't log it currently. There is a wider conversation going on about which things we should log and which things we should not log ... but for the time being I've submitted this: https://review.openstack.org/36773 to add -infra. I think we talk about enough things that have ramifications on everyone in there that we should really capture it. On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller d...@dmllr.de mailto:d...@dmllr.de wrote: See for example https://bugs.launchpad.net/horizon/+bug/1196823 This is arguably a deficiency of mox, which (apparently?) doesn't let us mock properties automatically. I agree, but it is just one example. other test-only issues can happen as well. Similar problem: the *client packages are not self-contained, they have pretty strict dependencies on other packages. One case I already run into was a dependency on python-requests: newer python-*client packages (rightfully) require requests = 1.x. running those on a system that has OpenStack services from Grizzly or Folsom installed cause a conflict: there are one or two that require requests to be 1.0. When you run gating on this scenario, I think the same flipping would happen on e.g. requests as well, due to *client or the module being installed in varying order. Greetings, Dirk ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth /Cloud Computing, OpenStack, Fitness, Basketball/ /OpenStack contributor/ /Company: UnitedStack http://www.ustack.com/ /My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me / /and I'll donate $1 or ￥1 to an open organization you specify./ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
Thanks, Monty but in my review https://review.openstack.org/#/c/36684/ , Doug said we will go without upper bound with those python-*clients and in this one https://review.openstack.org/#/c/36753/ , keystoneclient still keep '0.4' and requirements test doesn't fail in keystoneclient ( https://jenkins.openstack.org/job/gate-cinder-requirements/96/console it failed on glanceclient) On Fri, Jul 12, 2013 at 11:54 AM, Monty Taylor mord...@inaugust.com wrote: On 07/11/2013 11:38 PM, Gareth wrote: I heard there's a talk about this issue in #openstack-infra last night (china standard time), what's the conclusion of that? BTW, how to find meeting log of #openstack-infra? I didn't find it in http://eavesdrop.openstack.org/ We don't log it currently. There is a wider conversation going on about which things we should log and which things we should not log ... but for the time being I've submitted this: https://review.openstack.org/36773 to add -infra. I think we talk about enough things that have ramifications on everyone in there that we should really capture it. On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller d...@dmllr.de mailto:d...@dmllr.de wrote: See for example https://bugs.launchpad.net/horizon/+bug/1196823 This is arguably a deficiency of mox, which (apparently?) doesn't let us mock properties automatically. I agree, but it is just one example. other test-only issues can happen as well. Similar problem: the *client packages are not self-contained, they have pretty strict dependencies on other packages. One case I already run into was a dependency on python-requests: newer python-*client packages (rightfully) require requests = 1.x. running those on a system that has OpenStack services from Grizzly or Folsom installed cause a conflict: there are one or two that require requests to be 1.0. When you run gating on this scenario, I think the same flipping would happen on e.g. requests as well, due to *client or the module being installed in varying order. Greetings, Dirk ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth /Cloud Computing, OpenStack, Fitness, Basketball/ /OpenStack contributor/ /Company: UnitedStack http://www.ustack.com/ /My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me / /and I'll donate $1 or ￥1 to an open organization you specify./ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth *Cloud Computing, OpenStack, Fitness, Basketball* *OpenStack contributor* *Company: UnitedStack http://www.ustack.com* *My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me * *and I'll donate $1 or ￥1 to an open organization you specify.* ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
so, what's the final conclusion about this issue? On Fri, Jul 12, 2013 at 12:11 PM, Gareth academicgar...@gmail.com wrote: Thanks, Monty but in my review https://review.openstack.org/#/c/36684/ , Doug said we will go without upper bound with those python-*clients and in this one https://review.openstack.org/#/c/36753/ , keystoneclient still keep '0.4' and requirements test doesn't fail in keystoneclient ( https://jenkins.openstack.org/job/gate-cinder-requirements/96/console it failed on glanceclient) On Fri, Jul 12, 2013 at 11:54 AM, Monty Taylor mord...@inaugust.comwrote: On 07/11/2013 11:38 PM, Gareth wrote: I heard there's a talk about this issue in #openstack-infra last night (china standard time), what's the conclusion of that? BTW, how to find meeting log of #openstack-infra? I didn't find it in http://eavesdrop.openstack.org/ We don't log it currently. There is a wider conversation going on about which things we should log and which things we should not log ... but for the time being I've submitted this: https://review.openstack.org/36773 to add -infra. I think we talk about enough things that have ramifications on everyone in there that we should really capture it. On Thu, Jul 11, 2013 at 11:35 PM, Dirk Müller d...@dmllr.de mailto:d...@dmllr.de wrote: See for example https://bugs.launchpad.net/horizon/+bug/1196823 This is arguably a deficiency of mox, which (apparently?) doesn't let us mock properties automatically. I agree, but it is just one example. other test-only issues can happen as well. Similar problem: the *client packages are not self-contained, they have pretty strict dependencies on other packages. One case I already run into was a dependency on python-requests: newer python-*client packages (rightfully) require requests = 1.x. running those on a system that has OpenStack services from Grizzly or Folsom installed cause a conflict: there are one or two that require requests to be 1.0. When you run gating on this scenario, I think the same flipping would happen on e.g. requests as well, due to *client or the module being installed in varying order. Greetings, Dirk ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth /Cloud Computing, OpenStack, Fitness, Basketball/ /OpenStack contributor/ /Company: UnitedStack http://www.ustack.com/ /My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me / /and I'll donate $1 or ￥1 to an open organization you specify./ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Gareth *Cloud Computing, OpenStack, Fitness, Basketball* *OpenStack contributor* *Company: UnitedStack http://www.ustack.com* *My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me * *and I'll donate $1 or ￥1 to an open organization you specify.* -- Gareth *Cloud Computing, OpenStack, Fitness, Basketball* *OpenStack contributor* *Company: UnitedStack http://www.ustack.com* *My promise: if you find any spelling or grammar mistakes in my email from Mar 1 2013, notify me * *and I'll donate $1 or ￥1 to an open organization you specify.* ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
On Wednesday, July 10, 2013, Sean Dague wrote: Yesterday in the very exciting run around to figure out why the gate was broken, we realized something interesting. Because of the way the gate process pip requirements (one project at a time), on a current gate run we actually install and uninstall python-keystoneclient 4 times in a normal run, flipping back and forth from HEAD to 0.2.5. http://paste.openstack.org/**show/39880/http://paste.openstack.org/show/39880/- shows what's going on The net of this means that if any of the projects specify a capped client, it has the potential for preventing that client from being tested in the gate. This is very possibly part of the reason we ended up with a broken python-keystoneclient 0.3.0 released. I think we need to get strict on projects and prevent them from capping their client requirements. That will also put burden on clients that they don't break backwards compatibility (which I think was a goal regardless). However there is probably going to be a bit of pain getting from where we are today, to this world. Thanks for investigating the underlying issue! I think the same policy should apply a bit further to any code we develop and consume ourselves as a community (oslo.config, etc). I have no doubt that's the standard we strive for, but it's all too easy to throw a cap into a requirements file and forget about it. This is both a heads up, and a time for discussion, before we start figuring out how to make this better in the gate. -Sean -- Sean Dague http://dague.net __**_ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/**cgi-bin/mailman/listinfo/**openstack-devhttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -Dolph ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] The danger of capping python-*clients in core projects, and forbidding it in the future
A useful tool that anvil has built into it (thanks to aababilov). It might be useful in this situation. https://github.com/stackforge/anvil/tree/master/tools#multipip It might be useful to use said tool (or a derivative) to detect this kind of version conflict earlier rather than later?? It is used in anvil to determine the same set of conflicts so that later when multiple single packages of openstack (say of a given release) that said multiple packages will all play nicely together (at least with respect to pip requirement versioning). -Josh On 7/10/13 2:42 PM, Sean Dague s...@dague.net wrote: Yesterday in the very exciting run around to figure out why the gate was broken, we realized something interesting. Because of the way the gate process pip requirements (one project at a time), on a current gate run we actually install and uninstall python-keystoneclient 4 times in a normal run, flipping back and forth from HEAD to 0.2.5. http://paste.openstack.org/show/39880/ - shows what's going on The net of this means that if any of the projects specify a capped client, it has the potential for preventing that client from being tested in the gate. This is very possibly part of the reason we ended up with a broken python-keystoneclient 0.3.0 released. I think we need to get strict on projects and prevent them from capping their client requirements. That will also put burden on clients that they don't break backwards compatibility (which I think was a goal regardless). However there is probably going to be a bit of pain getting from where we are today, to this world. This is both a heads up, and a time for discussion, before we start figuring out how to make this better in the gate. -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
