Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-10 Thread Robert Collins
On 5 January 2014 02:02, Sean Dague s...@dague.net wrote: So we used to do that the apps against release libraries. And the result was more and more full day gate breaks. We did 2 consecutive ones in 2 weeks. Basically, once you get to be a certain level of coupled in OpenStack we can no

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-10 Thread Sean Dague
On 01/10/2014 04:13 AM, Robert Collins wrote: On 5 January 2014 02:02, Sean Dague s...@dague.net wrote: So we used to do that the apps against release libraries. And the result was more and more full day gate breaks. We did 2 consecutive ones in 2 weeks. Basically, once you get to be a

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-07 Thread Doug Hellmann
On Mon, Jan 6, 2014 at 6:21 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-01-06 17:23:31 -0500 (-0500), Doug Hellmann wrote: [...] The global requirements syncing seems to have fixed the issue for apps, although it just occurred to me that I'm not sure we check that the requirements

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-06 Thread Doug Hellmann
On Sat, Jan 4, 2014 at 8:02 AM, Sean Dague s...@dague.net wrote: On 01/03/2014 08:27 PM, Robert Collins wrote: On 4 January 2014 08:44, Doug Hellmann doug.hellm...@dreamhost.com wrote: It seems safer to gate changes to libraries against the apps' trunk (to avoid making

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-06 Thread Doug Hellmann
On Sun, Jan 5, 2014 at 5:02 PM, James E. Blair jebl...@openstack.orgwrote: Joshua Harlow harlo...@yahoo-inc.com writes: It seems simple to have variations of venvs (or something similar) that taskflow tox.ini can have that specify the different 0.7, 0.8, 0.9, when sqlalchemy 1.0 comes out

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-06 Thread Jeremy Stanley
On 2014-01-06 17:23:31 -0500 (-0500), Doug Hellmann wrote: [...] The global requirements syncing seems to have fixed the issue for apps, although it just occurred to me that I'm not sure we check that the requirements lists are the same when we cut a release. Do we do that already? Not yet in

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-05 Thread Joshua Harlow
With regards to the futures module it should just work fine with the packaging of https://pypi.python.org/pypi/futures which is a backport of the 3.2 concurrent futures package into 2.6,2.7, so that's the package there. Feel free to bug me on irc if u want any other help with dependencies, the

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-05 Thread Clint Byrum
I've skimmed the rest of the thread and not seen something mentioned that seems like it matters a lot. If I missed this suggestion buried deep in the ensuing discussion, I apologize for that. Since TaskFlow wants to be generally consumable and not only driven as an OpenStack component, it should

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-05 Thread Joshua Harlow
Agreed, we are going to expand it and work on figuring out how to test against multiple versions. It does work with 0.8 and it seems even like 0.9 works fine also. But all compatible also means I can't guarantee 0.10 (if it comes out) will work since afaik semver means sqlalchemy could still

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-05 Thread James E. Blair
Joshua Harlow harlo...@yahoo-inc.com writes: It seems simple to have variations of venvs (or something similar) that taskflow tox.ini can have that specify the different 0.7, 0.8, 0.9, when sqlalchemy 1.0 comes out then this should become a nonissue (hopefully). I will bug the infra folks to

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Sean Dague
On 01/03/2014 08:27 PM, Robert Collins wrote: On 4 January 2014 08:44, Doug Hellmann doug.hellm...@dreamhost.com wrote: It seems safer to gate changes to libraries against the apps' trunk (to avoid making backwards-incompatible changes), and then gate changes to the apps against the released

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Thomas Goirand
Sean, Before everything, I'd like to thank you for insisting in making the transition to SQLA 0.8.x. Since it has been uploaded to Sid, this SQLA 0.7.99 has been without any doubt the biggest reoccurring pain in the but with the packaging of OpenStack. Without people like you, insisting again

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Thomas Goirand
On 01/04/2014 07:53 AM, Joshua Harlow wrote: So another idea that was talked about on IRC. Taskflow exposes entrypoints for these storage backends (like your storage callback/interface idea). It currently provides 3 such 'default' backends [sqlalchemy, file/dir based, in-memory -- mainly

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Joshua Harlow
Such a bad state seems like FUD. Taskflow was just syncing its requirements with the same requirements that everyone else is... Those global requirements have 0.7.99 in them as we speak (which is why taskflow picked up that version). The issue here will be worked through and fixed, it won't

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Joshua Harlow
I was more of referring to general dependency issues, sqlalchemy hopefully never again but one never knows :P Sent from my really tiny device... On Jan 4, 2014, at 8:40 AM, Thomas Goirand z...@debian.org wrote: On 01/05/2014 12:12 AM, Joshua Harlow wrote: it won't be the last time a

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Robert Collins
On 5 January 2014 04:22, Thomas Goirand z...@debian.org wrote: Sean, Before everything, I'd like to thank you for insisting in making the transition to SQLA 0.8.x. Since it has been uploaded to Sid, this SQLA 0.7.99 has been without any doubt the biggest reoccurring pain in the but with the

[openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
Given that sqla 0.9 just came out, I wanted to explore, again, what the state of the world was with sqla 0.8 (especially given that Ubuntu and Red Hat are both shipping 0.8 in their OpenStack bundles) - https://review.openstack.org/#/c/64831/ The answer is not great. But more importantly, the

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
So taskflow was tested with the version of sqlalchemy that was available and in the requirements at the time of its 0.1 release (taskflow syncs it's requirements from the same global requirements). From what I remember this is the same requirement that everyone else is bound to:

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 11:37 AM, Joshua Harlow wrote: So taskflow was tested with the version of sqlalchemy that was available and in the requirements at the time of its 0.1 release (taskflow syncs it's requirements from the same global requirements). From what I remember this is the same requirement

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
Ok, I think I'm fine with that (although not really sure what that entails). What does the living under the 'oslo program' change? Does that entail getting sucked into the incubator (which seems to be what your graduating link is about). I don't think its a good idea for taskflow to be in the

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 12:45 PM, Joshua Harlow wrote: Ok, I think I'm fine with that (although not really sure what that entails). What does the living under the 'oslo program' change? Does that entail getting sucked into the incubator (which seems to be what your graduating link is about). I don't

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
Sounds good to me. Talked on #openstack-infra with some folks there and just awaiting next steps. Doesn't seem like should be anything to hard to adjust/move/... -Josh On 1/3/14, 11:27 AM, Sean Dague s...@dague.net wrote: On 01/03/2014 12:45 PM, Joshua Harlow wrote: Ok, I think I'm fine

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Doug Hellmann
On Fri, Jan 3, 2014 at 12:45 PM, Joshua Harlow harlo...@yahoo-inc.comwrote: Ok, I think I'm fine with that (although not really sure what that entails). What does the living under the 'oslo program' change? Does that entail getting sucked into the incubator (which seems to be what your

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org, Sean Dague s...@dague.netmailto:s...@dague.net Subject: Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade On Fri, Jan 3, 2014 at 12:45

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 02:44 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 12:45 PM, Joshua Harlow harlo...@yahoo-inc.com mailto:harlo...@yahoo-inc.com wrote: Ok, I think I'm fine with that (although not really sure what that entails). What does the living under the 'oslo program'

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Jeremy Stanley
On 2014-01-03 14:44:40 -0500 (-0500), Doug Hellmann wrote: [...] It seems safer to gate changes to libraries against the apps' trunk (to avoid making backwards-incompatible changes), and then gate changes to the apps against the released libraries (to ensure they work with something available

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Doug Hellmann
On Fri, Jan 3, 2014 at 3:13 PM, Sean Dague s...@dague.net wrote: On 01/03/2014 02:44 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 12:45 PM, Joshua Harlow harlo...@yahoo-inc.com mailto:harlo...@yahoo-inc.com wrote: Ok, I think I'm fine with that (although not really sure what that

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 03:30 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 3:13 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 01/03/2014 02:44 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 12:45 PM, Joshua Harlow harlo...@yahoo-inc.com

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Doug Hellmann
On Fri, Jan 3, 2014 at 4:08 PM, Sean Dague s...@dague.net wrote: On 01/03/2014 03:30 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 3:13 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 01/03/2014 02:44 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 12:45

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 04:17 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 4:08 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 01/03/2014 03:30 PM, Doug Hellmann wrote: On Fri, Jan 3, 2014 at 3:13 PM, Sean Dague s...@dague.net mailto:s...@dague.net

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Ivan Melnikov
On 04.01.2014 01:29, Sean Dague wrote: On 01/03/2014 04:17 PM, Doug Hellmann wrote: [...] That's what made me think of the solution. But isn't setuptools in fact telling us that somehow the versions of things we expected to have installed are no longer installed and so something *is* broken

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 05:10 PM, Ivan Melnikov wrote: On 04.01.2014 01:29, Sean Dague wrote: On 01/03/2014 04:17 PM, Doug Hellmann wrote: [...] That's what made me think of the solution. But isn't setuptools in fact telling us that somehow the versions of things we expected to have installed are no

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
Since the library model is what most everyone else uses outside of openstack (I assume?) what can we do to get there so that this model works as well? Expanding dependencies recursively seems like it could help? This could then detect transitive dependency issues (and doesn't seem so hard to do).

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Sean Dague
On 01/03/2014 06:14 PM, Joshua Harlow wrote: Since the library model is what most everyone else uses outside of openstack (I assume?) what can we do to get there so that this model works as well? Expanding dependencies recursively seems like it could help? This could then detect transitive

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
So another idea that was talked about on IRC. Taskflow exposes entrypoints for these storage backends (like your storage callback/interface idea). It currently provides 3 such 'default' backends [sqlalchemy, file/dir based, in-memory -- mainly for testing]. A 4th one is in progress for icehouse