Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On Wed, Jan 21, 2015 at 5:03 AM, Sean Dague s...@dague.net wrote: On 01/20/2015 08:15 PM, Robert Collins wrote: On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. Well, I think Clark and I have different opinions of how much of a pain unwinding the requirements are, and how long these tend to leave the gate broken. I am happy to also put it in a somebody elses problem field for resolving the issues. :) Honestly, I think we're actually at a different point, where we need to stop assuming that the sane way to deal with python is to install it into system libraries, and just put every service in a venv and get rid of global requirements entirely. Global requirements was a scaling fix for getting to 10 coexisting projects. I don't think it actually works well with 50 ecosystem projects. Which is why I proposed the domains solution instead. ++ using per service virtual environments would help us avoid a whole class of nasty issues. On the flip side doing this makes things harder for distros to find a set of non-conflicting dependencies etc. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. Fully agreed, I am looking into avoiding pips dependency solver for stable branches only right now. But using per service venvs would be even better. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
TripleO has done per service venvs for a couple years now, and it doesn't solve the fragility issue that our unbounded deps cause. It avoids most but not all conflicting deps within OpenStack, and none of the 'upstream broke us' cases. -Rob On 27 January 2015 at 09:01, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jan 21, 2015 at 5:03 AM, Sean Dague s...@dague.net wrote: On 01/20/2015 08:15 PM, Robert Collins wrote: On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. Well, I think Clark and I have different opinions of how much of a pain unwinding the requirements are, and how long these tend to leave the gate broken. I am happy to also put it in a somebody elses problem field for resolving the issues. :) Honestly, I think we're actually at a different point, where we need to stop assuming that the sane way to deal with python is to install it into system libraries, and just put every service in a venv and get rid of global requirements entirely. Global requirements was a scaling fix for getting to 10 coexisting projects. I don't think it actually works well with 50 ecosystem projects. Which is why I proposed the domains solution instead. ++ using per service virtual environments would help us avoid a whole class of nasty issues. On the flip side doing this makes things harder for distros to find a set of non-conflicting dependencies etc. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. Fully agreed, I am looking into avoiding pips dependency solver for stable branches only right now. But using per service venvs would be even better. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 01/26/2015 02:29 PM, Robert Collins wrote: TripleO has done per service venvs for a couple years now, and it doesn't solve the fragility issue that our unbounded deps cause. It avoids most but not all conflicting deps within OpenStack, and none of the 'upstream broke us' cases. Note that a lot of us (our CI included) are no longer running with separate venvs due to the time it takes to build them. See common-venv from http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_gate_test.sh#n21 Also, I don't see how this could work with distro packages (I'm not saying there isn't a way, but I don't know what it would be), so I think that's something I would definitely want an answer on before we decide this is the way forward. -Ben -Rob On 27 January 2015 at 09:01, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jan 21, 2015 at 5:03 AM, Sean Dague s...@dague.net wrote: On 01/20/2015 08:15 PM, Robert Collins wrote: On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. Well, I think Clark and I have different opinions of how much of a pain unwinding the requirements are, and how long these tend to leave the gate broken. I am happy to also put it in a somebody elses problem field for resolving the issues. :) Honestly, I think we're actually at a different point, where we need to stop assuming that the sane way to deal with python is to install it into system libraries, and just put every service in a venv and get rid of global requirements entirely. Global requirements was a scaling fix for getting to 10 coexisting projects. I don't think it actually works well with 50 ecosystem projects. Which is why I proposed the domains solution instead. ++ using per service virtual environments would help us avoid a whole class of nasty issues. On the flip side doing this makes things harder for distros to find a set of non-conflicting dependencies etc. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. Fully agreed, I am looking into avoiding pips dependency solver for stable branches only right now. But using per service venvs would be even better. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 27 January 2015 at 09:56, Clint Byrum cl...@fewbar.com wrote: moved post to bottom for us backwards folk who see the quotes in original order /pedant TripleO has done per service venvs for a couple years now, and it doesn't solve the fragility issue that our unbounded deps cause. It avoids most but not all conflicting deps within OpenStack, and none of the 'upstream broke us' cases. Note that we are not testing per-service venvs anymore because of the extreme high cost of building the controller images with so many separate venvs. We just put the openstack namespaced pieces in one big openstack venv now. Yes, but the experience we have about the limitations of per-service venvs is still relevant, no? Are you really saying 'do per-service venvs because they work well', or are you agreeing with me that they don't solve the problems plaguing the gate? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On Tue, 27 Jan 2015, Robert Collins wrote: Yes, but the experience we have about the limitations of per-service venvs is still relevant, no? Are you really saying 'do per-service venvs because they work well', or are you agreeing with me that they don't solve the problems plaguing the gate? To take this topic even further from its original subject I'd just like to remind at least myself that the reason we have problems in the gate is because there are real problems that actually exist or will exist soon and that catching them before they leak out is is presumably exactly why we have a gate. Given a lot more resources we should always pip -U and never put upper version caps in requirements.txt. Then when stuff breaks do the good work to untangle the mess and fix the problem. Not just for us but for the commonweal. Breaking stuff tends to be how stuff gets fixed. Yes, I'm talking idealistic impractical hooey here, but I think it is important to remember these things. So whereas earlier I said I was keen on virtualenvs, now that I actually think about it some I think we're better off creating a chaotic crucible of everything in one fiery cauldron of truth and using breakage in the gate as a way of marshalling resources. Do I expect to see that happen? Sadly, not really. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
Excerpts from Robert Collins's message of 2015-01-26 12:29:37 -0800: On 27 January 2015 at 09:01, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Jan 21, 2015 at 5:03 AM, Sean Dague s...@dague.net wrote: On 01/20/2015 08:15 PM, Robert Collins wrote: On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. Well, I think Clark and I have different opinions of how much of a pain unwinding the requirements are, and how long these tend to leave the gate broken. I am happy to also put it in a somebody elses problem field for resolving the issues. :) Honestly, I think we're actually at a different point, where we need to stop assuming that the sane way to deal with python is to install it into system libraries, and just put every service in a venv and get rid of global requirements entirely. Global requirements was a scaling fix for getting to 10 coexisting projects. I don't think it actually works well with 50 ecosystem projects. Which is why I proposed the domains solution instead. ++ using per service virtual environments would help us avoid a whole class of nasty issues. On the flip side doing this makes things harder for distros to find a set of non-conflicting dependencies etc. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. Fully agreed, I am looking into avoiding pips dependency solver for stable branches only right now. But using per service venvs would be even better. moved post to bottom for us backwards folk who see the quotes in original order TripleO has done per service venvs for a couple years now, and it doesn't solve the fragility issue that our unbounded deps cause. It avoids most but not all conflicting deps within OpenStack, and none of the 'upstream broke us' cases. Note that we are not testing per-service venvs anymore because of the extreme high cost of building the controller images with so many separate venvs. We just put the openstack namespaced pieces in one big openstack venv now. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 22 January 2015 at 02:48, Chris Dent chd...@redhat.com wrote: On Wed, 21 Jan 2015, Sean Dague wrote: On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. Actually buildout is precisely that today - you specify exact versions and get them. pip's inability to duplicate that behaviour is one of the reasons buildout is still a thing; bypassing the resolver to get the same behaviour *for the same reasons!* isn't at all odd. Undesirable perhaps - and we should work on pip upstream too - but not odd. Agreed. Skipping the pip resolver will move OpenStack and friends further down into a weird world of its own rather than Python norms. See above - I disagree. Python norms are -very- broad, and buildout has a healthy ecosystem within Python web shops, for exactly the issues we're trying to solve here. If possible we should try to resolve the complexity, not bandaid the problems caused by the complexity. If we can't do that then per service virtualenv (or even containers) provides a far more contained[1] bandage. Per service virtualenvs are good IMO, but orthogonal to the unbounded thing we have today, which we have largely *because* of pip not having a resolver. (Thats driven by random things getting upgraded because of transitive dependencies - caused by the resolver). Related: I personally find it completely bewildering that things are managed and run in such an unbounded fashion. My intuition is that this is the result of many/most projects being on the same release cycle and the belief that my master must integrate with your master. We should release (far) more often and my master should integrate with your lastest release. Yes, I would like to see that, as a complement to the trunk-trunk tests. We need to know that something we're about to release won't break folk as well as knowing that the thing we're about to release works with the releases of other things that are already out there. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
Clarkb, As Robert said, you are missing the point. I didn't say that Rally wants this lib so it should be in global requirements. I asked only about python clients of stackforge projects that are regarding all rules: (Like py3k support, license, are in projects.txt and so on). From my point of view, if clients are regarding all reules there is no compatibility issues = it is easy to add them to g-r. Am I wrong? All, Sorry I wasn't able to be on meeting yesterday. First of all, we don't have any issues with having Murano/Mistral benchmarks and testing them in gates and supporting them in our tree. (We already have rally-dsvm-murano and rally-dsvm-mistral dsvm jobs that works well) The real issue is very bad user dev experience caused by soft decencies. Let's take a look at actions of typical user (running Murano benchamrk): 1) Try to run tests for Murano 2) Input validation error: MuranoClient is missing please install it 3) WTF??? Let's take a look at actions of end developer (writing Murano benchmark): 1) Try to make new Murano benchmark 2) Need to import some exceptions from Murano client 3) All unit tests are failing.. 4) WTF??? Adding extra steps/conditions that are required for using product are adding exponential complexity to it. So having 5 such steps/conditions will make product inconsumable. So the only thing that I would like is to remove one extra step by adding good python clients to g-r. Best regards, Boris Pavlovic On Wed, Jan 21, 2015 at 4:15 AM, Robert Collins robe...@robertcollins.net wrote: On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 01/20/2015 08:15 PM, Robert Collins wrote: On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. Well, I think Clark and I have different opinions of how much of a pain unwinding the requirements are, and how long these tend to leave the gate broken. I am happy to also put it in a somebody elses problem field for resolving the issues. :) Honestly, I think we're actually at a different point, where we need to stop assuming that the sane way to deal with python is to install it into system libraries, and just put every service in a venv and get rid of global requirements entirely. Global requirements was a scaling fix for getting to 10 coexisting projects. I don't think it actually works well with 50 ecosystem projects. Which is why I proposed the domains solution instead. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On Wed, 21 Jan 2015, Sean Dague wrote: On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. I think if we are talking seriously about bypassing the pip resolver, we should step back and think about that fact. Because now we're producting a custom installation process that will produce an answer for us, which is completely different than any answer that anyone else is getting for how to get a coherent system. Agreed. Skipping the pip resolver will move OpenStack and friends further down into a weird world of its own rather than Python norms. If possible we should try to resolve the complexity, not bandaid the problems caused by the complexity. If we can't do that then per service virtualenv (or even containers) provides a far more contained[1] bandage. Related: I personally find it completely bewildering that things are managed and run in such an unbounded fashion. My intuition is that this is the result of many/most projects being on the same release cycle and the belief that my master must integrate with your master. We should release (far) more often and my master should integrate with your lastest release. [1] Oh, my sides. -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On Wed, Jan 14, 2015, at 04:50 AM, Sean Dague wrote: On 01/14/2015 03:55 AM, Flavio Percoco wrote: On 13/01/15 14:31 -0500, Sean Dague wrote: On 01/13/2015 02:10 PM, Jay Pipes wrote: On 01/13/2015 12:39 PM, Zane Bitter wrote: On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. +1 I think as we grow global-requirements probably needs to be broken up into functional lists. What's allowed in base-iass, what's allowed in application services, what's allowed in docs, etc, etc. The complexity of keeping the g-r list actually working goes up sort of n^2 as the size of it, because pip doesn't have a real solver. This is a barely functional set of plate spinning so just add everything misses the point that the breaks get more and more difficult to unwind. If someone is up for stepping up and contributing to breaking up into domains, please do. But I think while this remains a global list we really do need to be a bit careful here. Do we really need a per-domain g-r? With the upcoming changes in the openstack governance model, I believe this won't scale even if we break it into per-domain basis. Some questions that need answeres are: - Who's going to review the per domain g-r ? If it's a centralized team, then it won't definitely scale. If it's the domain team then, what's the real point of having these requirements? - How are we going to support the creation and maintnance of these g-r in the gate? Are they actually worth it? While I understand why we have g-r, I really don't think it'll scale for a broader community as we're envisioning it. With that in mind, I'd probably recommend to have 1 requirements group for the base-caas integrated gate and let other projects and groups have their own requirements. That is, non base-caas projects should get the versions of their common requirements from g-r so that they won't conflict if they're installed alongside with any of the base-caas projects. But other than those base requirements, I'd let non base-caas projects have what they need (which is pretty much what heat's team has done with the contrib packages, AFAICT). I know this opens the doors for version conflicts between the requirements of non base-caas projects but I think this is a more welcoming (and non-blocking) way to do things until we've a better one. Hope the above makes some sense I blame the lack of coffee if it doesn't, Flavio We have the base infrastructure of that in devstack today with a SOFT requirements update - https://github.com/openstack-dev/devstack/blob/master/functions-common#L1696-L1704 If you are not in projects.txt, and you set your REQUIREMENTS_MODE=soft in your devstack run, we will leave extra dependencies untouched. Probably a few more bits required to make it really easy to expose, but that direction is also feasable. The reason I brought up domains though is that some groups really want the facility to have common requirements lists across a domain. Like the OpenStack Docs team. They've currently inserted a ton of stuff in g-r that really shouldn't be there because they have a lot of git trees, and the synchronization is a nice feature. However, if there were domain specific lists, that would be fine. A collection of projects that want to coordinate could all agree on a domain specific set of lists, and off we go. Maybe that list wouldn't even need to be contained in the main repo, it could be in a sub repo so have different approvers. This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting.
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 21 January 2015 at 10:21, Clark Boylan cboy...@sapwetik.org wrote: ... This ml thread came up in the TC meeting today and I am responding here to catch the thread up with the meeting. The soft update option is the suggested fix for non openstack projects that want to have most of their requirements managed by global requirements. For the project structure reform opening things up we should consider loosening the criteria to get on the list and make it primarily based on technical criteria such as py3k support, license compatibility, upstream support/activity, and so on (basically the current criteria with less of a focus on where the project comes from if it is otherwise healthy). Then individual projects would choose the subset they need to depend on. This model should be viable with different domains as well if we go that route. The following is not from the TC meeting but addressing other portions of this conversation: At least one concern with this option is that as the number of total requirements goes up is the difficulty in debugging installation conflicts becomes more difficult too. I have suggested that we could write tools to help with this. Install bisection based on pip logs for example, but these tools are still theoretical so I may be overestimating their usefulness. To address the community scaling aspect I think you push a lot of work back on deployers/users if we don't curate requirements for anything that ends up tagged as production ready (or whatever the equivalent tag becomes). Essentially we are saying this doesn't scale for us so now you deal with the fallout. Have fun, which isn't very friendly to people consuming the software. We already have an absurd number of requirements and management of them has appeared to scale. I don't foresee my workload going up if we open up the list as suggested. Perhaps I missed something, but the initial request wasn't about random packages, it was about other stackforge clients - these are things in the ecosystem! I'm glad we have technical solutions, but it just seems odd to me that adding them would ever have been controversial. On the pip solver side, joe gordon was working on a thing to install a fixed set of packages by bypassing the pip resolver... not sure how thats progressing. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 13/01/15 14:31 -0500, Sean Dague wrote: On 01/13/2015 02:10 PM, Jay Pipes wrote: On 01/13/2015 12:39 PM, Zane Bitter wrote: On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. +1 I think as we grow global-requirements probably needs to be broken up into functional lists. What's allowed in base-iass, what's allowed in application services, what's allowed in docs, etc, etc. The complexity of keeping the g-r list actually working goes up sort of n^2 as the size of it, because pip doesn't have a real solver. This is a barely functional set of plate spinning so just add everything misses the point that the breaks get more and more difficult to unwind. If someone is up for stepping up and contributing to breaking up into domains, please do. But I think while this remains a global list we really do need to be a bit careful here. Do we really need a per-domain g-r? With the upcoming changes in the openstack governance model, I believe this won't scale even if we break it into per-domain basis. Some questions that need answeres are: - Who's going to review the per domain g-r ? If it's a centralized team, then it won't definitely scale. If it's the domain team then, what's the real point of having these requirements? - How are we going to support the creation and maintnance of these g-r in the gate? Are they actually worth it? While I understand why we have g-r, I really don't think it'll scale for a broader community as we're envisioning it. With that in mind, I'd probably recommend to have 1 requirements group for the base-caas integrated gate and let other projects and groups have their own requirements. That is, non base-caas projects should get the versions of their common requirements from g-r so that they won't conflict if they're installed alongside with any of the base-caas projects. But other than those base requirements, I'd let non base-caas projects have what they need (which is pretty much what heat's team has done with the contrib packages, AFAICT). I know this opens the doors for version conflicts between the requirements of non base-caas projects but I think this is a more welcoming (and non-blocking) way to do things until we've a better one. Hope the above makes some sense I blame the lack of coffee if it doesn't, Flavio That said, I'd like to suggest another possible workaround: in Heat we keep resource plugins for non-official projects in the /contrib tree, and each plugin has it's own setup.cfg and requirements.txt. For example: http://git.openstack.org/cgit/openstack/heat/tree/contrib/heat_barbican So the user has the option to install each plugin, and it comes with its own requirements, independent of the main Heat installation. Perhaps Rally could consider something similar. Seems like a very reasonable solution to me. Thanks for bringing it up, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- @flaper87 Flavio Percoco pgpSTIBS8VTg9.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 01/14/2015 03:55 AM, Flavio Percoco wrote: On 13/01/15 14:31 -0500, Sean Dague wrote: On 01/13/2015 02:10 PM, Jay Pipes wrote: On 01/13/2015 12:39 PM, Zane Bitter wrote: On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. +1 I think as we grow global-requirements probably needs to be broken up into functional lists. What's allowed in base-iass, what's allowed in application services, what's allowed in docs, etc, etc. The complexity of keeping the g-r list actually working goes up sort of n^2 as the size of it, because pip doesn't have a real solver. This is a barely functional set of plate spinning so just add everything misses the point that the breaks get more and more difficult to unwind. If someone is up for stepping up and contributing to breaking up into domains, please do. But I think while this remains a global list we really do need to be a bit careful here. Do we really need a per-domain g-r? With the upcoming changes in the openstack governance model, I believe this won't scale even if we break it into per-domain basis. Some questions that need answeres are: - Who's going to review the per domain g-r ? If it's a centralized team, then it won't definitely scale. If it's the domain team then, what's the real point of having these requirements? - How are we going to support the creation and maintnance of these g-r in the gate? Are they actually worth it? While I understand why we have g-r, I really don't think it'll scale for a broader community as we're envisioning it. With that in mind, I'd probably recommend to have 1 requirements group for the base-caas integrated gate and let other projects and groups have their own requirements. That is, non base-caas projects should get the versions of their common requirements from g-r so that they won't conflict if they're installed alongside with any of the base-caas projects. But other than those base requirements, I'd let non base-caas projects have what they need (which is pretty much what heat's team has done with the contrib packages, AFAICT). I know this opens the doors for version conflicts between the requirements of non base-caas projects but I think this is a more welcoming (and non-blocking) way to do things until we've a better one. Hope the above makes some sense I blame the lack of coffee if it doesn't, Flavio We have the base infrastructure of that in devstack today with a SOFT requirements update - https://github.com/openstack-dev/devstack/blob/master/functions-common#L1696-L1704 If you are not in projects.txt, and you set your REQUIREMENTS_MODE=soft in your devstack run, we will leave extra dependencies untouched. Probably a few more bits required to make it really easy to expose, but that direction is also feasable. The reason I brought up domains though is that some groups really want the facility to have common requirements lists across a domain. Like the OpenStack Docs team. They've currently inserted a ton of stuff in g-r that really shouldn't be there because they have a lot of git trees, and the synchronization is a nice feature. However, if there were domain specific lists, that would be fine. A collection of projects that want to coordinate could all agree on a domain specific set of lists, and off we go. Maybe that list wouldn't even need to be contained in the main repo, it could be in a sub repo so have different approvers. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
Flavio, fyi, nova-docker is aspiring to be part of nova tree itself, but we had a couple of python libraries we needed for docker that are not in global, so we used the soft requirements to do that. https://github.com/stackforge/nova-docker/blob/master/contrib/devstack/gate_hook.sh#L16 thanks, dims On Wed, Jan 14, 2015 at 7:50 AM, Sean Dague s...@dague.net wrote: On 01/14/2015 03:55 AM, Flavio Percoco wrote: On 13/01/15 14:31 -0500, Sean Dague wrote: On 01/13/2015 02:10 PM, Jay Pipes wrote: On 01/13/2015 12:39 PM, Zane Bitter wrote: On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. +1 I think as we grow global-requirements probably needs to be broken up into functional lists. What's allowed in base-iass, what's allowed in application services, what's allowed in docs, etc, etc. The complexity of keeping the g-r list actually working goes up sort of n^2 as the size of it, because pip doesn't have a real solver. This is a barely functional set of plate spinning so just add everything misses the point that the breaks get more and more difficult to unwind. If someone is up for stepping up and contributing to breaking up into domains, please do. But I think while this remains a global list we really do need to be a bit careful here. Do we really need a per-domain g-r? With the upcoming changes in the openstack governance model, I believe this won't scale even if we break it into per-domain basis. Some questions that need answeres are: - Who's going to review the per domain g-r ? If it's a centralized team, then it won't definitely scale. If it's the domain team then, what's the real point of having these requirements? - How are we going to support the creation and maintnance of these g-r in the gate? Are they actually worth it? While I understand why we have g-r, I really don't think it'll scale for a broader community as we're envisioning it. With that in mind, I'd probably recommend to have 1 requirements group for the base-caas integrated gate and let other projects and groups have their own requirements. That is, non base-caas projects should get the versions of their common requirements from g-r so that they won't conflict if they're installed alongside with any of the base-caas projects. But other than those base requirements, I'd let non base-caas projects have what they need (which is pretty much what heat's team has done with the contrib packages, AFAICT). I know this opens the doors for version conflicts between the requirements of non base-caas projects but I think this is a more welcoming (and non-blocking) way to do things until we've a better one. Hope the above makes some sense I blame the lack of coffee if it doesn't, Flavio We have the base infrastructure of that in devstack today with a SOFT requirements update - https://github.com/openstack-dev/devstack/blob/master/functions-common#L1696-L1704 If you are not in projects.txt, and you set your REQUIREMENTS_MODE=soft in your devstack run, we will leave extra dependencies untouched. Probably a few more bits required to make it really easy to expose, but that direction is also feasable. The reason I brought up domains though is that some groups really want the facility to have common requirements lists across a domain. Like the OpenStack Docs team. They've currently inserted a ton of stuff in g-r that really shouldn't be there because they have a lot of git trees, and the synchronization is a nice feature. However, if there were domain specific lists, that would be fine. A collection of projects that want to coordinate could all agree on a domain specific set of lists, and off we go. Maybe that list
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. That said, I'd like to suggest another possible workaround: in Heat we keep resource plugins for non-official projects in the /contrib tree, and each plugin has it's own setup.cfg and requirements.txt. For example: http://git.openstack.org/cgit/openstack/heat/tree/contrib/heat_barbican So the user has the option to install each plugin, and it comes with its own requirements, independent of the main Heat installation. Perhaps Rally could consider something similar. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 01/13/2015 02:10 PM, Jay Pipes wrote: On 01/13/2015 12:39 PM, Zane Bitter wrote: On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. +1 I think as we grow global-requirements probably needs to be broken up into functional lists. What's allowed in base-iass, what's allowed in application services, what's allowed in docs, etc, etc. The complexity of keeping the g-r list actually working goes up sort of n^2 as the size of it, because pip doesn't have a real solver. This is a barely functional set of plate spinning so just add everything misses the point that the breaks get more and more difficult to unwind. If someone is up for stepping up and contributing to breaking up into domains, please do. But I think while this remains a global list we really do need to be a bit careful here. That said, I'd like to suggest another possible workaround: in Heat we keep resource plugins for non-official projects in the /contrib tree, and each plugin has it's own setup.cfg and requirements.txt. For example: http://git.openstack.org/cgit/openstack/heat/tree/contrib/heat_barbican So the user has the option to install each plugin, and it comes with its own requirements, independent of the main Heat installation. Perhaps Rally could consider something similar. Seems like a very reasonable solution to me. Thanks for bringing it up, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 01/13/2015 12:39 PM, Zane Bitter wrote: On 13/01/15 10:01, Jeremy Stanley wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. FWIW I don't really agree with this advice. The purpose of openstack/requirements is to ensure that it's possible to create a distribution of OpenStack without conflicting version requirements, not to force every project to use every dependency listed. As such, some co-ordination around client libraries for related projects seems like it ought to be an uncontroversial addition. (Obviously it's easy to imagine potential additions that would legitimately be highly controversial, but IMHO this isn't one of them.) Saying that we require people to use our tools to get into the club but that our tools are not going to accommodate them in any way until they are in sounds a bit too close to Go away to my ears. +1 That said, I'd like to suggest another possible workaround: in Heat we keep resource plugins for non-official projects in the /contrib tree, and each plugin has it's own setup.cfg and requirements.txt. For example: http://git.openstack.org/cgit/openstack/heat/tree/contrib/heat_barbican So the user has the option to install each plugin, and it comes with its own requirements, independent of the main Heat installation. Perhaps Rally could consider something similar. Seems like a very reasonable solution to me. Thanks for bringing it up, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
I support this, it will help to not break compatibility with global-requirements for sole purpose to include python client for other stackforge project. On Tue, Jan 13, 2015 at 12:14 AM, Boris Pavlovic bo...@pavlovic.me wrote: Hello TC, I would like to propose to allow adding all python-clients from stackforge (that are regarding global-requirements) to global requirements. It doesn't cost anything and simplifies life for everybody on stackforge. P.S. We already have billions libs in global requirements that aren't even on stackforge. Having few more or less doesn't make any sense... Best regards, Boris Pavlovic __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Serg Melikyan, Senior Software Engineer at Mirantis, Inc. http://mirantis.com | smelik...@mirantis.com +7 (495) 640-4904, 0261 +7 (903) 156-0836 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 13/01/15 01:14 +0400, Boris Pavlovic wrote: Hello TC, I would like to propose to allow adding all python-clients from stackforge (that are regarding global-requirements) to global requirements. I believe this is not a thing for the TC to decided but the dev community, especially the infra team, requirement's cores, etc. What python-clients are you referring to? Are you referring to libs that live in stackforge or dependencies for things that live in stackforge? Do you have an (or many) example? Flavio -- @flaper87 Flavio Percoco pgps2sjt5rSu5.pgp Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
Flavio, Current 2 python clients are Mistral and Murano that I would like to use in Rally. I can't add them to requirements = so I need to work everywhere in lazy-mode + print exceptions to end users e.g. you don't have this client, so install it - which is really not user friendly. Best regards, Boris Pavlovic On Tue, Jan 13, 2015 at 1:46 PM, Flavio Percoco fla...@redhat.com wrote: On 13/01/15 01:14 +0400, Boris Pavlovic wrote: Hello TC, I would like to propose to allow adding all python-clients from stackforge (that are regarding global-requirements) to global requirements. I believe this is not a thing for the TC to decided but the dev community, especially the infra team, requirement's cores, etc. What python-clients are you referring to? Are you referring to libs that live in stackforge or dependencies for things that live in stackforge? Do you have an (or many) example? Flavio -- @flaper87 Flavio Percoco __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 01/13/2015 06:56 AM, Boris Pavlovic wrote: Flavio, Current 2 python clients are Mistral and Murano that I would like to use in Rally. I can't add them to requirements = so I need to work everywhere in lazy-mode + print exceptions to end users e.g. you don't have this client, so install it - which is really not user friendly. Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. -Sean -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
Jeremy, Sean, First of all because I would like to make a Rally official part of OpenStack and if I remove Rally from project.txt this won't happen. On other side I would like to make OpenStack better. And OpenStack includes as well projects on StackForge. That for some reason were rejected. In my opinion all projects in equal portion deserver good testing framework. And I as PTL of Rally would like to adopt it for everybody. P.S. It's sad that in one thread we are talking about making Big Tenant and how it is cool to remove programs. In another we are blocking adding python clients from projects that are not Core and suggesting another making incompatible with OpenStack. Best regards, Boris Pavlovic On Tue, Jan 13, 2015 at 6:01 PM, Jeremy Stanley fu...@yuggoth.org wrote: On 2015-01-13 07:46:38 -0500 (-0500), Sean Dague wrote: Why doesn't rally just remove itself from projects.txt, then there would be no restrictions on what it adds. I second this recommendation. If Rally wants to depend on things which are not part of OpenStack and not required to run or test OpenStack in an official capacity (and since it's not an official OpenStack project it's entirely within its rights to want to do that), then forcing it to comply with the list of requirements for official OpenStack projects is an unnecessarily restrictive choice. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tc][python-clients] More freedom for all python clients
On 2015-01-13 19:53:28 +0400 (+0400), Boris Pavlovic wrote: [...] P.S. It's sad that in one thread we are talking about making Big Tenant and how it is cool to remove programs. In another we are blocking adding python clients from projects that are not Core and suggesting another making incompatible with OpenStack. The big tent proposal is not just a free-for-all, and will entail most of the same barriers to entry for officialness as you currently see for programs (aside from addresses similar needs as a current official team). Quality control and selectiveness of design will still be extremely important, and I don't see our openstack/requirements tooling, workflow and choices changing significantly in the face of that. -- Jeremy Stanley __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [tc][python-clients] More freedom for all python clients
Hello TC, I would like to propose to allow adding all python-clients from stackforge (that are regarding global-requirements) to global requirements. It doesn't cost anything and simplifies life for everybody on stackforge. P.S. We already have billions libs in global requirements that aren't even on stackforge. Having few more or less doesn't make any sense... Best regards, Boris Pavlovic __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev