Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
Random comments inline Robert Collins wrote: [...] So I think the test should actually be something like: Apply caution if it is not packaged AND packaging it is hard. Things that make packaging a Python package hard: - nonstandard build systems - C dependencies that aren't already packaged - unusual licences E.g. things which are easy, either because they can just use existing dependencies, or they're pure python, we shouldn't worry about. Beyond being difficult to package, I think there was the idea of facilitating the distributions work by keeping them in the loop when we add dependencies. In the past we had cases where OpenStack devs wanted to add a library and distros had an already-packaged, functionally-equivalent, and sometimes better-maintained library to suggest instead. In some other, they had security issues with a library that upstream wouldn't fix, and moved to another library as a result. Involving them is a win-win. Basically, it's not just about packaging: tapping into the distros expertise when it comes to the state of the Python ecosystem was (and imho still is) highly valuable to us. Saying we'll pick whatever we come up with as long as we think it's not a pain to package is a bit one-sided. That said, on this repository we haven't seen the kind of involvement from distros that we expected in return... On 22 August 2015 at 11:50, Dave Walker em...@daviey.com wrote: The release schedule used to document a DepFreeze[0] to avoid nasty surprises for distros, which used to be at the same point of FeatureFreeze[1]. This reference seems to have been removed from the last few cycles, but I would suggest that it could be re-added. As more components of OpenStack are moving away from big-bang releases, DepFreeze would make less and less sense to me. The integrated release is on the way out. I'm not so sure. DepFreeze is about facilitating the work of distributions at the end of a development cycle so that they can release packaged versions as close as possible to the end of cycle (a.k.a. release) date. OpenStack gradually moving away from big-bang releases has limited impact on that -- distros still do big-bang releases and appreciate us slowing down on the dep addition front every 6 months. We could say we no longer care about distro release cycle alignment, but it served us well in the past. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
On 22 August 2015 at 11:50, Dave Walker em...@daviey.com wrote: On 22 August 2015 at 00:04, Matthew Thode prometheanf...@gentoo.org wrote: On 08/21/2015 05:59 PM, Robert Collins wrote: On 22 August 2015 at 10:57, Matthew Thode prometheanf...@gentoo.org wrote: Packaging for us is fairly easy, but it is annoying to have to add 5-6 deps each release, (which means we are adding cruft over time). We're adding functionality by bringing in existing implementations. Surely thats better than reinventing *everything* ? -Rob totally, more of a minor annoyance :P A strong reason that requirements was created was to give distros a voice and avoid incompatible versions, which was more of a problem for distros than it was for each different service at that point. So, incompatible versions with which/what packages? We guarantee co-installability within OpenStack, but if e.g. RHEL and Ubuntu are mutually incompatible in their versions of some package, what should OpenStack do? That seems like a problem that is intrinsically unsolvable for OpenStack as its a consistency issue between a large and growing number of independent groups - and inevitably, in that situation, one loses. So they have to solve it locally - for Ubuntu via click, for RHEL via software collections. So if its locally solvable, why should OpenStack consider it? On the voice: please do get more distributors reviewing requirements changes! I'm not sure that a requirement has ever been not included because it *wasn't* packaged, but perhaps because it *couldn't* be packaged. Is there an example that has caused you to raise this? I'm a new core there and trying to make sure what we document and what we want to happen are the same thing. 'Apply caution' doesn't mean much to me other than to feel scared! So I'm trying to elucidate what should be there instead. The is-it-packaged-test was added at a time where large changes were happening in OpenStack right up to the (release) wire and cause scary changes for distros that were tracking the release. Now, Feature development has become more mature with the scary stuff being front loaded, I'm not quite sure this is such a problem. The release schedule used to document a DepFreeze[0] to avoid nasty surprises for distros, which used to be at the same point of FeatureFreeze[1]. This reference seems to have been removed from the last few cycles, but I would suggest that it could be re-added. As more components of OpenStack are moving away from big-bang releases, DepFreeze would make less and less sense to me. The integrated release is on the way out. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
Excerpts from Robert Collins's message of 2015-08-20 15:24:03 +1200: We currently have a test where we ask if things are packaged in distros. http://git.openstack.org/cgit/openstack/requirements/tree/README.rst#n268 I think we should modify that, in two ways. The explanation for the question ignores a fairly large audience of deployers who don't wait for distributions - so they too need to package things, but unlike distributions packaging stuff is itself incidental to their business, rather than being it. So I think we should consider their needs too. Secondly, all the cases of this I've seen so far we've essentially gone 'sure, fine'. I think thats because there's really nothing to them. So I think the test should actually be something like: Apply caution if it is not packaged AND packaging it is hard. Things that make packaging a Python package hard: - nonstandard build systems - C dependencies that aren't already packaged - unusual licences E.g. things which are easy, either because they can just use existing dependencies, or they're pure python, we shouldn't worry about. -Rob I think this interpretation is fine. It's more or less what I've been doing anyway. Is it safe to assume that if a package is available on PyPI and can be installed with pip, packaging it for a distro isn't technically difficult? (It might be difficult due to vendoring, licensing, or some other issue that would be harder to test for.) Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
On 22 August 2015 at 09:08, Doug Hellmann d...@doughellmann.com wrote: Excerpts from Robert Collins's message of 2015-08-20 15:24:03 +1200: We currently have a test where we ask if things are packaged in distros. http://git.openstack.org/cgit/openstack/requirements/tree/README.rst#n268 I think we should modify that, in two ways. The explanation for the question ignores a fairly large audience of deployers who don't wait for distributions - so they too need to package things, but unlike distributions packaging stuff is itself incidental to their business, rather than being it. So I think we should consider their needs too. Secondly, all the cases of this I've seen so far we've essentially gone 'sure, fine'. I think thats because there's really nothing to them. So I think the test should actually be something like: Apply caution if it is not packaged AND packaging it is hard. Things that make packaging a Python package hard: - nonstandard build systems - C dependencies that aren't already packaged - unusual licences E.g. things which are easy, either because they can just use existing dependencies, or they're pure python, we shouldn't worry about. -Rob I think this interpretation is fine. It's more or less what I've been doing anyway. Is it safe to assume that if a package is available on PyPI and can be installed with pip, packaging it for a distro isn't technically difficult? (It might be difficult due to vendoring, licensing, or some other issue that would be harder to test for.) Licensing we already assess separately, anything we are ok with (Apache2, MIT, BSD) distros and operators should fine easy. Operators shouldn't be caring about vendoring: being on the CD train is exactly what vendoring aims at [reliability on the assumption that you'll upgrade all the things as fast as possible to ensure security]. Distributors probably care about vendoring, but IMO we should ignore that. Projects that vendor do so with consideration for stability and user experience - and even so is pretty rare in the Python space. Distros are making a different choice - which is their right - but its strictly additional work that has significant costs, and they've already factored in that cost in aggregate in choosing to devendor things. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
On 08/21/2015 04:08 PM, Doug Hellmann wrote: Excerpts from Robert Collins's message of 2015-08-20 15:24:03 +1200: We currently have a test where we ask if things are packaged in distros. http://git.openstack.org/cgit/openstack/requirements/tree/README.rst#n268 I think we should modify that, in two ways. The explanation for the question ignores a fairly large audience of deployers who don't wait for distributions - so they too need to package things, but unlike distributions packaging stuff is itself incidental to their business, rather than being it. So I think we should consider their needs too. Secondly, all the cases of this I've seen so far we've essentially gone 'sure, fine'. I think thats because there's really nothing to them. So I think the test should actually be something like: Apply caution if it is not packaged AND packaging it is hard. Things that make packaging a Python package hard: - nonstandard build systems - C dependencies that aren't already packaged - unusual licences E.g. things which are easy, either because they can just use existing dependencies, or they're pure python, we shouldn't worry about. -Rob I think this interpretation is fine. It's more or less what I've been doing anyway. Is it safe to assume that if a package is available on PyPI and can be installed with pip, packaging it for a distro isn't technically difficult? (It might be difficult due to vendoring, licensing, or some other issue that would be harder to test for.) Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Packaging for us is fairly easy, but it is annoying to have to add 5-6 deps each release, (which means we are adding cruft over time). -- Matthew Thode (prometheanfire) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
On 22 August 2015 at 10:57, Matthew Thode prometheanf...@gentoo.org wrote: Packaging for us is fairly easy, but it is annoying to have to add 5-6 deps each release, (which means we are adding cruft over time). We're adding functionality by bringing in existing implementations. Surely thats better than reinventing *everything* ? -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
On 22 August 2015 at 00:04, Matthew Thode prometheanf...@gentoo.org wrote: On 08/21/2015 05:59 PM, Robert Collins wrote: On 22 August 2015 at 10:57, Matthew Thode prometheanf...@gentoo.org wrote: Packaging for us is fairly easy, but it is annoying to have to add 5-6 deps each release, (which means we are adding cruft over time). We're adding functionality by bringing in existing implementations. Surely thats better than reinventing *everything* ? -Rob totally, more of a minor annoyance :P A strong reason that requirements was created was to give distros a voice and avoid incompatible versions, which was more of a problem for distros than it was for each different service at that point. I'm not sure that a requirement has ever been not included because it *wasn't* packaged, but perhaps because it *couldn't* be packaged. Is there an example that has caused you to raise this? The is-it-packaged-test was added at a time where large changes were happening in OpenStack right up to the (release) wire and cause scary changes for distros that were tracking the release. Now, Feature development has become more mature with the scary stuff being front loaded, I'm not quite sure this is such a problem. The release schedule used to document a DepFreeze[0] to avoid nasty surprises for distros, which used to be at the same point of FeatureFreeze[1]. This reference seems to have been removed from the last few cycles, but I would suggest that it could be re-added. [0] https://wiki.openstack.org/wiki/DepFreeze [1] https://wiki.openstack.org/wiki/FeatureFreeze -- Kind Regards, Dave Walker __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [requirements] modifying the 'is it packaged' test
On 08/21/2015 05:59 PM, Robert Collins wrote: On 22 August 2015 at 10:57, Matthew Thode prometheanf...@gentoo.org wrote: Packaging for us is fairly easy, but it is annoying to have to add 5-6 deps each release, (which means we are adding cruft over time). We're adding functionality by bringing in existing implementations. Surely thats better than reinventing *everything* ? -Rob totally, more of a minor annoyance :P -- Matthew Thode (prometheanfire) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [requirements] modifying the 'is it packaged' test
We currently have a test where we ask if things are packaged in distros. http://git.openstack.org/cgit/openstack/requirements/tree/README.rst#n268 I think we should modify that, in two ways. The explanation for the question ignores a fairly large audience of deployers who don't wait for distributions - so they too need to package things, but unlike distributions packaging stuff is itself incidental to their business, rather than being it. So I think we should consider their needs too. Secondly, all the cases of this I've seen so far we've essentially gone 'sure, fine'. I think thats because there's really nothing to them. So I think the test should actually be something like: Apply caution if it is not packaged AND packaging it is hard. Things that make packaging a Python package hard: - nonstandard build systems - C dependencies that aren't already packaged - unusual licences E.g. things which are easy, either because they can just use existing dependencies, or they're pure python, we shouldn't worry about. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
