Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Mark McLoughlin
Hi Nick, On Tue, 2013-03-05 at 17:56 +1000, Nick Coghlan wrote: On Tue, Mar 5, 2013 at 8:29 AM, Mark McLoughlin mar...@redhat.com wrote: On Mon, 2013-03-04 at 12:44 -0500, Donald Stufft wrote: On Monday, March 4, 2013 at 12:36 PM, Mark McLoughlin wrote: If parallel incompatible installs

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Donald Stufft
On Tuesday, March 5, 2013 at 7:27 AM, Mark McLoughlin wrote: Hi Nick, On Tue, 2013-03-05 at 17:56 +1000, Nick Coghlan wrote: On Tue, Mar 5, 2013 at 8:29 AM, Mark McLoughlin mar...@redhat.com (mailto:mar...@redhat.com) wrote: On Mon, 2013-03-04 at 12:44 -0500, Donald Stufft wrote:

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Mark McLoughlin
On Tue, 2013-03-05 at 07:34 -0500, Donald Stufft wrote: On Tuesday, March 5, 2013 at 7:27 AM, Mark McLoughlin wrote: On Tue, 2013-03-05 at 17:56 +1000, Nick Coghlan wrote: ... I will note that making this kind of idea more feasible is one of the reasons I am making compatible release

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Donald Stufft
On Tuesday, March 5, 2013 at 7:50 AM, Mark McLoughlin wrote: I still don't really see how this is related to PEP426 unless PEP426 has gotten a lot larger since I last looked at it. Where in particular a distribution gets installed is left up to the installers to sort out. And making sure

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Nick Coghlan
On Tue, Mar 5, 2013 at 10:34 PM, Donald Stufft donald.stu...@gmail.com wrote: On Tuesday, March 5, 2013 at 7:27 AM, Mark McLoughlin wrote: If Python library maintainers will see PEP426 as a license to make incompatible changes more often so long as they bump their major number, then that

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Nick Coghlan
On Tue, Mar 5, 2013 at 10:27 PM, Mark McLoughlin mar...@redhat.com wrote: Ok, there's a tonne of details there about pip, virtualenv and .pth files that are going over my head right now, but the general idea I'm taking away is: - the system has multiple versions of somedep installed under

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-05 Thread Nick Coghlan
On Tue, Mar 5, 2013 at 11:28 PM, Daniel Holth dho...@gmail.com wrote: You will be amazed at how much backwards compatible cruft some developers will keep around rather than endure the psychological anguish of bumping the major version number. We also have Python 3 to hit them over the head

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Nick Coghlan
On Mon, Mar 4, 2013 at 1:54 AM, Mark McLoughlin mar...@redhat.com wrote: I've tried to digest PEP426 and collected my thoughts below. Apologies if you read through them and they offer you nothing useful. I'm trying to imagine the future we're driving towards here where OpenStack is no longer

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Daniel Holth
Just a short note to state the obvious - the specific pycparsing library from the example did exactly what it should have done and can do in the current system, by incrementing its major version number for a backwards-incompatible release. Dependency management is a job though.

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Barry Warsaw
On Mar 04, 2013, at 06:11 PM, Nick Coghlan wrote: My point of view is that the system Python is there primarily to run system utilities and user scripts, rather than arbitrary Python applications. Users can install alternate versions of software into their user site directories, or into virtual

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Daniel Holth
On Mon, Mar 4, 2013 at 10:59 AM, Barry Warsaw ba...@python.org wrote: On Mar 04, 2013, at 06:11 PM, Nick Coghlan wrote: My point of view is that the system Python is there primarily to run system utilities and user scripts, rather than arbitrary Python applications. Users can install alternate

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Ned Deily
In article cag8k2+74cuh6p_mmgvs2hlknreim_iepbq0u7vf9dtpxk4+...@mail.gmail.com, Daniel Holth dho...@gmail.com wrote: Apple does (or did) something very useful by not including the stdlib source code in OS X's builtin Python, making the system Python so hilariously useless for development that

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Mark McLoughlin
Hi Nick, Thanks for the detailed reply. I'll stick to PEP426 related topics in this mail. Generally speaking, PEP426 looks like good progress, but the biggest problem I see now is the lack of parallel installs for incompatible versions. On Mon, 2013-03-04 at 18:11 +1000, Nick Coghlan wrote: On

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Daniel Holth
On Mon, Mar 4, 2013 at 12:12 PM, Ned Deily n...@acm.org wrote: In article cag8k2+74cuh6p_mmgvs2hlknreim_iepbq0u7vf9dtpxk4+...@mail.gmail.com, Daniel Holth dho...@gmail.com wrote: Apple does (or did) something very useful by not including the stdlib source code in OS X's builtin Python,

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Mark McLoughlin
Hey, On parallel installs ... On Mon, 2013-03-04 at 18:11 +1000, Nick Coghlan wrote: On Mon, Mar 4, 2013 at 1:54 AM, Mark McLoughlin mar...@redhat.com wrote: - Incompatible versions of the same library are routinely installed in parallel. Does PEP426 here, or is all the work to be

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Donald Stufft
On Monday, March 4, 2013 at 12:36 PM, Mark McLoughlin wrote: Hey, On parallel installs ... On Mon, 2013-03-04 at 18:11 +1000, Nick Coghlan wrote: On Mon, Mar 4, 2013 at 1:54 AM, Mark McLoughlin mar...@redhat.com (mailto:mar...@redhat.com) wrote: - Incompatible versions of the same

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Daniel Holth
On Mon, Mar 4, 2013 at 12:23 PM, Mark McLoughlin mar...@redhat.com wrote: Hi Nick, Thanks for the detailed reply. I'll stick to PEP426 related topics in this mail. Generally speaking, PEP426 looks like good progress, but the biggest problem I see now is the lack of parallel installs for

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Mark McLoughlin
On Mon, 2013-03-04 at 12:44 -0500, Donald Stufft wrote: On Monday, March 4, 2013 at 12:36 PM, Mark McLoughlin wrote: If parallel incompatible installs is a hopeless problem in Python, why the push to semantic versioning then rather than saying that incompatible API changes should mean a

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Barry Warsaw
On Mar 04, 2013, at 10:29 PM, Mark McLoughlin wrote: The approach that some Fedora folks are trying out is called Software Collections. It's not Python specific, but it's basically the same as a virtual environment. It's a serious problem, and I think it will be made more so by the incursions

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Donald Stufft
On Monday, March 4, 2013 at 5:29 PM, Mark McLoughlin wrote: On Mon, 2013-03-04 at 12:44 -0500, Donald Stufft wrote: On Monday, March 4, 2013 at 12:36 PM, Mark McLoughlin wrote: If parallel incompatible installs is a hopeless problem in Python, why the push to semantic versioning

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-04 Thread Nick Coghlan
On Tue, Mar 5, 2013 at 8:29 AM, Mark McLoughlin mar...@redhat.com wrote: On Mon, 2013-03-04 at 12:44 -0500, Donald Stufft wrote: On Monday, March 4, 2013 at 12:36 PM, Mark McLoughlin wrote: If parallel incompatible installs is a hopeless problem in Python, why the push to semantic

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-03 Thread Mark McLoughlin
Hey On Thu, 2013-02-28 at 10:46 -0500, Daniel Holth wrote: Briefly in PEP 426 we are likely to copy the Ruby behavior as the default (without using the ~ operator itself) which is to depend on the remainder of a particular release series. In Ruby gems ~ 4.2.3 means = 4.2.3, 4.3.0 and the

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-03 Thread Mark McLoughlin
On Fri, 2013-03-01 at 01:06 -0800, Glyph wrote: On Feb 28, 2013, at 7:39 AM, Mark McLoughlin mar...@redhat.com wrote: I always felt that the Python community tended more towards the former approach, but there always exceptions to the rule - to unfairly pick one one project,

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-03 Thread Reinout van Rees
On 03-03-13 17:07, Mark McLoughlin wrote: I consider it good form for a setup.py to declare as loose dependencies as possible (no version qualifier or a = version qualifier) and for an application to provide a requires.txt or a buildout that has stricter requirements. Interesting! I feel like

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-03-01 Thread Glyph
On Feb 28, 2013, at 7:39 AM, Mark McLoughlin mar...@redhat.com wrote: I always felt that the Python community tended more towards the former approach, but there always exceptions to the rule - to unfairly pick one one project, sqlalchemy seems to have an API that often changes incompatibly.

[Distutils] Library instability on PyPI and impact on OpenStack

2013-02-28 Thread Mark McLoughlin
Hey, (I sent this to the wrong list and was directed here by Nick. I wasn't aware of the very promising sounding PEP426 and haven't read it yet, so apologies on that. Just wanted to resend my mail here ASAP to prevent the discussion happening on the wrong list. Thanks!) Generally speaking, when

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-02-28 Thread Daniel Holth
On Thu, Feb 28, 2013 at 10:39 AM, Mark McLoughlin mar...@redhat.com wrote: Hey, (I sent this to the wrong list and was directed here by Nick. I wasn't aware of the very promising sounding PEP426 and haven't read it yet, so apologies on that. Just wanted to resend my mail here ASAP to prevent

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 10:39 AM, Mark McLoughlin wrote: Hey, (I sent this to the wrong list and was directed here by Nick. I wasn't aware of the very promising sounding PEP426 and haven't read it yet, so apologies on that. Just wanted to resend my mail here ASAP to prevent the

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-02-28 Thread Reinout van Rees
On 28-02-13 16:39, Mark McLoughlin wrote: Generally speaking, when a project has a large list of dependencies on libraries outside of its control, it can take one of two approaches to those dependencies: 1) specify the minimum required version of each library and assume new releases of

Re: [Distutils] Library instability on PyPI and impact on OpenStack

2013-02-28 Thread Marcus Smith
yes, to what Reinout said. also, I thought this post from one of your threads was good http://lists.openstack.org/pipermail/openstack-dev/2013-February/006103.html if you want repeatable installs and testing throughout your release cycle, you have to pin your whole dependency tree. but in order