Re: [Python-Dev] PEP 397 - Last Comments
It looks good to me! Also, I see no reason not to always use a 32bit version of the launcher other than I'll change it, then - the strong reason *for* always using a 32-bit launcher is packaging, as the 32-bit installer would otherwise have to include both a 32-bit launcher and a 64-bit launcher, and install the right one depending on what the target system is. Thanks to Martin for updating it - I agree it is vastly improved! Thanks! Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Am 19.06.2012 23:46, schrieb Éric Araujo: Thanks for the detailed explanation, Éric. Just quoting this paragraph, since it contains the possibilities to judge: With beta coming, a way to deal with that unfortunate situation needs to be found. We could (a) grant an exception to packaging to allow changes after beta1; (b) keep packaging as it is now under a provisional status, with due warnings that many things are expected to change; (c) remove the unstable parts and deliver a subset that works (proposed by Tarek to the Pyramid author on distutils-sig); (d) not release packaging as part of Python 3.3 (I think that was also suggested on distutils-sig last month). (a) and (b) are certainly out of the question. packaging must be solid when shipped, and there's not enough time. (c) might work (depending on what features we're talking about), but you say yourself that you won't be able to spend the time required, so I agree with basically everybody else that (d) is the way to go (together with a PEP). Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Tue, Jun 19, 2012 at 11:46 PM, Éric Araujo mer...@netwok.org wrote: I don’t think (a) would give us enough time; we really want a few months (and releases) to hash out the API (most notably with the pip and buildout developers) and clean the bdist situation. Likewise (c) would require developer (my) time that is currently in short supply. (b) also requires time and would make development harder, not to mention probable user pain. This leaves (d), after long reflection, as my preferred choice, even though I disliked the idea at first (and I fully expect Tarek to feel the same way). It's a pity, but it sounds like the way to go. This may be crazy, but just idly wondering: is there an opportunity for the PSF to make things better by throwing some money at it? Packaging appears to be one of those Hard problems, it might be a good investment. Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wednesday, June 20, 2012 at 2:36 AM, Victor Stinner wrote: What is the status of the third party module on PyPI (distutils2)? Does it contain all fixes done in the packaging module? Does it have exactly the same API? Does it support Python 2.5 to 3.3, or maybe also 2.4? How is the distutils2 module installed? Installed manually? Using pip or setuptools? Is distutils2 included in some Linux distributions? If it's simple to install distutils2, it's not a big deal to not have it in the stdlib. -- It is sometimes a pain to have a buggy module in Python. For example, I got a lot of issues with the subprocess module of Python 2.5. I started to include a copy of the subprocess module from Python 2.7 in my projects to workaround these issues. In my previous work we did also backport various modules to get the last version of the xmlrpc client on Python 2.5 (especially for HTTP/1.1, to not open a new TCP socket at each request). I don't want to reopen the discussion the stdlib should be an external project. I just want to confirm that it is better to wait until important users of the packaging API have finished their work (on porting their project to distutils2, especially pip), before we can declare the module (and its API) as stable. By the way, what is the status of pip using distutils2? Some students started on a pip2 that was based on distutils2, but I don't think they've really done much/anything with actually using distutils2 and have mostly been working on other parts. Victor ___ Python-Dev mailing list Python-Dev@python.org (mailto:Python-Dev@python.org) http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep 362 - 5th edition
On Tue, Jun 19, 2012 at 12:38:38PM -0400, Yury Selivanov wrote: class Signature: . . . def equivalent(self, other): compares two Signatures for equality (ignores parameter names) . . . I don't think that comparing signatures will be a common operation, so maybe we can postpone adding any additional methods for that? I think it may be. Consider callback functions: the caller may wish to check that the callback function takes the right number of positional arguments, but without caring whether those arguments are given any particular name. Checking for compatible signatures is a little more subtle than just ignoring names. For example, keyword-only arguments need to always be compared by names. Also, you might want to ignore optional arguments with defaults, or at least _private optional arguments. I think equality should be strict, including names, and we should defer any decision about less-strict comparisons until 3.4, when we'll have more solid use-cases for it. I guess that's a long-winded way of saying +1 :) -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
This may be crazy, but just idly wondering: is there an opportunity for the PSF to make things better by throwing some money at it? Packaging appears to be one of those Hard problems, it might be a good investment. Only if somebody steps forward to take the money - and somebody who can be trusted to achieve something, as well. The general problem is that issues may only occur when packages actually use the library; so it may even be difficult to fix it in a concerted effort since that fixing may actually spread over several months (or years). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 362 minor nits
On Tue, Jun 19, 2012 at 08:11:26PM -0400, Yury Selivanov wrote: So using the signature will be OK for 'Foo.bar' and 'Foo().bar', but not for 'Foo.__dict__['bar']' - which I think is fine (since staticmethod classmethod instances are not callable) There has been some talk on Python-ideas about making staticmethod and classmethod instances callable. Speaking of non-instance method descriptors, please excuse this silly question, I haven't quite understood the implementation well enough to answer this question myself. Is there anything needed to make signature() work correctly with custom method-like descriptors such as this? http://code.activestate.com/recipes/577030-dualmethod-descriptor -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/19/12 11:46 PM, Éric Araujo wrote: ... I don’t think (a) would give us enough time; we really want a few months (and releases) to hash out the API (most notably with the pip and buildout developers) and clean the bdist situation. Likewise (c) would require developer (my) time that is currently in short supply. (b) also requires time and would make development harder, not to mention probable user pain. This leaves (d), after long reflection, as my preferred choice, even though I disliked the idea at first (and I fully expect Tarek to feel the same way). Yeah I feel the same way. +1 for (d). I had unfortunately no time lately. Thanks for picking up things. We want a solid distutils replacement, and I think we wrote solid PEPs and seemed to have find consensus for most issues in the past two years. So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, Jun 20, 2012 at 10:55 AM, Tarek Ziadé ta...@ziade.org wrote: So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... That policy has been a bit annoying. Gentoo has been carrying patches forever to improve compilation with C++ stuff (mostly about correctly passing on environment variables), and forward-porting them on every release gets tedious, but the packaging/distutils2 effort has made it harder to get them included in plain distutils. I understand there shouldn't be crazy patching in distutils, but allowing it to inch forward a little would make the lives of the Gentoo Python team easier, at least. Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 15:00:52 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 11:23 AM, Antoine Pitrou solip...@pitrou.net wrote: The question is what will happen after 3.3. There doesn't seem to be a lot of activity around the project, does it? I think the desire is there, What makes you think that, exactly? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 11:05:43 +0200 Dirkjan Ochtman dirk...@ochtman.nl wrote: On Wed, Jun 20, 2012 at 10:55 AM, Tarek Ziadé ta...@ziade.org wrote: So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... That policy has been a bit annoying. Gentoo has been carrying patches forever to improve compilation with C++ stuff (mostly about correctly passing on environment variables), and forward-porting them on every release gets tedious, but the packaging/distutils2 effort has made it harder to get them included in plain distutils. I understand there shouldn't be crazy patching in distutils, but allowing it to inch forward a little would make the lives of the Gentoo Python team easier, at least. I think the whole idea that distutils should be frozen and improvements should only go in distutils2 has been misled. Had distutils been improved instead, many of those enhancements would already have been available in 3.2 (and others would soon be released in 3.3). Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raw string syntax inconsistency
Am 18.06.2012 17:12, schrieb Guido van Rossum: Ok, banning ru... and ur... altogether is fine too (assuming it's fine with the originators of the PEP). It's gone for good. http://hg.python.org/cpython/rev/8e47e9af826e (My first push for a very long time. Man, that feels good!) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Tue, 19 Jun 2012 21:36:35 -0700 Guido van Rossum gu...@python.org wrote: Nick nailed it (again). Let's make things clear: packaging is suffering from a lack of developer involvement, and a lack of user interest. What makes you think that removing packaging from 3.3, and adding the constraint of a new PEP to be written, will actually *improve* things? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:05 AM, Dirkjan Ochtman wrote: On Wed, Jun 20, 2012 at 10:55 AM, Tarek Ziadéta...@ziade.org wrote: So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... That policy has been a bit annoying. Gentoo has been carrying patches forever to improve compilation with C++ stuff (mostly about correctly passing on environment variables), and forward-porting them on every release gets tedious, but the packaging/distutils2 effort has made it harder to get them included in plain distutils. I understand there shouldn't be crazy patching in distutils, but allowing it to inch forward a little would make the lives of the Gentoo Python team easier, at least. Cheers, If distutils gets new features I think it's killing the packaging effort. Maybe these new features could be implemented in packaging, then bridged in Distutils ? the Compilation feature is isolated enough to do this. In any case, I guess we should have some kind of policy in place where we list the exceptions when distutils can be changed. Maybe in the packaging PEP ? Cheers Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:04 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 15:00:52 +1000 Nick Coghlanncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 11:23 AM, Antoine Pitrousolip...@pitrou.net wrote: The question is what will happen after 3.3. There doesn't seem to be a lot of activity around the project, does it? I think the desire is there, What makes you think that, exactly? Maybe because the packaging fatigue occurs around 3 years after you start fighting that best, and we do have fresh blood working on it ? :) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:12 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 11:05:43 +0200 Dirkjan Ochtmandirk...@ochtman.nl wrote: On Wed, Jun 20, 2012 at 10:55 AM, Tarek Ziadéta...@ziade.org wrote: So I prefer to hold it and have a solid implementation in the stldib. The only thing I am asking is to retain ourselves to do *anything* in distutils and continue to declare it frozen, because I know it will be tempting to do stuff there... That policy has been a bit annoying. Gentoo has been carrying patches forever to improve compilation with C++ stuff (mostly about correctly passing on environment variables), and forward-porting them on every release gets tedious, but the packaging/distutils2 effort has made it harder to get them included in plain distutils. I understand there shouldn't be crazy patching in distutils, but allowing it to inch forward a little would make the lives of the Gentoo Python team easier, at least. I think the whole idea that distutils should be frozen and improvements should only go in distutils2 has been misled. Had distutils been improved instead, many of those enhancements would already have been available in 3.2 (and others would soon be released in 3.3). I tried to improve Distutils and I was stopped and told to start distutils2, because distutils is so rotten, any *real* change/improvment potentially brakes the outside world. This has not changed. Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. So what are your suggesting, since you seem to know what's a mistake and what's not ? (time-travel machine not allowed) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Antoine Pitrou solipsis at pitrou.net writes: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? Regards, Vinay Sajip ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 11:22:07 +0200 Tarek Ziadé ta...@ziade.org wrote: I tried to improve Distutils and I was stopped and told to start distutils2, because distutils is so rotten, any *real* change/improvment potentially brakes the outside world. If distutils was so rotten, distutils2 would not reuse much of its structure and concepts (and test suite), would it? Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. So what are your suggesting, since you seem to know what's a mistake and what's not ? I don't have any suggestion apart from keeping packaging in 3.3. But I also think it would be better for the community if people were not delusional when making decisions. Removing packaging from 3.3 is a big risk: users and potential contributors will be even less interested than they already are. Here's a datapoint: distribute (*) is downloaded 100x more times than distutils2 (**). (*) http://pypi.python.org/pypi/distribute/ (**) http://pypi.python.org/pypi/Distutils2/ Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 09:51:03 + (UTC) Vinay Sajip vinay_sa...@yahoo.co.uk wrote: Antoine Pitrou solipsis at pitrou.net writes: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? As an incentive for users to start using the features that are finished enough, and exercise the migration path from distutils. The module can be marked provisional so as to allow further API variations. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:49 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 11:22:07 +0200 Tarek Ziadéta...@ziade.org wrote: I tried to improve Distutils and I was stopped and told to start distutils2, because distutils is so rotten, any *real* change/improvment potentially brakes the outside world. If distutils was so rotten, distutils2 would not reuse much of its structure and concepts (and test suite), would it? 'much' is pretty vague here. distutils2 is a fork of distutils, that has evolved a *lot* if you look at the code, beside the compilation part and some commands, most things are different. distutils is rotten because when you change its internals, you might break some software that rely on them. Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? I started there, remember ? And we ended up saying it was impossible to continue without breaking the packaging world. Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. So what are your suggesting, since you seem to know what's a mistake and what's not ? I don't have any suggestion apart from keeping packaging in 3.3. But I also think it would be better for the community if people were not delusional when making decisions. Removing packaging from 3.3 is a big risk: users and potential contributors will be even less interested than they already are. That's a good point. But if no one works on its polishing *now*, it's going to be the same effect on people: they'll likely to be very annoyed if the replacer is not rock solid. Here's a datapoint: distribute (*) is downloaded 100x more times than distutils2 (**). (*) http://pypi.python.org/pypi/distribute/ (**) http://pypi.python.org/pypi/Distutils2/ why would you expect a different datapoint ? - Distutils2 was released as a beta software, and not really promoted yet - Distribute is downloaded automatically by many stacks out there, and PyPI does not make a difference whether the hit was from a human behind pip, or from a stack like zc.buildout Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 11:54 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 09:51:03 + (UTC) Vinay Sajipvinay_sa...@yahoo.co.uk wrote: Antoine Pitrousolipsisat pitrou.net writes: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? As an incentive for users to start using the features that are finished enough, and exercise the migration path from distutils. The module can be marked provisional so as to allow further API variations. It's true that some modules are quite mature and already useful: - packaging.version (PEP 386) - packaging.pypi - packaging.metadata (PEP 345) - packaging.database (PEP 386) the part that is not ready is the installer and some setuptools bridging Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 12:30:51 +0200 Tarek Ziadé ta...@ziade.org wrote: Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? I started there, remember ? And we ended up saying it was impossible to continue without breaking the packaging world. we were only certain people, AFAIR. why would you expect a different datapoint ? I wasn't expecting a different datapoint, I'm pointing that shipping packaging in the stdlib would provide a much better exposure. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 12:39 PM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 12:30:51 +0200 Tarek Ziadéta...@ziade.org wrote: Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? I started there, remember ? And we ended up saying it was impossible to continue without breaking the packaging world. we were only certain people, AFAIR. That was the BDFL decision after a language summit. Having tried to innovate in Distutils in the past, I think it's a very good decision, ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Am 20.06.2012 12:39, schrieb Antoine Pitrou: On Wed, 20 Jun 2012 12:30:51 +0200 Tarek Ziadé ta...@ziade.org wrote: Most of the distutils2 improvements (new PEPs, setup.cfg, etc.) were totally possible in distutils, weren't they? I started there, remember ? And we ended up saying it was impossible to continue without breaking the packaging world. we were only certain people, AFAIR. Yes. The people willing to work on packaging in Python, to be exact. why would you expect a different datapoint ? I wasn't expecting a different datapoint, I'm pointing that shipping packaging in the stdlib would provide a much better exposure. But not if it's not ready for prime time. (And providing the finished distutils2 for Python 2 will provide even better exposure at the moment.) Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 20 June 2012 10:12, Antoine Pitrou solip...@pitrou.net wrote: I think the whole idea that distutils should be frozen and improvements should only go in distutils2 has been misled. Had distutils been improved instead, many of those enhancements would already have been available in 3.2 (and others would soon be released in 3.3). The problem seems to be that in the past, any changes in distutils have been met with cries of you can't do that, basically because the lack of a clearly distinguished extension API means that the assumption is that for any bit of the internals, someone somewhere has monkeypatched or relied on it. The issue is compounded by the fact that a lot of distutils is undocumented, or at least badly documented, so saying if it's not documented, it's internal doesn't work :-( Maybe if we could be a bit more aggressive in saying what counts as internal and resist the cries of but I use it, modifying distutils might be a more viable option, But there doesn't seem to be anyone willing to take and defend that step. IIRC, Tarek proposed distutils2/packaging after getting frustrated with how little he could usefully do on distutils itself. Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. I see your point, but without sufficient developer resource, the question is whether packaging is in a usable state at all. Nobody but Éric is actually working on packaging (and as he says, even he is not able to at the moment), so what alternatives are there? I guess one extra option not mentioned by Éric is to make the packaging issues into release blocker bugs. That would effectively stall the release of 3.3 until packaging could be brought to an acceptable state, effectively a form of blackmail. I can't imagine anyone wants to take that approach. And yet, some of the existing bugs would clearly be release blockers if they were in any other part of Python. I think the first question is, do we need an enhanced distutils in the stdlib? As far as I can see, this one has been answered strongly, in the affirmative, a few times now. And yet, the need seems to be a diffuse thing, with no real champion (Tarek and Éric have both tried to take that role, and both appear to have been unable to invest the necessary amount of time - which doesn't surprise me, it seems to be a massive task). Removing packaging from 3.3, in my mind acknowledges that as it stands the approach was a failed experiment[1]. Better to get it taken out before it appears in a released version of Python. We need to rethink the approach. I see a number of options going forward, all of which are based round the need to ensure enough developer involvement, so that Tarek and Éric get help, and don't simply burn out before we have anything useful. 1. Reconsider the decision to freeze distutils, with a view to migrating incrementally to the feature set we want from packaging. That'll be hard as we'd need to take a much stronger line on making changes that could break existing code (stronger in the sense of so we broke your code, tough - you were using undocumented/internal APIs). And I suspect Tarek wouldn't be willing to take this route, so we immediately lose one resource. Maybe the other core developers could take up the slack, though. For example, Antoine, you seem to be implying that you would have worked on distutils if this had happened. 2. Free up distutils2 to develop as an external package, and then have a PEP proposing its inclusion in the stdlib in due course, when it is ready and has been proven in the wild. The benefit here is that I assume that as a separate project, becoming a committer would be easier than becoming a Python core developer, so there's a wider pool of developers. The downside is that the timescales would be a lot longer (I doubt we'd see anything in 3.4 this way, and maybe not even 3.5). 3. Write a PEP describing precisely what the packaging module will do, get consensus/agreement, and then restart development based on a solid scope and spec. This is the correct route for getting something added direct to the stdlib, and it's a shame it didn't happen in the first place for packaging. Having said that, the PEP would likely be huge, given the scope of packaging, and would require a lot of time from a champion. There's no guarantee that championing a PEP wouldn't burn someone out just as rapidly as developing the module itself :-( And also, given that the packaging documentation is one of its weak areas, I'd have to say I have concerns as to whether a PEP would come together in any case... The assumption here, though, is that the PEP process creates the debate, and results in interested parties coming together in the discussion. If we can keep that momentum, we get a pool of interested developers who may well assist in the coding aspects. The one option I don't like is taking packaging out, releasing 3.3, and then putting it straight back in as is, and simply carry on as now,
Re: [Python-Dev] Status of packaging in 3.3
On 20 June 2012 11:34, Tarek Ziadé ta...@ziade.org wrote: On 6/20/12 11:54 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 09:51:03 + (UTC) Vinay Sajipvinay_sa...@yahoo.co.uk wrote: Antoine Pitrousolipsisat pitrou.net writes: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? As an incentive for users to start using the features that are finished enough, and exercise the migration path from distutils. The module can be marked provisional so as to allow further API variations. It's true that some modules are quite mature and already useful: - packaging.version (PEP 386) - packaging.pypi - packaging.metadata (PEP 345) - packaging.database (PEP 386) the part that is not ready is the installer and some setuptools bridging I've never seen that information mentioned before. So that's (good) news. A question, then. Why is it not an option to: 1. Rip out all bar those 4 modules. 2. Make sure they are documented and tested solidly (they may already be, I don't know). 3. Declare that to be what packaging *is* for Python 3.3. Whether any of those modules are of any use in isolation, is a slightly more complex question. As is whether the APIs are guaranteed to be sufficient for further development on the rest of packaging, given that by doing this we commit to API stability and backward compatibility. Your comment quite mature and already useful is not quite firm enough to reassure me that we're ready to set those modules in stone (although presumably the 3 relating to the PEPs are, simply because they implement what the PEPs say). Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 06/20, Antoine Pitrou wrote: Let's make things clear: packaging is suffering from a lack of developer involvement, Absolutely. And to be more precise: solid hands-on leadership. Eric wrote it in his original mail: both packaging maintainers are burned out/busy. That’s a state that is very unlikely to attract more developers – myself included. and a lack of user interest. Maybe I'm getting you wrong here, but ISTM that proper packaging is in the short list on nearly everybody’s “things I wish they'd fix in Python”. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 1:19 PM, Paul Moore wrote: On 20 June 2012 11:34, Tarek Ziadéta...@ziade.org wrote: On 6/20/12 11:54 AM, Antoine Pitrou wrote: On Wed, 20 Jun 2012 09:51:03 + (UTC) Vinay Sajipvinay_sa...@yahoo.co.ukwrote: Antoine Pitrousolipsisatpitrou.netwrites: Deciding to remove packaging from 3.3 is another instance of the same mistake, IMO. What's the rationale for leaving it in, when it's known to be incomplete/unfinished? As an incentive for users to start using the features that are finished enough, and exercise the migration path from distutils. The module can be marked provisional so as to allow further API variations. It's true that some modules are quite mature and already useful: - packaging.version (PEP 386) - packaging.pypi - packaging.metadata (PEP 345) - packaging.database (PEP 386) the part that is not ready is the installer and some setuptools bridging I've never seen that information mentioned before. So that's (good) news. A question, then. Why is it not an option to: 1. Rip out all bar those 4 modules. 2. Make sure they are documented and tested solidly (they may already be, I don't know). 3. Declare that to be what packaging *is* for Python 3.3. Whether any of those modules are of any use in isolation, is a slightly more complex question. As is whether the APIs are guaranteed to be sufficient for further development on the rest of packaging, given that by doing this we commit to API stability and backward compatibility. Your comment quite mature and already useful is not quite firm enough to reassure me that we're ready to set those modules in stone (although presumably the 3 relating to the PEPs are, simply because they implement what the PEPs say). The last time I checked: packaging.version is the implementation of PEP 386, and stable. It's one building block that would be helpful as-is in the stdlib. it's completely standalone. packaging.metadata is the implementation of all metadata versions. standalone too. packaging.pypi is the PyPI crawler, and has fairly advanced features. I defer to Alexis to tell us is it's completely stable packaging.database is where PEP 376 is. It has the most innovations, implements PEP 376 packaging.config is the setup.cfg reader. Ity's awseome because together with packaging.database and packaging.markers, it gives you OS-independant data files. see http://docs.python.org/dev/packaging/setupcfg.html#resources Yeah maybe this subset could be left in 3.3 and we'd remove packaging-the-installer part (pysetup, commands, compilers) I think it's a good idea ! Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 13:20:04 +0200 Hynek Schlawack h...@ox.cx wrote: and a lack of user interest. Maybe I'm getting you wrong here, but ISTM that proper packaging is in the short list on nearly everybody’s “things I wish they'd fix in Python”. I agree, but I think people have also been burnt by the setuptools maintenance problem, the setuptools - distribute migration, the easy_install / pip duality, and other stuff. I'm not sure they want to try out yet another third-party distutils improvement from the cheeseshop. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Raw string syntax inconsistency
On Wed, Jun 20, 2012 at 7:22 PM, Christian Heimes li...@cheimes.de wrote: Am 18.06.2012 17:12, schrieb Guido van Rossum: Ok, banning ru... and ur... altogether is fine too (assuming it's fine with the originators of the PEP). It's gone for good. http://hg.python.org/cpython/rev/8e47e9af826e (My first push for a very long time. Man, that feels good!) And I just pushed an update to PEP 414 that should appear on the site soon. There's probably more text in that section than the issue really deserves, but at least it's captured now :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, 20 Jun 2012 12:11:03 +0100 Paul Moore p.f.mo...@gmail.com wrote: I think the first question is, do we need an enhanced distutils in the stdlib? I would answer a different question: we definitely need a better distutils/packaging story. Whether it's in the form of distutils enhancements, or another package, is not clear-cut. By the way, let me point out that the distutils freeze has been broken to implement PEP 405 (I approve the breakage myself): http://hg.python.org/cpython/rev/294f8aeb4d4b#l4.1 As far as I can see, this one has been answered strongly, in the affirmative, a few times now. And yet, the need seems to be a diffuse thing, with no real champion Packaging is not a very motivating topic for many developers (myself included). It's like the build process or the buildbot fleet :-) 2. Free up distutils2 to develop as an external package, and then have a PEP proposing its inclusion in the stdlib in due course, when it is ready and has been proven in the wild. [...] The downside is that the timescales would be a lot longer (I doubt we'd see anything in 3.4 this way, and maybe not even 3.5). Agreed, especially if the proven in the wild criterion is required (people won't rush to another third-party distutils replacement, IMHO). 3. Write a PEP describing precisely what the packaging module will do, get consensus/agreement, and then restart development based on a solid scope and spec. I think it's the best way to sink the project definitively. Our community isn't organized for such huge monolithic undertakings. [1] That reads really harshly. I don't mean to criticise any of the work that's been done, I'm a huge fan of the idea of packaging, and its goals. The experiment in this case is around process - developing something as big and important as packaging with limited developer resource, relatively directly in the core (bringing it in from distutils2 sooner rather than later) and working from a series of smaller PEPs focused on particular details, rather than an overall one covering the whole package. I cannot speak for Tarek, but one of the reasons it's been done as a set of smaller PEPs is that these PEPs were meant to be included in *distutils*, not distutils2. That is, the module already existed and the PEPs were individual, incremental improvements. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, Jun 20, 2012 at 9:31 PM, Tarek Ziadé ta...@ziade.org wrote: Yeah maybe this subset could be left in 3.3 and we'd remove packaging-the-installer part (pysetup, commands, compilers) I think it's a good idea ! OK, to turn this into a concrete suggestion based on the packaging docs. Declare stable, include in 3.3 -- packaging.version — Version number classes packaging.metadata — Metadata handling packaging.markers — Environment markers packaging.database — Database of installed distributions Maybe needed as dependencies for above? packaging.errors — Packaging exceptions packaging.util — Miscellaneous utility functions It seems to me that stripping the library, docs and tests back to just these 4 modules and their dependencies shouldn't be much harder than stripping packaging in its entirety, but my question is what benefit would we gain from having these (and just these) in the 3.3 stdlib over having them available on PyPI in distutils2? Third party projects over the next couple of years are going to want to support more than just 3.3, so simply depending on distutils2 for this functionality seems like a far more sensible option. OTOH, it does send a clear message that progress *is* being made, we just tried to take too big a jump from implementing these lower level standards up to wholesale replacement of distutils without first clearly documenting exactly what was wrong with the status quo and what we wanted to change about it as a sequence of PEPs. I've broken up the rest of packaging's functionality below into a series of candidate PEPs that may be more manageable than a single monolothic fix packaging PEP. If we can get multiple interested parties focusing on different aspects, that may also help with reducing the burnout factor. Python's current packaging and distribution story is held together with duct tape and baling wire due to decisions that were made years ago - unwinding some of those decisions and replacing them with an alternative that is built on a solid architectural foundation backed by some level of community consensus is *not* an easy task, and not one that will be completed quickly (undue haste will fail the some level of community consensus part, thus missing much of the point of the exercise). That said, I don't think it's unsolvable either, and there's a reasonable chance to get something cleaner in place for 3.4. 3.4 PEP: Distutils replacement: Packaging, Distribution Installation # This is one of the big balls of mud w.r.t distutils where third party projects dig deep into the implementation details because that is the only way to get things done # It may even be the case that this can be broken up even further packaging.install — Installation tools packaging.dist — The Distribution class packaging.manifest — The Manifest class packaging.command — Standard Packaging commands packaging.command.cmd — Abstract base class for Packaging commands 3.4 PEP: Distutils replacement: Compiling Extension Modules # Another big ball of mud. It sounds like the Gentoo folks may have some feedback in this space. packaging.compiler — Compiler classes packaging.compiler.ccompiler — CCompiler base class packaging.compiler.extension — The Extension class 3.4 PEP: Standard library package downloader (pysetup) -- # Amongst other things, this needs to have a really good security story (refusing to install unsigned packages by default, etc) packaging.depgraph — Dependency graph builder packaging.pypi — Interface to projects indexes packaging.pypi.client — High-level query API packaging.pypi.base — Base class for index crawlers packaging.pypi.dist — Classes representing query results packaging.pypi.simple — Crawler using the PyPI “simple” interface packaging.pypi.xmlrpc — Crawler using the PyPI XML-RPC interface packaging.tests.pypi_server — PyPI mock server packaging.fancy_getopt — Wrapper around the getopt module # Why does this exist? 3.4 PEP: Simple binary package distribution format -- bdist_simple has been discussed enough times, finally seeing a PEP for it would be nice :) I think the main lesson to be learned here is that fix packaging is simply too big a task to be managed sensibly. Smaller goals like Standardise versioning, Fix package metadata, Support uninstall are hard enough to achieve, but also provide concrete milestones along the way to the larger goal. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev
Re: [Python-Dev] PEP 362 minor nits
On 2012-06-20, at 4:30 AM, Steven D'Aprano wrote: On Tue, Jun 19, 2012 at 08:11:26PM -0400, Yury Selivanov wrote: So using the signature will be OK for 'Foo.bar' and 'Foo().bar', but not for 'Foo.__dict__['bar']' - which I think is fine (since staticmethod classmethod instances are not callable) There has been some talk on Python-ideas about making staticmethod and classmethod instances callable. Speaking of non-instance method descriptors, please excuse this silly question, I haven't quite understood the implementation well enough to answer this question myself. Is there anything needed to make signature() work correctly with custom method-like descriptors such as this? http://code.activestate.com/recipes/577030-dualmethod-descriptor Well, as Nick said -- the PEP way is to create a new Signature with a first parameter skipped. But in this particular case you can rewrite it (I'd say preferred way): class dualmethod: def __init__(self, func): self.func = func def __get__(self, instance, owner): if instance is None: return types.MethodType(self.func, owner) else: return types.MethodType(self.func, instance) Or another way, using functools.partial: class dualmethod: def __init__(self, func): self.func = func def __get__(self, instance, owner): if instance is None: return functools.partial(self.func, owner) else: return functools.partial(self.func, instance) Since 'MethodType' and 'partial' are supported by signature(), everything will work automatically (i.e. first argument will be skipped) - Yury ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, Jun 20, 2012 at 9:46 PM, Antoine Pitrou solip...@pitrou.net wrote: Agreed, especially if the proven in the wild criterion is required (people won't rush to another third-party distutils replacement, IMHO). The existence of setuptools means that proven in the wild is never going to fly - a whole lot of people use setuptools and easy_install happily, because they just don't care about the downsides it has in terms of loss of control of a system configuration. I cannot speak for Tarek, but one of the reasons it's been done as a set of smaller PEPs is that these PEPs were meant to be included in *distutils*, not distutils2. That is, the module already existed and the PEPs were individual, incremental improvements. That initial set of PEPs were also aimed at defining interoperability standards that multiple projects could implement independently, even *without* support in the standard library. As I wrote in my other email, I think one key aspect of where we went wrong after that point was in never clearly spelling out just what we collectively meant by fix packaging. Most of the burden of interpreting that phrase thus landed directly on the shoulders of the distutils2 project lead. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 20 June 2012 13:53, Nick Coghlan ncogh...@gmail.com wrote: [...] 3.4 PEP: Simple binary package distribution format -- bdist_simple has been discussed enough times, finally seeing a PEP for it would be nice :) I had a PEP for this one part written - Éric had a brief look at it but I never posted it publicly. Before it'll go anywhere, a bit more of the infrastructure PEPs you mentioned and I trimmed would need to be completed, but I'd be willing to resurrect it when we get to that stage... Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 20/06/2012 14:53, Nick Coghlan wrote: 3.4 PEP: Standard library package downloader (pysetup) -- # Amongst other things, this needs to have a really good security story (refusing to install unsigned packages by default, etc) packaging.depgraph — Dependency graph builder packaging.pypi — Interface to projects indexes packaging.pypi.client — High-level query API packaging.pypi.base — Base class for index crawlers packaging.pypi.dist — Classes representing query results packaging.pypi.simple — Crawler using the PyPI “simple” interface packaging.pypi.xmlrpc — Crawler using the PyPI XML-RPC interface packaging.tests.pypi_server — PyPI mock server packaging.fancy_getopt — Wrapper around the getopt module # Why does this exist? I'm okay and willing to work on this part. I started a full review of the code I wrote years ago, and which clearly needs some cleaning. Alos, I'm not sure to understand what having a PEP to manage this means: should I describe all the API in a text document (with examples) so we can discuss this and validate before doing the changes/cleanings to the API? Alexis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 20/06/2012 13:31, Tarek Ziadé wrote: packaging.metadata is the implementation of all metadata versions. standalone too. packaging.pypi is the PyPI crawler, and has fairly advanced features. I defer to Alexis to tell us is it's completely stable packaging.pypi is functionally working but IMO the API can (and probably should) be improved (we really lack feedback to know that). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, Jun 20, 2012 at 11:19 PM, Alexis Métaireau ale...@notmyidea.org wrote: On 20/06/2012 14:53, Nick Coghlan wrote: 3.4 PEP: Standard library package downloader (pysetup) -- # Amongst other things, this needs to have a really good security story (refusing to install unsigned packages by default, etc) packaging.depgraph — Dependency graph builder packaging.pypi — Interface to projects indexes packaging.pypi.client — High-level query API packaging.pypi.base — Base class for index crawlers packaging.pypi.dist — Classes representing query results packaging.pypi.simple — Crawler using the PyPI “simple” interface packaging.pypi.xmlrpc — Crawler using the PyPI XML-RPC interface packaging.tests.pypi_server — PyPI mock server packaging.fancy_getopt — Wrapper around the getopt module # Why does this exist? I'm okay and willing to work on this part. I started a full review of the code I wrote years ago, and which clearly needs some cleaning. Alos, I'm not sure to understand what having a PEP to manage this means: should I describe all the API in a text document (with examples) so we can discuss this and validate before doing the changes/cleanings to the API? There would be two main parts to such a PEP: - defining the command line interface and capabilities (pysetup) - defining the programmatic API (packaging.pypi and the dependency graph management) I would suggest looking at PEP 405 (venv) and PEP 397 (Windows launcher) to get an idea of the kind of content that might be appropriate. It's definitely not necessary to reproduce the full API details verbatim in the PEP text - it's OK to provide highlights and point to a reference implementation for the full details. The PEP process can also be a good way to get feedback on an API design that otherwise may not be forthcoming (cf. the recent inspect.Signature discussions). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Le mer. 20 juin 2012 15:28:56 CEST, Nick Coghlan a écrit : There would be two main parts to such a PEP: - defining the command line interface and capabilities (pysetup) - defining the programmatic API (packaging.pypi and the dependency graph management) Okay. I don't think that the command line has anything to do with packaging.pypi and dependency management tools. One is dealing with the hole cli for different tings (install / remove / search etc.) while the other one is only how to communicate with the indexes and build dependency graphs from there. We probably should put the cli part in a separate PEP, as the scopes aren't the same that the ones I see for packaging.pypi / depgraph I would suggest looking at PEP 405 (venv) and PEP 397 (Windows launcher) to get an idea of the kind of content that might be appropriate. It's definitely not necessary to reproduce the full API details verbatim in the PEP text - it's OK to provide highlights and point to a reference implementation for the full details. Thanks for the pointers, will read them and try to come back with a PEP proposal. Alexis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 20 June 2012 14:16, Alexis Métaireau ale...@notmyidea.org wrote: On 20/06/2012 13:31, Tarek Ziadé wrote: packaging.metadata is the implementation of all metadata versions. standalone too. packaging.pypi is the PyPI crawler, and has fairly advanced features. I defer to Alexis to tell us is it's completely stable packaging.pypi is functionally working but IMO the API can (and probably should) be improved (we really lack feedback to know that). I wasn't aware of this - I've had a look and my first thought is that the documentation needs completing. At the moment, there's a lot that isn't documented, and we should avoid getting into the same situation as with distutils where people have to use undocumented APIs to get anything done. There are a lot of examples, but not so much formal API documentation. I don't mean to pick on this one module - unless things have changed a lot, the same is probably true of much of the rest of packaging. Lack of documentation is the #1 criticism I've seen. Are there people willing to do some serious documentation work to get the docs for the agreed stable parts of packaging complete? There's more time to do this (doc changes don't have to be done before the beta), but by deciding to retain parts of packaging, we *are* making a commitment to complete that documentation, in my view. Paul. PS packaging.pypi is nice - I wish I'd known of its existence for a bit of work I was doing a little while ago... ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Nick Coghlan ncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 9:46 PM, Antoine Pitrou solip...@pitrou.net wrote: Agreed, especially if the proven in the wild criterion is required (people won't rush to another third-party distutils replacement, IMHO). The existence of setuptools means that proven in the wild is never going to fly - a whole lot of people use setuptools and easy_install happily, because they just don't care about the downsides it has in terms of loss of control of a system configuration. Maybe not happily :-). Speaking for myself, I'd love to find an alternative, but setuptools seems to be the only system that knows how to build shared libraries across all my platforms. I've got little interest in a packaging module that doesn't include the compiler magic to do that. Bill ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Packaging documentation and packaging.pypi API
On 20 June 2012 14:47, Paul Moore p.f.mo...@gmail.com wrote:
On 20 June 2012 14:16, Alexis Métaireau ale...@notmyidea.org wrote:
packaging.pypi is functionally working but IMO the API can (and probably
should) be improved (we really lack feedback to know that).
I wasn't aware of this - I've had a look and my first thought is that
the documentation needs completing. At the moment, there's a lot that
isn't documented, and we should avoid getting into the same situation
as with distutils where people have to use undocumented APIs to get
anything done. There are a lot of examples, but not so much formal API
documentation.
As a specific example, one thing I would like to do is to be able to
set up a packaging.pypi client object that lets me query and download
distributions. However, rather than just querying PyPI (the default)
I'd like to be able to set up a list of locations (PyPI, a local
server, and maybe some distribution files stored on my PC) and combine
results from all of them. This differs from the mirror support in that
I want to combine the lists, not use one as a fallback if the other
doesn't exist. From the documentation, I can't tell if this is
possible, or a feature request, or unsupported... (Actually, there's
not even any documentation saying how the URL(s) in index_url should
behave, so how exactly do I set up a local repository anyway...?)
On a similar note, at some point, crawler.get_releases('pywin32')
needs to work. I believe the issue here is technically with pywin32,
which uses non-standard version numbers (214) and is hosted externally
(Sourceforge) but I'd expect that packaging.pypi should be able to
access anything that's on PyPI, even if other APIs like
packaging.version can't deal with them.
Ideally, these would be simply things I'd raise as issues on
bugs.python.org. But as things stand, such issues aren't getting
fixed, and we don't move forward. And without the documentation to
back up a debate, it's hard to argue X is a bug, Y is a feature
request, Z is behaving correctly.
Paul.
PS As I write this, it suggests to me that maybe even the pick out
the stable APIs approach isn't as simple as we'd like to think it is.
You can either read this email as a set of specific documentation
points to fix, or as evidence that we should drop packaging.pypi as
well for now, even though it's a pretty cool feature...
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Hi all, Sorry I can’t take the time to reply to all messages, this week I’m fully busy with work and moving out. To answer or correct a few things: - I am lacking time these months, but that’s because I’m still getting used to having a full-time job and being settled into a new country. With the feedback we’ve been getting from people recently, I am motivated, not burned out. - I have started building a group of distutils2 contributors here in Montreal. People are motivated, but it takes time to learn the codebase and tackle on the big things. - The four modules identified as minimal, standalone, good subset all have big problems (the PEPs have open issues, and the modules' APIs need improvements). - Tarek, Georg and Guido have pronounced. With all the respect I have for Antoine’s opinion, and the valid concerns he raises and that I don’t answer here, I consider option (d) accepted and will scrap one hour to do it before b1. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Prefer assertEqual to simply assert per recommendation in issue6727.
Am 20.06.2012 16:25, schrieb jason.coombs: http://hg.python.org/cpython/rev/24369f6c4a22 changeset: 77525:24369f6c4a22 user:Jason R. Coombs jar...@jaraco.com date:Wed Jun 20 10:24:24 2012 -0400 summary: Prefer assertEqual to simply assert per recommendation in issue6727. Clarified comment on disabled code to reference issue15093. files: Lib/test/test_import.py | 11 --- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_import.py b/Lib/test/test_import.py --- a/Lib/test/test_import.py +++ b/Lib/test/test_import.py @@ -707,14 +707,19 @@ os.mkdir(self.tagged) init_file = os.path.join(self.tagged, '__init__.py') open(init_file, 'w').close() -assert os.path.exists(init_file) +self.assertEqual(os.path.exists(init_file), True) # now create a symlink to the tagged package # sample - sample-tagged os.symlink(self.tagged, self.package_name) -# assert os.path.isdir(self.package_name) # currently fails -assert os.path.isfile(os.path.join(self.package_name, '__init__.py')) +# disabled because os.isdir currently fails (see issue 15093) +# self.assertEqual(os.path.isdir(self.package_name), True) + +self.assertEqual( +os.path.isfile(os.path.join(self.package_name, '__init__.py')), +True, +) Actually, in this case self.assertTrue() is the correct method. cheers, Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
Am 20.06.2012 17:34, schrieb Éric Araujo: Hi all, Sorry I can’t take the time to reply to all messages, this week I’m fully busy with work and moving out. To answer or correct a few things: - I am lacking time these months, but that’s because I’m still getting used to having a full-time job and being settled into a new country. With the feedback we’ve been getting from people recently, I am motivated, not burned out. - I have started building a group of distutils2 contributors here in Montreal. People are motivated, but it takes time to learn the codebase and tackle on the big things. - The four modules identified as minimal, standalone, good subset all have big problems (the PEPs have open issues, and the modules' APIs need improvements). Tarek seems to think otherwise... looks like in the end, this subset could only be included as provisional. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging documentation and packaging.pypi API
Hi Paul, On 06/20/2012 09:29 AM, Paul Moore wrote: As a specific example, one thing I would like to do is to be able to set up a packaging.pypi client object that lets me query and download distributions. However, rather than just querying PyPI (the default) I'd like to be able to set up a list of locations (PyPI, a local server, and maybe some distribution files stored on my PC) and combine results from all of them. This differs from the mirror support in that I want to combine the lists, not use one as a fallback if the other doesn't exist. From the documentation, I can't tell if this is possible, or a feature request, or unsupported... (Actually, there's not even any documentation saying how the URL(s) in index_url should behave, so how exactly do I set up a local repository anyway...?) This is perhaps a tangent, as your point here is to point out what the API of packaging.pypi ought to allow - but pip's PackageFinder class can easily do exactly this for you. Feel free to follow up with me for details if this is actually still a problem you need to solve. Carl ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On 6/20/12 5:44 PM, Georg Brandl wrote: Am 20.06.2012 17:34, schrieb Éric Araujo: Hi all, Sorry I can’t take the time to reply to all messages, this week I’m fully busy with work and moving out. To answer or correct a few things: - I am lacking time these months, but that’s because I’m still getting used to having a full-time job and being settled into a new country. With the feedback we’ve been getting from people recently, I am motivated, not burned out. - I have started building a group of distutils2 contributors here in Montreal. People are motivated, but it takes time to learn the codebase and tackle on the big things. - The four modules identified as minimal, standalone, good subset all have big problems (the PEPs have open issues, and the modules' APIs need improvements). Tarek seems to think otherwise... looks like in the end, this subset could only be included as provisional. I defer to Eric -- My answers are probably missing recent changes he knows Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ziade.tarek%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging documentation and packaging.pypi API
On 20 June 2012 17:07, Carl Meyer c...@oddbird.net wrote: Hi Paul, On 06/20/2012 09:29 AM, Paul Moore wrote: As a specific example, one thing I would like to do is to be able to set up a packaging.pypi client object that lets me query and download distributions. However, rather than just querying PyPI (the default) I'd like to be able to set up a list of locations (PyPI, a local server, and maybe some distribution files stored on my PC) and combine results from all of them. This differs from the mirror support in that I want to combine the lists, not use one as a fallback if the other doesn't exist. From the documentation, I can't tell if this is possible, or a feature request, or unsupported... (Actually, there's not even any documentation saying how the URL(s) in index_url should behave, so how exactly do I set up a local repository anyway...?) This is perhaps a tangent, as your point here is to point out what the API of packaging.pypi ought to allow - but pip's PackageFinder class can easily do exactly this for you. Feel free to follow up with me for details if this is actually still a problem you need to solve. Thanks - as you say, it's not so much the actual problem as the principle of what the packaging API offers that matters here. Although it does make a good point - to what extent do the packaging APIs draw on existing experience like that of pip? Given that tools like pip are used widely to address real requirements, it would seem foolish to *not* draw on that experience in designing a stdlib API. (As regards my actual use, it's very much a back burner project of mine - I keep dabbling with writing utilities to grab bdist_wininst installers and unpack/install them in virtualenvs. The PackageFinder class might well be useful there. I'll keep it in mind for the next time I go back to that problem). Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Accepting PEP 397
As the PEP czar for 397, after Martin's final updates, I hereby pronounce this PEP accepted! Thanks to Mark Hammond for kicking it off, Vinay Sajip for writing up the code, Martin von Loewis for recent updates, and everyone in the community who contributed to the discussions. I will begin integration work this evening. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging documentation and packaging.pypi API
Le mer. 20 juin 2012 18:45:23 CEST, Paul Moore a écrit : Thanks - as you say, it's not so much the actual problem as the principle of what the packaging API offers that matters here. Although it does make a good point - to what extent do the packaging APIs draw on existing experience like that of pip? Given that tools like pip are used widely to address real requirements, it would seem foolish to *not* draw on that experience in designing a stdlib API. IIRC, pip relies nly onthe XML/RPC API to get information about the distributions from the cheeseshop. the code that's in packaging.pypi was built with the implementation in setuptools in mind, so we keep compatibility with setuptools easy_install. But this is for the internal implementation. You're right and I will have a deep look at what the API in pip looks like when it comes to download packages from PyPI. That is, this leverages one question more on my side: was/is pip intended to be used as a library rather than as a tool / are there some people that are actually building tools on top of pip this way? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging documentation and packaging.pypi API
On 20/06/2012 17:29, Paul Moore wrote:
I wasn't aware of this - I've had a look and my first thought is that
the documentation needs completing. At the moment, there's a lot that
isn't documented, and we should avoid getting into the same situation
as with distutils where people have to use undocumented APIs to get
anything done. There are a lot of examples, but not so much formal API
documentation.
So that's something we definitely want to fix. The code is heavily annotated,
and this had been made to generate the documentation automatically with sphinx
in the first time, so… that would make no sense to not make it.
This is for the format API documentation, which seems to be easy to hook to
sphinx.
I'll also review all the documentation there to make sure that it perfectly makes sense.
As a specific example, one thing I would like to do is to be able to
set up a packaging.pypi client object that lets me query and download
distributions. However, rather than just querying PyPI (the default)
I'd like to be able to set up a list of locations (PyPI, a local
server, and maybe some distribution files stored on my PC) and combine
results from all of them. This differs from the mirror support in that
I want to combine the lists, not use one as a fallback if the other
doesn't exist. From the documentation, I can't tell if this is
possible, or a feature request, or unsupported... (Actually, there's
not even any documentation saying how the URL(s) in index_url should
behave, so how exactly do I set up a local repository anyway...?)
that's not something possible out of the box using the crawler the way
they are defined (iow, that's not one supported use case), *but* it's
possible to make a class on top of the existing ones which could provide
this kind of fallback feature. I'm not sure that we want or don't want
that to be a part of packaging.pypi, but that's definitely something
that this API makes possible without too much trouble.
On a similar note, at some point, crawler.get_releases('pywin32')
needs to work. I believe the issue here is technically with pywin32,
which uses non-standard version numbers (214) and is hosted externally
(Sourceforge) but I'd expect that packaging.pypi should be able to
access anything that's on PyPI, even if other APIs like
packaging.version can't deal with them.
If this is not working / following the links that are present in the
cheeseshp then this should be considered a bug.
Ideally, these would be simply things I'd raise as issues on
bugs.python.org. But as things stand, such issues aren't getting
fixed, and we don't move forward. And without the documentation to
back up a debate, it's hard to argue X is a bug, Y is a feature
request, Z is behaving correctly.
Alright, so this is a true documentation issue. I will make it clearer
what packaging.pypi makes and doesn't make possible.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging documentation and packaging.pypi API
Hi Alexis, On 06/20/2012 10:57 AM, Alexis Métaireau wrote: Le mer. 20 juin 2012 18:45:23 CEST, Paul Moore a écrit : Thanks - as you say, it's not so much the actual problem as the principle of what the packaging API offers that matters here. Although it does make a good point - to what extent do the packaging APIs draw on existing experience like that of pip? Given that tools like pip are used widely to address real requirements, it would seem foolish to *not* draw on that experience in designing a stdlib API. IIRC, pip relies nly onthe XML/RPC API to get information about the distributions from the cheeseshop. the code that's in packaging.pypi was built with the implementation in setuptools in mind, so we keep compatibility with setuptools easy_install. No, this is not accurate. Pip's PackageFinder uses setuptools-compatible link-scraping, not the XMLRPC API, and it is the PackageFinder that is used to actually find distributions to install. I think PackageFinder is pretty much equivalent to what packaging.pypi is intended to do. Pip does have a separate search command that uses the XMLRPC API - this is the only part of pip that uses XMLRPC. I consider this a bug in pip, because the results can be inconsistent with actual installation using PackageFinder, and search can't be used with mirrors or private indexes (unless they implement the XMLRPC API). The search command should just use PackageFinder instead. That is, this leverages one question more on my side: was/is pip intended to be used as a library rather than as a tool / are there some people that are actually building tools on top of pip this way? Pip's internal APIs are not documented, and they aren't the cleanest APIs ever, but some of them (particularly PackageFinder and InstallRequirement/RequirementSet) can be used without too much difficulty, and some people are using them. Not a lot of people, I don't think; I don't have hard numbers. I haven't seen much in the way of public reusable tools built atop pip, but I've talked with a few people building internal deployment tools that use pip as a library. Carl ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Wed, Jun 20, 2012 at 9:02 AM, Nick Coghlan ncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 9:46 PM, Antoine Pitrou solip...@pitrou.net wrote: Agreed, especially if the proven in the wild criterion is required (people won't rush to another third-party distutils replacement, IMHO). The existence of setuptools means that proven in the wild is never going to fly - a whole lot of people use setuptools and easy_install happily, because they just don't care about the downsides it has in terms of loss of control of a system configuration. Um, this may be a smidge off topic, but what loss of control are we talking about here? AFAIK, there isn't anything it does that you can't override with command line options or the config file. (In most cases, standard distutils options or config files.) Do you just mean that most people use the defaults and don't care about there being other options? And if that's the case, which other options are you referring to? If the long-term goal is to draw setuptools users over to packaging, then AFAIK the packaging effort is still missing a few things, like build-time dependencies and alternatives to setuptools' entry points and extras, as well as the ability to integrate version control for building sdists (without requiring the sdist's recipient to *also* have the version control integration in order to build the package or recreate a new sdist). These are just the missing features that I know of, from recent distutils-sig discussions; I don't know how complete a list this is. While no single one of these features is directly used by every project or even a majority of such projects, there is a correlation between size of a project and the likelihood that they are depending on one or more of these features. i.e., the bigger and more widely-used the project, the more likely it is to either use one of these features, or depend on a project that does. Some of these features could be built on top of packaging, in more or less the same way setuptools is built on top of distutils. But whether they're done inside or outside of the packaging library, somebody's going to have to do them, for people to be able to migrate off of setuptools. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Packaging documentation and packaging.pypi API
Hi Carl, Thanks for clarifying this. This means that indeed we have the same goals. I'll have a closer look at the internal pip APIs, as they are probably really useful and already used in production environment :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Status of packaging in 3.3
On Thu, Jun 21, 2012 at 3:29 AM, PJ Eby p...@telecommunity.com wrote: On Wed, Jun 20, 2012 at 9:02 AM, Nick Coghlan ncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 9:46 PM, Antoine Pitrou solip...@pitrou.net wrote: Agreed, especially if the proven in the wild criterion is required (people won't rush to another third-party distutils replacement, IMHO). The existence of setuptools means that proven in the wild is never going to fly - a whole lot of people use setuptools and easy_install happily, because they just don't care about the downsides it has in terms of loss of control of a system configuration. Um, this may be a smidge off topic, but what loss of control are we talking about here? AFAIK, there isn't anything it does that you can't override with command line options or the config file. (In most cases, standard distutils options or config files.) Do you just mean that most people use the defaults and don't care about there being other options? And if that's the case, which other options are you referring to? No, I mean there are design choices in setuptools that explain why many people don't like it and are irritated when software they want to use depends on it without a good reason. Clearly articulating the reasons that just include setuptools is no longer being considered as an option should be one of the goals of any PEPs associated with adding packaging back for 3.4. The reasons I'm personally aware of: - it's a unilateral runtime fork of the standard library that bears a lot of responsibility for the ongoing feature freeze in distutils. Standard assumptions about the behaviour of site and distutils cease to be valid once setuptools is installed - overuse of *.pth files and the associated sys.path changes for all Python programs running on a system. setuptools gleefully encourages the inclusion of non-trivial code snippets in *.pth files that will be executed by all programs. - advocacy for the egg format and the associated sys.path changes that result for all Python programs running on a system - too much magic that is enabled by default and is hard to switch off (e.g. http://rhodesmill.org/brandon/2009/eby-magic/) System administrators (and developers that think like system administrators when it comes to configuration management) *hate* what setuptools (and setuptools based installers) can do to their systems. It doesn't matter that package developers don't *have* to do those things - what matters is that the needs and concerns of system administrators simply don't appear to have been anywhere on the radar when setuptools was being designed. (If those concerns actually were taken into account at some point, it's sure hard to tell from the end result and the choices of default behaviour) setuptools is a masterful achievement built on shaky foundations that will work most of the time. However, when it doesn't work, you're probably screwed, and as soon as it's present on a system, you know that your assumptions about understanding the Python interpreter's startup sequences are probably off. The efforts around distutils2/packaging have been focused on taking the time to *fix the foundations first* rather than accepting the inevitable shortcomings of trying to build something in the middle of a swamp. If the long-term goal is to draw setuptools users over to packaging, then AFAIK the packaging effort is still missing a few things, like build-time dependencies and alternatives to setuptools' entry points and extras, as well as the ability to integrate version control for building sdists (without requiring the sdist's recipient to *also* have the version control integration in order to build the package or recreate a new sdist). Right - clearly enumerating the features that draw people to use setuptools over just using distutils should be a key element in any PEP for 3.4 I honestly think a big part of why packaging ended up being incomplete for 3.3 is that we still don't have a clearly documented answer to two critical questions: 1. Why do people choose setuptools over distutils? 2. What's wrong with setuptools that meant the idea of including it directly in the stdlib was ultimately dropped and eventually replaced with the goal of incorporating distutils2? I imagine there are answers to both of those questions embedded in past python-dev, distutils-sig, setuptools and distutils2 mailing list discussions, but that's no substitute for having them clearly documented in a PEP (or PEPs, given the scope of the questions). We've tried to shortcircuit this process twice now, first with just include setuptools back around 2.5, and again now with just include distutils2 as packaging for 3.3. It hasn't worked, so maybe it's time to try doing it properly and clearly articulating the desired end result. If the end goal is the bulk of the setuptools feature set without the problematic features and default behaviours that make system administrators break out the torches
Re: [Python-Dev] Status of packaging in 3.3
On 06/20/2012 11:57 PM, Nick Coghlan wrote: On Thu, Jun 21, 2012 at 3:29 AM, PJ Ebyp...@telecommunity.com wrote: On Wed, Jun 20, 2012 at 9:02 AM, Nick Coghlanncogh...@gmail.com wrote: On Wed, Jun 20, 2012 at 9:46 PM, Antoine Pitrousolip...@pitrou.net wrote: Agreed, especially if the proven in the wild criterion is required (people won't rush to another third-party distutils replacement, IMHO). The existence of setuptools means that proven in the wild is never going to fly - a whole lot of people use setuptools and easy_install happily, because they just don't care about the downsides it has in terms of loss of control of a system configuration. Um, this may be a smidge off topic, but what loss of control are we talking about here? AFAIK, there isn't anything it does that you can't override with command line options or the config file. (In most cases, standard distutils options or config files.) Do you just mean that most people use the defaults and don't care about there being other options? And if that's the case, which other options are you referring to? No, I mean there are design choices in setuptools that explain why many people don't like it and are irritated when software they want to use depends on it without a good reason. Clearly articulating the reasons that just include setuptools is no longer being considered as an option should be one of the goals of any PEPs associated with adding packaging back for 3.4. The reasons I'm personally aware of: - it's a unilateral runtime fork of the standard library that bears a lot of responsibility for the ongoing feature freeze in distutils. Standard assumptions about the behaviour of site and distutils cease to be valid once setuptools is installed - overuse of *.pth files and the associated sys.path changes for all Python programs running on a system. setuptools gleefully encourages the inclusion of non-trivial code snippets in *.pth files that will be executed by all programs. - advocacy for the egg format and the associated sys.path changes that result for all Python programs running on a system - too much magic that is enabled by default and is hard to switch off (e.g. http://rhodesmill.org/brandon/2009/eby-magic/) All of these are really pretty minor issues compared with the main benefit of not needing to ship everything with everything else. The killer feature is that developers can specify dependencies and users can have those dependencies installed automatically in a cross-platform way. Everything else is complete noise if this use case is not served. IMO, the second and third things you mention above (use of pth files and eggs) are actually features when compared against the result of something like pip, which installs things using --single-version-externally-managed and then tries to manage the resulting potentially-intertwined directories. Eggs are *easier* to manage than files potentially overlapping files and directories installed into some other directory. Either they exist or they don't. Either they're mentioned in a .pth file or they aren't. It's not really that hard. In any case, any tool that tries to manage distribution installation will need somewhere to keep distribution metadata. It's a minor mystery to me why people think it could be done much better than in something very close to egg format. System administrators (and developers that think like system administrators when it comes to configuration management) *hate* what setuptools (and setuptools based installers) can do to their systems. It doesn't matter that package developers don't *have* to do those things - what matters is that the needs and concerns of system administrators simply don't appear to have been anywhere on the radar when setuptools was being designed. (If those concerns actually were taken into account at some point, it's sure hard to tell from the end result and the choices of default behaviour) I think you mean easy_install here. And I guess you mean managing .pth files. Note that if you use pip, neither thing needs to happen. And even easy_install lets you install a distribution that way (with --single-version-externally-managed). So I think, as you mention, this is a matter of defaults (tool and or flag defaults) rather than core functionality. setuptools is a masterful achievement built on shaky foundations that will work most of the time. However, when it doesn't work, you're probably screwed, and as soon as it's present on a system, you know that your assumptions about understanding the Python interpreter's startup sequences are probably off. It's true setuptools is based on shaky foundations. The rest of the stuff you say above is pretty darn specious, I think. The efforts around distutils2/packaging have been focused on taking the time to *fix the foundations first* rather than accepting the inevitable shortcomings of trying to build something in the middle of a swamp. If the
