Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 10:35 AM, Dino Viehland di...@microsoft.com wrote: Stefan wrote: It /does/ make some static assumptions in that it considers builtins true builtins. However, it does not prevent you from replacing them in your code, as long as you do it inside the module. Certainly a restriction compared to Python, where you can import a module into a changed dict environment that redefines 'object', but not a major restriction IMO, and certainly not one that impacts much code. To me this is a deal breaker which prevents Cython from being a Python implementation. From a talk given by Colin Winter at the LLVM dev meeting (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to do something like this as well and Guido said no. In this case the breaking change is so subtle that I'd personally hate to run into something like this porting code to Cython and having to figure out why it's not working. To clarify, I was joking when I told that story (or at least I was joking with Guido when I asked him if we could break that). It clearly *would* be easier if we could just ignore this point of Python compatibility, but that's not an option, so we've had to optimize around it. It's not that hard to do, but it's still extra work. Collin Winter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
At 11:27 PM 11/4/2009 +, Floris Bruynooghe wrote: cynical mode You have time to read this thread but no time to read What's New In Python 3? /cynical mode Given that the average developer using tons of existing libraries on 2.x is unlikely to see any killer benefits in moving to 3.x, it's a natural attitude to have. I fought this same battle with setuptools for a long time before it sank in that people who don't perceive a need aren't going to RTFM, no matter how much time said RTFMing would save them in the long run, vs. sitting around complaining. IOW, once someone has become annoyed by the mere appearance of a necessity to deal with something that appears to be being foisted on them (whether it's setuptools or Python 3), the natural tendency is to minimize any actual work that would move in the direction of the thing they feel forced to deal with. For me, the closest thing to a killer feature in 3.x is argument type declarations, and it'd be a mild convenience at best. From a distance, many of the other changes appear like anti-features, if only because they're changing what I've been used to for twelve-plus years. (A few, like the removal of __metaclass__-in-locals support, are an active hindrance to porting.) So no, I haven't actually tried to port anything, nor have I done more than lightly skim the porting docs... looking for some reason why I'd *want* to move to Python 3. Heck, I have yet to use 2.6 to run any production code, and find some of *its* changes a bit annoying from a porting perspective. (E.g. dropping the sets module.) To make Py3 migration worthwhile to developers with heavy investment in the 2.x lines (and especially those supporting all the way back to 2.3 and 2.4), it'd have to have some *really* killer features. That is, be more like a Python 3000, and less like a Python 2.6 with a few bells and whistles, hampered by having to relearn some of the basic types and a soon-to-be-rebuilt standard library. Even if, in truth, the cost-benefit ratio right now *is* good for migrating to 3.x, nobody's doing a good job at promoting what those benefits are. (And it being easy to port to is NOT a benefit: nobody cares how easy it is to do something they don't see a reason to do in the first place.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cython as a Python implementation
Stefan, I think your attempts to see Cython accepted as one of the major Python implementations is misguided. It is not self-contained, it is an add-on tool for CPython (like its ancestor PyRex). I think Cython is incredibly useful (and I spoke to many very happy users yesterday at UCB) but trying to present it as a separate Python implementation is not useful. Please stop the discussion about this topic, they are distracting. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cython as a Python implementation
Maciej Fijalkowski, 05.11.2009 19:01: Most Python implementations do not reimplement the stdlib, or at most a minor part of it, so that's right out of the discussion. Did you actually check? Well, I know that Jython uses the original stdlib modules through svn:externals in the build, last thing I heard about IronPython was that they are allowed to use other people's code now, so I imagine they do the same thing, and I wouldn't expect PyPy to rewrite the existing Python code that exists in the stdlib. So the only remaining problem are the C extensions in CPython, and that's the minor part I mentioned above. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
Mike Krell wrote: Well, 3to2 would then be an option for you: use Python 3 as the source language. Making life easier for 3to2 is an *excellent* rationale for backports. I'm skeptical. If new features get added to 2.7: why would that simplify 3to2? It couldn't *use* these features, since it surely would have to support 2.6 and earlier as well. Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, perhaps, the nonlocal keyword). If it currently gives up, it then may offer you to restrict your target versions to 2.7+. Not sure whether users would use that option, though - perhaps they rather stop using nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are interested in. Perhaps 3to2 has a work-around that still provides a good backport in most cases. Then, the backport would not make the tool any simpler: if 3to2 would start using the backport, it would actually get more complicated (not easier), as it now needs to support two cases. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
2009/11/4 Martin v. Löwis mar...@v.loewis.de: Keep in mind also that the 2.x translation process is extremely slow and results in a clunky development process. There's no '2to3 --interactive' commandline that lets me type python 2 at a prompt and get python 3 results out so that I can try experiments on the 3.x interpreter; I have to actually put my experiments into my unit tests and wait 10 minutes to see if it works. It's like writing C++. That's not my experience. I see a change in source (say, on Django) available for 3.x within 5 seconds. True, but you need to set up a process that will convert only the changed files, and before Distribute came with 3.0 support, that was tedious. Now it's easy, if you want to use distribute. (Except that there is some bug I promised to look at this week, but haven't) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
Martin v. Löwis wrote: Mike Krell wrote: Well, 3to2 would then be an option for you: use Python 3 as the source language. Making life easier for 3to2 is an *excellent* rationale for backports. I'm skeptical. If new features get added to 2.7: why would that simplify 3to2? It couldn't *use* these features, since it surely would have to support 2.6 and earlier as well. Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, perhaps, the nonlocal keyword). If it currently gives up, it then may offer you to restrict your target versions to 2.7+. Not sure whether users would use that option, though - perhaps they rather stop using nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are interested in. I would have thought you could translate nonlocal with the following: Python 3: def scope(): name = value do_something_with(name) def inner(): nonlocal name name = new_value do_something_else(name) Python 2 def scope(): name = [value] do_something_with(name[0]) def inner(): name[0] = new_value do_something_else(name[0]) I would love to see nonlocal backported to 2.7 as it cleans up a simple pattern that I use relatively often for testing. Suppose you have an class and you want to test that method a calls method b, in Python 2 you might write something like this: def test_method_a_calls_method_b(): instance = SomeClass() was_called = [] def method_b(): was_called.append(True) instance.method_b = method_b instance.method_a() assert was_called == [True] in Python 3 you can replace this with the slightly nicer: def test_method_a_calls_method_b(): instance = SomeClass() was_called = False def method_b(): nonlocal was_called was_called = True instance.method_b = method_b instance.method_a() assert was_called As to the argument that releasing 2.7 is pointless as few people would use it for several years, the success of Python 2.6 shows that although *many* people don't / can't use new versions of Python for several years many other people are able to and do use new versions of Python. All the best, Michael Foord Perhaps 3to2 has a work-around that still provides a good backport in most cases. Then, the backport would not make the tool any simpler: if 3to2 would start using the backport, it would actually get more complicated (not easier), as it now needs to support two cases. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.ironpythoninaction.com/ ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
Martin v. Löwis wrote: Mike Krell wrote: Well, 3to2 would then be an option for you: use Python 3 as the source language. Making life easier for 3to2 is an *excellent* rationale for backports. I'm skeptical. If new features get added to 2.7: why would that simplify 3to2? It couldn't *use* these features, since it surely would have to support 2.6 and earlier as well. Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, perhaps, the nonlocal keyword). If it currently gives up, it then may offer you to restrict your target versions to 2.7+. Not sure whether users would use that option, though - perhaps they rather stop using nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are interested in. But surely someday 2.7 will be the oldest targetted 2.x version of Python for 3to2 and other tools (regardless of whether there's a 2.8). When that day comes, 3to2 can be made simpler, or can increase the amount of Python 3.x it can convert (or both) if we add 3.x features to 2.7. Of course, planning for a time so far in the future is difficult, and possibly pointless. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
2009/11/4 sstein...@gmail.com sstein...@gmail.com: Maybe the 3.x line should just be put out of our misery, merged back to 2.7, 2.8, 2.9, and proceed as Glyph suggested in passing with increasing levels of deprecation until it just turns into 3.x on its own by running out of numbers. Yeah, maybe. If people haven't moved over to Python 3 in 2015 I think we should start considering it. Let's discuss this again then. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)
2009/11/4 Nick Coghlan ncogh...@gmail.com: In writing it up, it occurred to me that having that kind of thing in a py3_compat compatibility module (to be used as, e.g., from py3_compat import str, bytes) would not only make it easier to use in multiple modules, but also easier for 2to3 to remove when forward porting. Well, when using 2to3 it already handles that stuff. But a module like that would be very handy if you want to support both 2.6 and 3.x without 2to3. With such a module it would be quite simple. In fact, I think the module should be called timemachine. ;-) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Py3k bytes type in 2.x (Re: nonlocal keyword in 2.x?)
2009/11/4 Nick Coghlan ncogh...@gmail.com: Lennart Regebro wrote: I also would really like to see a real port of the bytes class to 2.6, but I have a vague memory that there was some reason that wouldn't work. Not so much that it wouldn't work, but that the interfaces to support using it effectively really aren't there - lots of areas in the standard library needed to be tweaked to cope with bytes objects in 3.x. Ah, right, that was the problem, the standard library. I knew I heard a good reason against it. Generally speaking, the bytes = str trick represents a reasonable compromise as the APIs that you would pass a bytes object to in 3.x expect an 8-bit str instance in 2.x. Yeah, the problem being that bytes and str has quite different API's. Ah well. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Thu, Nov 5, 2009 at 09:34, Yuvgoog Greenle ubershme...@gmail.com wrote: On Thu, Nov 5, 2009 at 2:31 PM, Nick Coghlan ncogh...@gmail.com wrote: While it may be hard to tell without knowing who is and isn't a core developer Maybe there should be badges or something, hmmm, sounds like making an svn-commits-hall-of-fame for python could be a nice project. You might be interested in http://www.ohloh.net/p/python/contributors . -Brett --yuv ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
Dino Viehland, 05.11.2009 19:35: Stefan wrote: It /does/ make some static assumptions in that it considers builtins true builtins. However, it does not prevent you from replacing them in your code, as long as you do it inside the module. Certainly a restriction compared to Python, where you can import a module into a changed dict environment that redefines 'object', but not a major restriction IMO, and certainly not one that impacts much code. To me this is a deal breaker which prevents Cython from being a Python implementation. From a talk given by Colin Winter at the LLVM dev meeting (http://llvm.org/devmtg/2009-10/) it seems like Unladen Swallow wanted to do something like this as well and Guido said no. In this case the breaking change is so subtle that I'd personally hate to run into something like this porting code to Cython and having to figure out why it's not working. I assume that this is artificially exaggerated to make a point, as this behaviour is obviously not a technical requirement but an optimisation, which could potentially be disabled. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Wed, Nov 4, 2009 at 7:07 PM, Raymond Hettinger pyt...@rcn.com wrote: [Steven D'Aprano] Anyway, given the level of opposition to the suggestion, I'm no longer willing to carry the flag for it. If anyone else -- perhaps the OP -- feels they want to take it any further, be my guest. I feel pretty strongly that it's a wart in the language, and a sufficiently strong one that it should be remedied. I'm happy to champion it, but haven't the faintest idea what that entails. Summarizing my opposition to a new set method: 1) there already are at least two succinct ways to get the same effect 2) those ways work with any container, not just sets 3) set implementations in other languages show that this isn't needed. 4) there is value to keeping the API compact 5) isn't needed for optimization (selecting the same value in a loop makes no sense) 6) absence of real-world code examples that would be meaningfully improved I would be happy to add an example to the docs so that this thread can finally end. Adding an example to the docs does not solve the problem, which is if you come across the following code: for x in s: break ... it really looks like it does nothing. It's only because of the slightly idiosyncratic way Python handles variable scoping that it has an effect at all, and that effect isn't overtly related to what the code says, which is Iterate over all the elements in this set, then immediately stop after the first one. s.get() or s.pick() are both more succinct and more clear, saying Get me an arbitrary element from this set. To make matters worse, for x in s: break fails silently when s is empty, and x = iter(s).next() raises a StopIteration exception. Neither is clear. The obvious way, for newcomers, of achieving the effect is: x = s.pop() s.add(x) ... and that's simply horrible in terms of efficiency. So the obvious way of doing it in Python is wrong(TM), and the correct way of doing it is obscure and raises misleading exceptions. I suppose, mulling things over, the method should be called .pick(), which avoids any confusion with .get(). And, as I've stated, I think it should return a member of the set, with no guarantees what member of the set is returned. It could be the same one every time, or a random one, or the last one placed in the set. For cases where people want to cycle through the members of the set in a predictable order, they can either copy the contents into a list (sorted or unsorted) *or* subclass set and override the .pick() method to place stronger guarantees on the API. So, summarizing my responses: 1) the two succinct ways are unclear and not immediately obvious 2) the existing methods aren't needed for other objects 3) set implementations in other languages are irrelevant 4) this is a small, targeted change which not make the API disordered or unruly 5) could very well be needed for optimization, in cases where constructing an iterator is expensive 6) there have been several real-world examples posted which would be improved by this change -- Chris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser ch...@subtlety.com wrote: .. and x = iter(s).next() raises a StopIteration exception. And that's why the documented recipe should probably recommend next(iter(s), default) instead. Especially because iter(s).next() is not even valid code in 3.0. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
I feel pretty strongly that it's a wart in the language, and a sufficiently strong one that it should be remedied. I'm happy to champion it, but haven't the faintest idea what that entails. There are two ways a) write a library that provides what you want, publish it on PyPI, and report back in a few years of how many users your library has, what they use it for, and why it should become builtin b) write a PEP, wait a few years for the language moratorium to be lifted, provide an implementation, and put the PEP for pronouncement. Careful reading of the Moratorium PEP may allow shortening of the wait. In any case, it seems that this specific change will see some opposition. So you will need to convince the opposition, one way or the other. The obvious way, for newcomers, of achieving the effect is: x = s.pop() s.add(x) ... and that's simply horrible in terms of efficiency. So the obvious way of doing it in Python is wrong(TM), and the correct way of doing it is obscure and raises misleading exceptions. If you chose to write a PEP, include a proof that this approach is indeed horrible in terms of efficiency; I question that. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
Michael Foord fuzzyman at voidspace.org.uk writes: I would love to see nonlocal backported to 2.7 as it cleans up a simple pattern that I use relatively often for testing. Well you know I'm sure that if someone proposes a proper patch it will eventually get accepted ;) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] No buildbot to test wide unicode?
I would certainly agree to setup two slaves on mine. There are ample resources available. I could do so as well. Gentoo is one of the distributions that uses the wide build by default, so that would make it a good test candidate as well. I have now setup two slaves, murray-gentoo-wide and pitrou-ubuntu-wide, same passwords. They configure with the option --with-wide-unicode (if I did that correctly). I think this means that the 2.x branches will have to grow this option also (although configure should ignore the option, so it should still build). Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
Stefan wrote: I assume that this is artificially exaggerated to make a point, as this behaviour is obviously not a technical requirement but an optimisation, which could potentially be disabled. If there's a way to disable this then that's fine and IMO when it was disabled you'd still be Python. But changing behavior in the name of optimization is no longer just an optimization. For what it's worth in IronPython we're going to look at making this faster in the future as well. We already cache built-in values locally in a module and invalidate the cache when the builtins changes. In the future I'd like to even combine loading a global and invoking it into a single call site for even better optimizations. And you could even imagine an extreme scenario where when built-ins changed all the code in the system that depended upon them is re-compiled punishing users who actually do this but I doubt we'll go that far. But the important thing is that the optimizations don't change the language behavior otherwise I don't think you're still Python. But of course that's what makes it so challenging and fun to try and optimize Python :) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum gu...@python.org wrote: I've checked draft (!) PEP 3003, Python Language Moratorium, into SVN. As authors I've listed Jesse, Brett and myself. I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
[GvR] I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. But it would have been so much fun to have a never ending python-ideas thread on BEGIN/END blocks ;-) C'est la vie, Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
2009-11-03 18:35:10 Guido van Rossum napisał(a): I've checked draft (!) PEP 3003, Python Language Moratorium, into SVN. As authors I've listed Jesse, Brett and myself. On python-ideas the moratorium idea got fairly positive responses (more positive than I'd expected, in fact) but I'm bracing myself for fierce discussion here on python-dev. It's important to me that if if this is accepted it is a rough consensus decision (working code we already have plenty of :-), not something enforced by a vocal minority or an influential individual such as myself. If there's too much opposition I'll withdraw the PEP so as not to waste everybody's time with a fruitless discussion. The PEP tries to spell out some gray areas but I'm sure there will be others; that's life. Do note that the PEP proposes to be *retroactive* back to the 3.1 release, i.e. the frozen version of the language is the state in which it was released as 3.1. Does moratorium allow to add support for e.g. 'from __future__ import yield_from' in Python 3.2? -- Arfrever Frehtes Taifersar Arahesis ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 14:20, Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com wrote: 2009-11-03 18:35:10 Guido van Rossum napisał(a): I've checked draft (!) PEP 3003, Python Language Moratorium, into SVN. As authors I've listed Jesse, Brett and myself. On python-ideas the moratorium idea got fairly positive responses (more positive than I'd expected, in fact) but I'm bracing myself for fierce discussion here on python-dev. It's important to me that if if this is accepted it is a rough consensus decision (working code we already have plenty of :-), not something enforced by a vocal minority or an influential individual such as myself. If there's too much opposition I'll withdraw the PEP so as not to waste everybody's time with a fruitless discussion. The PEP tries to spell out some gray areas but I'm sure there will be others; that's life. Do note that the PEP proposes to be *retroactive* back to the 3.1 release, i.e. the frozen version of the language is the state in which it was released as 3.1. Does moratorium allow to add support for e.g. 'from __future__ import yield_from' in Python 3.2? No. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Thu, Nov 5, 2009 at 3:21 PM, Martin v. Löwis mar...@v.loewis.de wrote: There are two ways a) write a library that provides what you want, publish it on PyPI, and report back in a few years of how many users your library has, what they use it for, and why it should become builtin This clearly isn't called for in this case. We're talking about a single function on a collection. In this case, importing an alternative set API (and maintaining the dependency) is more work than just writing your own workaround. The purpose of adding a method is to prevent the need of everyone writing their own workaround. b) write a PEP, wait a few years for the language moratorium to be lifted, provide an implementation, and put the PEP for pronouncement. Careful reading of the Moratorium PEP may allow shortening of the wait. Clearly, I'll need to write up the PEP. In any case, it seems that this specific change will see some opposition. So you will need to convince the opposition, one way or the other. I doubt some of the people on either side are going to be convinced. I'd settle for convincing most of the fence-sitters, along with a few of the loyal opposition. -- Chris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
[Chris Bergstresser] Clearly, I'll need to write up the PEP. Why not write a short, fast get_first() function for your utils directory and be done with it? That could work with sets, mappings, generators, and other containers and iterators. No need to fatten the set/frozenset API for something so trivial and so rarely needed. Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé ziade.ta...@gmail.com wrote: PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme. Perphaps.. But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code... Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010. But there's no plan to include a new *distribution* format in Distutils. I wasn't suggesting that - at all. And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years. No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic. Be fair... I'm saying finish what is already there.. or take out the crap .. It isn't fair to suggest that I am somehow asking for some big change when I am simply pointing out all the junk that's in there that's already half built. In any case those PEPs are not finished yet, so everyone can help at distutils-SIG True - and False. But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet. Seriously, if you won't allocate work out to people then how can it get done? Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway? I'm a fairly proficient develper, but I can't get assigned a single work item.. And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items... Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010.. David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 23:05, Guido van Rossum gu...@python.org wrote: I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? Cheers, Dirkjan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
On 2009-11-05 16:29 PM, David Lyon wrote: But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet. Seriously, if you won't allocate work out to people then how can it get done? With rare exceptions, open source projects don't really assign work. If you want to help, help. Don't wait for someone to tell you exactly what to do. No one will. Go to the tracker, find something interesting, and do it. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
[me]
Why not write a short, fast get_first() function for your utils directory and
be done with it?
That could work with sets, mappings, generators, and other containers and
iterators.
No need to fatten the set/frozenset API for something so trivial and so rarely
needed.
Forgot to post the code. It is short, fast, and easy. It is explicit about handing the case with an empty input. And it is
specific about which value it returns (always the first iterated value; not an arbitrary one). There's no guessing about what it
does. It gets the job done.
def first(iterable):
'Return the first value from a container or iterable. If empty, raises
LookupError'
for value in iterable:
return value
raise LookupError('no first value; iterable is empty')
If desired, it is not hard to change to the last time to return a default value
or some other exception.
Raymond
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
On Thu, Nov 5, 2009 at 11:29 PM, David Lyon david.l...@preisshare.net wrote: [...] But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code... Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010. The work that was done during the last year was brainstorming on PEPs. You can't count that work in SLOCs, but in trying to find a consensus on packaging problems. And I can fairly say that we are very close to something that can get accepted by the community, then turned into code in Distutils/pkgutil. But there's no plan to include a new *distribution* format in Distutils. I wasn't suggesting that - at all. And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years. They are new to Distutils. They are not new to the Python packaging eco-system of course, and PEP 376 rely on many ideas created in Setuptools. No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic. The consistency will come through the standard we are building in PEP 376. [..] But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet. Seriously, if you won't allocate work out to people then how can it get done? I am sorry that you feel that way. We don't allocate coding work to people in this process. That's not the way it works. Rather, people help in building the PEPs by providing their own feedback/experience. At the end, we are trying to have PEPs that adresses the wider range of cases. Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway? I'm a fairly proficient develper, but I can't get assigned a single work item.. We have already prototypes for each PEP so people can try them out, enhance them while the PEPs are being built. If you want to help in their coding, you are more than welcome. They are on a DVCS at bitbucket. And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items... Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010.. Again, the big challenge is not on the coding part. If it was, Distutils would have them already. The challenge is on the PEPs, and on making sure we collect all PoVs and feedbacks, before we change Distutils in Python 2.7/3.2 Regards Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky alexander.belopol...@gmail.com wrote: On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser ch...@subtlety.com wrote: .. and x = iter(s).next() raises a StopIteration exception. And that's why the documented recipe should probably recommend next(iter(s), default) instead. Especially because iter(s).next() is not even valid code in 3.0. This seems reasonably legible to you? Strikes me as coding by incantation. Also, while I've heard people say that the naive approach is slower, I'm not getting that result. Here's my test: smrt = timeit.Timer(next(iter(s)), s=set(range(100))) smrt.repeat(10) [1.2845709323883057, 0.60247397422790527, 0.59621405601501465, 0.59133195877075195, 0.58387589454650879, 0.56839084625244141, 0.56839680671691895, 0.56877803802490234, 0.56905913352966309, 0.56846404075622559] naive = timeit.Timer(x=s.pop();s.add(x), s=set(range(100))) naive.repeat(10) [0.93139314651489258, 0.53566789627075195, 0.53674602508544922, 0.53608798980712891, 0.53634309768676758, 0.53557991981506348, 0.53578495979309082, 0.53666114807128906, 0.53576493263244629, 0.53491711616516113] Perhaps my test is flawed in some way? Geremy Condra ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Nov 5, 2009, at 6:04 PM, geremy condra wrote: Perhaps my test is flawed in some way? Yes: you're testing the speed of something that makes absolutely no sense to do in a tight loop, so *who the heck cares how fast any way of doing it is*! Is this thread over yet? James ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
David, you have an attitude problem. Your contributions (the post below is just an example) don't sound healthy to me -- you just complain and whine and denigrate Tarek's work. In a previous post you claimed to have had a particular idea first (it doesn't matter which idea) and you managed to make it sound bad that your idea was eventually accepted. This is not a productive attitude. Surely the problem isn't writing 300 lines of code. The problem is getting everyone to agree on which 300 lines of code should be written. That is the problem at hand, and claiming that nothing happened because no code was written and all that was agreed on amounts to 300 lines of code is outright demeaning. Stop it. You are wearing out your welcome. --Guido PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG. On Thu, Nov 5, 2009 at 2:29 PM, David Lyon david.l...@preisshare.net wrote: On Thu, 5 Nov 2009 11:35:41 +0100, Tarek Ziadé ziade.ta...@gmail.com wrote: PEP 376 is working on a default, unified, *installation* format, that tries to gather the good ideas of Pip, Setuptools etc.. and propose a unified format for our site-packages. This new standard will come with APIs in pkgutil to be able to query installed distribution etc. This work is also linked to PEP 345 work where we are modifying the Metadata, and to PEP 386 that proposes a standard version comparison scheme. Perphaps.. But if you put all these PEPs together, implementing all the new features can't come to more than 300 lines of code... Since we hardly got anywhere on them in 2009, it will be interesting to see how much of it gets done in 2010. But there's no plan to include a new *distribution* format in Distutils. I wasn't suggesting that - at all. And saying that 'eggs' are a *new* python package format isn't really really helpful because to my understanding they've been around for some number of years. No, i won't raise why we have EGG_INFO directories and a whole lot of half working egg stuff in standard python... I'm just asking why it can't be more consistant? while we're on the refactoring topic. Be fair... I'm saying finish what is already there.. or take out the crap .. It isn't fair to suggest that I am somehow asking for some big change when I am simply pointing out all the junk that's in there that's already half built. In any case those PEPs are not finished yet, so everyone can help at distutils-SIG True - and False. But I've been on the list for some twelve months asking for work to help out with, and haven't been assigned a single task to do yet. Seriously, if you won't allocate work out to people then how can it get done? Whilst I personally think a lot of the stuff in those PEPs is not high on quality, why don't we just get them implemented anyway? I'm a fairly proficient develper, but I can't get assigned a single work item.. And to me, it doesn't seem any harder than just selecting 'djlyon' on the python tracker for some work items... Surely those PEPs all amount to 300+ lines of code. With two people working on it, that's only surely 150+ lines of code each... That shouldn't be such a big challenge for 2010.. David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Thu, Nov 5, 2009 at 23:05, Guido van Rossum gu...@python.org wrote: I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? Cheers, Dirkjan I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal. So, -1 from me ;) jesse ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 14:53, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Thu, Nov 5, 2009 at 23:05, Guido van Rossum gu...@python.org wrote: I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? So you are basically asking for a moratorium on stdlib deprecations. Considering how much was removed in Python 3 from 2.6 this is a really minor worry. And the only deprecation that I can see potentially coming down the pipeline is optparse for argparse and cProfile to help finish PEP 3108. And, as Jesse said in another reply, it would be nice to take this time to reflect upon where we want the stdlib to go and to potentially clear other things out. So I am -0 on this as leaving stuff in for an extra release isn't going to kill us, but I would prefer to not put it off. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller jnol...@gmail.com wrote: On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Thu, Nov 5, 2009 at 23:05, Guido van Rossum gu...@python.org wrote: I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? Cheers, Dirkjan I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal. So, -1 from me ;) jesse Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth. For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings. I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 6:26 PM, Guido van Rossum gu...@python.org wrote: I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal. So, -1 from me ;) jesse Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth. For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings. I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. I'm interested in hearing how to handle pending removals other than deprecation warnings - I guess I'm against the idea that we shouldn't remove/plan to remove things from the stdlib and signal those intentions to users during the moratorium. The mechanics of that can be something other than deprecation warnings, we can add a line to the current moratorium that puts the nix on deprecation warnings (because yes, Dirkjan is right - they can be a pain) but we should outline the alternative process. jesse ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 15:26, Guido van Rossum gu...@python.org wrote: On Thu, Nov 5, 2009 at 3:21 PM, Jesse Noller jnol...@gmail.com wrote: On Thu, Nov 5, 2009 at 5:53 PM, Dirkjan Ochtman dirk...@ochtman.nl wrote: On Thu, Nov 5, 2009 at 23:05, Guido van Rossum gu...@python.org wrote: I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. Let me state first, I think the PEP is great, and I have no objection to its current form. I do have one qualm, where I wonder if the PEP shouldn't be a little stricter. As a gentoo developer and Mercurial maintainer, most of the pain in the recent migration towards 2.6 has not been in language changes, but in the standard library. Unfortunately, it's exempt from the moratorium in the PEP. Which makes me wonder, why are we not adding another moratorium, on deprecations in the standard library? In other words, let's not deprecate things like md5 or sha or the popen family of functions, but keep all of that as it is, for both 2.x and 3.x, so people can direct their energy towards other things (hopefully porting their 2.x codebase to 3.x). The standard library has already been through a lot of evolution in the 2.x to 3.x transition, so one might assume there's not a lot of stuff in the 3.x stdlib that would need deprecation in the short term. And for 2.x, well, I'd certainly hope we don't need to deprecate much more there before it finally gets EOL'ed, so it should be a relatively light maintenance load to bear. Is this just crazy talk? Cheers, Dirkjan I'm against restricting deprecation warnings within the stdlib as part of this. I actually want more things cleaned up and possibly deprecated. That being said, a deprecation warning just means we will remove it One Day - anything being deprecated will need a PEP and follow the long path to actual removal. So, -1 from me ;) jesse Actually, I think Dirkjan has a point. I'm not sure that we need another moratorium (that's a rather dramatic kind of decision which should be very rare indeed) but I do agree that deprecations are often more of a pain than they're worth. For example, take the deprecation of the md5 and sha modules in Python 2.6. They make it a bit of a pain to write code that *cleanly* supports Python 2.4 (doesn't have hashlib) through 2.6 (warns when importing md5 instead of hashlib). You can silence the warning, but that is in itself not particularly clean, and users really hate having the warnings. I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. What exactly are those better ways? Document as deprecated only? -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Fri, Nov 6, 2009 at 1:17 AM, James Y Knight f...@fuhm.net wrote: Is this thread over yet? Sorry, I just had to point out that pop/add has a side effect that would be apparent on a set that multiple threads access - it loses an item and then gets it back. Sounds like a sleeper race condition that's going to be rare but extremely hard to find if it does occur. Crooked as a gil. --yuv ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
What exactly are those better ways? Document as deprecated only? -Brett A switch to ENABLE those warnings? Lord knows I'm sick of filtering them out of logs. A switch to enable deprecation warnings would give developers a chance to see them when migrating to a new version of python. And it would prevent users from having to deal with them. -- Bobby R. Ward -- bobbyrw...@gmail.com http://github.com/bobbyrward http://launchpad.net/~bobbyrward While many languages can be used to encrypt data, PERL has something built-in that gives you encryption. Perl calls it `syntax`. -- http://uncyclopedia.wikia.com/wiki/Perl ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
Tarek, Guido, Forgive my grumpy tone.. Looking positive, if we now have a rough consensus that the PEPS might represent some 300+ lines of code... then good - lets get started, that's all that I meant. I'm glad above all, that you noticed that part foremost. If it's a simple case that alternative's to the solutions cannot be considered - because I am grumpy - whatever, then that's something I can live with. I didn't know that proposing a different solution wasn't a good idea. PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG. Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal. The proposal is in the final stages of being wrapped up. Here are the discussion References that have occurred over the year: http://mail.python.org/pipermail/distutils-sig/2009-November/thread.html November 2009 Archives by thread * Messages sorted by: [ subject ] [ author ] [ date ] * More info on this list... Starting: Sun Nov 1 09:31:46 CET 2009 Ending: Fri Nov 6 00:09:22 CET 2009 Messages: 65 * [Distutils] Alternate static metadata PEP submission... Georg Brandl o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe + [Distutils] Alternate static metadata PEP submission... Georg Brandl + [Distutils] Alternate static metadata PEP submission... David Lyon # [Distutils] Alternate static metadata PEP submission... Georg Brandl # [Distutils] Alternate static metadata PEP submission... David Lyon http://mail.python.org/pipermail/distutils-sig/2009-October/thread.html # [Distutils] Alternate static metadata PEP submission... David Lyon * [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Eric Smith o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon * [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau * [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Floris Bruynooghe o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Tarek Ziadé o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Ian Bicking o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... Kevin Teague o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... Fred Drake o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP submission... David Lyon o [Distutils] Alternate static metadata PEP submission... David Cournapeau o [Distutils] Alternate static metadata PEP
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Fri, Nov 6, 2009 at 1:55 AM, Bobby R. Ward bobbyrw...@gmail.com wrote: A switch to ENABLE those warnings? I think a few people I know would still be raising strings like exceptions if not for the deprecation warnings. Most people won't turn on the switch with the extra warnings. --yuv ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On 5 Nov, 11:55 pm, bobbyrw...@gmail.com wrote: What exactly are those better ways? Document as deprecated only? -Brett A switch to ENABLE those warnings? Lord knows I'm sick of filtering them out of logs. A switch to enable deprecation warnings would give developers a chance to see them when migrating to a new version of python. And it would prevent users from having to deal with them. PendingDeprecationWarning is silent by default and requires a switch to be enabled. Jean-Paul ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Thu, Nov 5, 2009 at 6:17 PM, James Y Knight f...@fuhm.net wrote: On Nov 5, 2009, at 6:04 PM, geremy condra wrote: Perhaps my test is flawed in some way? Yes: you're testing the speed of something that makes absolutely no sense to do in a tight loop, so *who the heck cares how fast any way of doing it is*! Is this thread over yet? James I'm testing the speed because the claim was made that the pop/add approach was inefficient. Here's the full quote: The obvious way, for newcomers, of achieving the effect is: x = s.pop() s.add(x) ... and that's simply horrible in terms of efficiency. So the obvious way of doing it in Python is wrong(TM), and the correct way of doing it is obscure and raises misleading exceptions. Since I use this in a graph theory library that I am currently trying to scale to handle 300 million node simulations, and this is used in several relevant algorithms, I was concerned. Geremy Condra ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
Guido van Rossum wrote: On Tue, Nov 3, 2009 at 9:35 AM, Guido van Rossum gu...@python.org wrote: I've checked draft (!) PEP 3003, Python Language Moratorium, into SVN. As authors I've listed Jesse, Brett and myself. I haven't seen substantial opposition against the PEP -- in fact I can't recall any, and many people have explicitly posted in support of it. So unless opposition suddenly appears in the next few days, I'll move it to the Accepted state next Monday. My view is that instead of there being new language features appearing in the next few years, what we have in Python 3 _are_ the new language features. We just need time to digest them before looking for more. :-) +1 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
2009/11/6 David Lyon david.l...@preisshare.net: [..] Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal. The proposal is in the final stages of being wrapped up. There are three different topics : 1. a standard for the installation format: PEP 376 2. changes in the metadata format : PEP 345 (-- work in progress here, some parts are still missing) 3. ways to built the metadata that are shipped into a distribution statically, using Distutils : PEP 390 was started for that And as a matter of fact, PEP 390 is not so important, because at the end, the changes that are made in PEP 345 are the most important ones. They allow describing metadata fields that can vary depending on the target platform. PEP 390 is just the simplest way to make it possible to describe static metadata using a setup.cfg ConfigParser section, rather than some code in setup.py. And when we built it, it helped us understand what we really wanted on PEP 345 side (with a big insights from people like Marc-André on this) But PEP 390 limits its scope to the Metadata fields and now waits for PEP 345 changes. Your proposal is a partial alternative to PEP 390, and is a quite interesting work to dig in, and the SIG is discussing it, but a very important point about it is that it does a lot more. It's a new *building* system on its own, and does not limit itself to describing metadata fields. That's fine, and that's quite interesting, but I doubt we will be able to push it in Distutils itself any sooner, because it a whole different system ala scons (which is a great tool don't get me wrong). I think (as I told you before IIRC) that you need to create a project to demonstrate it and I think it can be great to continue getting some help from Distutils-SIG for this. Furthermore, building such a tool on the top of Distutils can improve Distutils itself, because it would be yet another project that would provide valuable feedback on the APIs and standards we provide. That's for example what we are getting from projects like PyPM or Enthought Enstaller. As a matter of fact, there's already great feedback from Active State on PEP 376 because they want to be early adopters of the upcoming standard and make sure we do the right choices in there. Enthought also started to use the prototype we have written for PEP 386 (version comparisons) IIRC. But in the existing PEPs discussions, we need to focus on their respective scopes. And they are all targeted to improve the existing stdlib tool : Distutils. (and a wanted side effect is to make Distutils smaller if we can, and a robust basis for all third party tools) That's why you can't just drop a counter-PEP to any existing PEP we have started for Distutils. Imho help us build those PEPs, and / or create your own build system, wether its based or not on Distutils. Regards Tarek ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
On Thu, Nov 5, 2009 at 15:55, David Lyon david.l...@preisshare.net wrote: Tarek, Guido, Forgive my grumpy tone.. Looking positive, if we now have a rough consensus that the PEPS might represent some 300+ lines of code... then good - lets get started, that's all that I meant. I'm glad above all, that you noticed that part foremost. If it's a simple case that alternative's to the solutions cannot be considered - because I am grumpy - whatever, then that's something I can live with. I didn't know that proposing a different solution wasn't a good idea. That is passive-aggressive and uncalled for. Proposing an opposing view is fine when done respectfully which you have not done here on python-dev. PS. Submitting a counter-PEP to the peps editors that hasn't been discussed on the SIG list at all is also a bad move. You really need to change the way you try to interact with the SIG. Here are the references to the discussions from distutils-sig. In these references you'll see that my interaction with the list was not grumpy but was entirely positive. I feel that I got a fair hearing on distutils list and I had a number of supporters to my proposal. That's fine, but it's Tarek's call and as a group we reached the conclusion that Tarek is doing a fine job and is being more than fair. It has been stated as such and yet you keep pushing us like we have not heard you. We did hear you the first time, but you just seem to not get the message. Adding to the fact that your tone has been an issue and that we don't want to listen to it anymore has pushed Guido to the brink and me over the edge. I honestly don't know how Tarek keeps his patience over on the distutils SIG if he has to deal with kind of crap constantly. Personally, I am now sending your emails to the trash since you can't seem to even reply to Guido's email w/o some nasty undertone. Hopefully someday you can learn to communicate in a better, friendlier manner. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Refactoring installation schemes
Tarek, Your proposal is a partial alternative to PEP 390, and is a quite interesting work to dig in, and the SIG is discussing it, but a very important point about it is that it does a lot more. It's a new *building* system on its own, and does not limit itself to describing metadata fields. Actually no, the proposal is only for a metadata installation of packages that addresses the security problem of requiring users to run untrusted and unverified setup.py's. That was where this discussion originated and that is what I am addressing. PEP-390 doesn't go there at all... My proposal uses the static metadata contained within existing package formats (PKG_INFO + sources.txt) and therefore doesn't require any changes to the build system. There isn't a new build system. I never proposed that. However PEP-390 does imply changing the build system. That's why you can't just drop a counter-PEP to any existing PEP we have started for Distutils. Imho help us build those PEPs, and / or create your own build system, wether its based or not on Distutils. Once again, I can't see why all the fuss. All I did was have a discussion on distutils-ml about a different way of specifying package dependencies that might work across different python platforms. It was suggested to me that I do a PEP.. So I took the documentation on face value that developers could submit a PEP-proposal. Imho help us build those PEPs, This could best apply to PEP-345: (PKG_INFO) Requires: sys Requires(python = 2.4): lxml Requires(windows): win32com Requires(linux): pyodbc Requires(linux ubuntu gnome python = 3.4): gix Requires(windows xp python 2.2): win32prn Thank's for reading my alternate-metadata installation proposal and I accept your feedback. What I can do is clarify that my proposal is not for a build system but for a metadata installation scheme based on a static setup.py, existing metadata and use of python -m setup install for invocation. I'm just trying to make this stuff no more complicated than it needs to be... David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger pyt...@rcn.com wrote: Forgot to post the code. It is short, fast, and easy. It is explicit about handing the case with an empty input. And it is specific about which value it returns (always the first iterated value; not an arbitrary one). There's no guessing about what it does. It gets the job done. I'm trying to take this suggestion in the best possible light, which is that you honestly think I didn't read past Chapter 3 of the Python Tutorial, and I am therefore in fact unfamiliar with function definitions. -- Chris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Thu, Nov 5, 2009 at 6:30 PM, geremy condra debat...@gmail.com wrote: I'm testing the speed because the claim was made that the pop/add approach was inefficient. Here's the full quote: The obvious way, for newcomers, of achieving the effect is: x = s.pop() s.add(x) ... and that's simply horrible in terms of efficiency. So the obvious way of doing it in Python is wrong(TM), and the correct way of doing it is obscure and raises misleading exceptions. I was talking mainly from a theoretical standpoint, and because the library I'm working on is designed to work seamlessly over the network. In those cases, where the set the user is working with is actually a proxy object across the wire, the time to acquire the locks, remove the object, release the locks, reacquire the locks, add the object, then rerelease the locks is *significantly* more expensive than just noting the set hasn't changed and returning a cached object from it. -- Chris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon br...@python.org wrote: On Thu, Nov 5, 2009 at 15:26, Guido van Rossum gu...@python.org wrote: I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. What exactly are those better ways? Document as deprecated only? Sorry, I have an existence proof, but no construction. :-) Ideas welcome. Silent deprecations, loud documentation, biweekly home visits, whatever, as long as it doesn't log a message by default. The thing is, in practice, people will be testing with a new release anyway, and that's the earliest time they are likely to take action (other than silencing the warnings). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Status of the Buildbot fleet and related bugs
The buildbot waterfall is much greener now. Thanks to all who have contributed to making it so (and it hasn't just been Mark and Antoine and I, though we've been the most directly active (and yes, Mark, you did contribute several fixes!)). The 'stable builders' fleet is green now except for: (1) issue 7269: occasional 2.6/trunk bsddb3 failures on windows (2) issue 6748: random 3.1/3.x failures on most buidbots. (3) the klose-debian-ppc builder being offline Of these, (2) is by _far_ the biggest issue, and the one that causes the most flap (success-failure-success). And flap is the thing that most harms the buildbot usefulness. Anyone who wants to debug this on a platform where it is consistently reproducible please email me your public key and I'll give you a shell account on my buildbot (Antoine already has one). In the 'unstable' builder fleet, Antoine's new builder seems to be stable across the board, while mine fails consistently on 3.1 and 3.x because of the test_telnetlib bug. Thomas Heller's OS X buildbot is consistently having lots of test failures (the same ones each time I've checked). The master claims the klose-debian-alpha buildbot doesn't know about branches, which is odd since it was working not too long ago. The remaining buildslaves appear to have been offline for some time. Open issues here are: (1) issue 3864: FreeBSD testing hangs consistently. According to the ticket this is a FreeBSD bug fixed in 6.4, so an OS upgrade on the buildslave would probably solve it. (2) issue 4970: consistent signal 32 error on the norwitz-x86 Gentoo buildslave in 3.1 and 3.x. This may be due to the box running an old threading library, but it does make one wonder what changed in 3.x that exposed it. Another issue that I've seen on the buildbots but that doesn't seem to be showing up right now (is it fixed?) is issue 7251, which Mark is working on. So, overall I think the buildbot fleet is in good shape, and if we can nail issue 6748 I think it will be back to being an important resource for sanity checking our checkins. By the way, Georg set up the IRC interface on the #python-dev channel, so you can hang out there if you want to get realtime reports of which buildbots have going from success to failure and vice versa. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7 Release? 2.7 == last of the 2.x line?
On Thu, Nov 5, 2009 at 1:08 PM, Martin v. Löwis mar...@v.loewis.dewrote: Mike Krell wrote: Well, 3to2 would then be an option for you: use Python 3 as the source language. Making life easier for 3to2 is an *excellent* rationale for backports. Clarifying a bit of potentially misleading editing: I wrote neither of the statements quoted above. M v L wrote the first and Nick C. wrote the second. I'm skeptical. If new features get added to 2.7: why would that simplify 3to2? It couldn't *use* these features, since it surely would have to support 2.6 and earlier as well. Not sure what 3to2 would do about difficult-to-convert 3.x feature (as, perhaps, the nonlocal keyword). If it currently gives up, it then may offer you to restrict your target versions to 2.7+. Not sure whether users would use that option, though - perhaps they rather stop using nonlocal in 3.x if 3to2 cannot support it for all 2.x versions they are interested in. Perhaps 3to2 has a work-around that still provides a good backport in most cases. Then, the backport would not make the tool any simpler: if 3to2 would start using the backport, it would actually get more complicated (not easier), as it now needs to support two cases. I basically agree with you here, except perhaps for the likely definition of versions they are interested in. You have suggested on several occasions that a 2.7 (or 2.8) containing new syntax such as nonlocal would be of limited value because the vast majority of developers interested in supporting 2.x would have to support 2.6 as well. I respectfully suggest that may not necessarily be the case. I suspect that there are lots of small fish out there like me who have the luxury of being able to hop onto whatever version of the language suits them the best. Not to mention all of the new users who will be drawn to Python over the next several years while the 3.x standard library and third party library situation becomes more stable and comprehensive. Why not make the 2.x feature set the best it can be given the likelihood that 2.x will be the most compelling alternative to many users until the 3.x libraries mature? Of course, it's easy for me to ask other people to the hard work. It might be fun to take a crack at implementing nonlocal myself, but I know next to nothing about the implementation of CPython. By the time I pestered y'all with enough questions to get up to speed, you'd probably wish you'd just implemented it yourself -- less work :-) Mike ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
geremy condra wrote: On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky alexander.belopol...@gmail.com wrote: On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser ch...@subtlety.com wrote: .. and x = iter(s).next() raises a StopIteration exception. And that's why the documented recipe should probably recommend next(iter(s), default) instead. Especially because iter(s).next() is not even valid code in 3.0. This seems reasonably legible to you? Strikes me as coding by incantation. Also, while I've heard people say that the naive approach is slower, I'm not getting that result. Here's my test: smrt = timeit.Timer(next(iter(s)), s=set(range(100))) smrt.repeat(10) [1.2845709323883057, 0.60247397422790527, 0.59621405601501465, 0.59133195877075195, 0.58387589454650879, 0.56839084625244141, 0.56839680671691895, 0.56877803802490234, 0.56905913352966309, 0.56846404075622559] naive = timeit.Timer(x=s.pop();s.add(x), s=set(range(100))) naive.repeat(10) [0.93139314651489258, 0.53566789627075195, 0.53674602508544922, 0.53608798980712891, 0.53634309768676758, 0.53557991981506348, 0.53578495979309082, 0.53666114807128906, 0.53576493263244629, 0.53491711616516113] Perhaps my test is flawed in some way? Geremy Condra Well, it also uses a fairly trivial set. 'set(range(100))' is going to generally have 0 collisions and everything will hash to a unique bucket. As such, pop ing an item out of the hash is a single val = table[int mask]; table[int mask] = _dummy, and then looking it up again requires 2 table lookups (one finds the _dummy, the next finds that the chain is broken and can rewrite the _dummy.) However, if a set is more full, or has more collisions, or ... then pop() and add() become relatively more expensive. Surprising to me, is that next(iter(s)) was actually slower than .pop() + .add() for 100 node set in my testing: $ alias TIMEIT=python -m timeit -s 's = set(range(100)' $ TIMEIT x = next(iter(s)) 100 loops, best of 3: 0.263 usec per loop $ TIMEIT x = s.pop(); s.add(x) 100 loops, best of 3: 0.217 usec per loop though both are much slower than the fastest we've found: $ TIMEIT for x in s: break 1000 loops, best of 3: 0.0943 usec per loop now, what about a set with *lots* of collisions. Create 100 keys that all hash to the same bucket: aliase TIMEIT=python -m timeit -s 's = set([x*1024*1024 for x in range(100)])' $ TIMEIT x = next(iter(s)) 100 loops, best of 3: 0.257 usec per loop $ TIMEIT x = s.pop(); s.add(x) 100 loops, best of 3: 0.218 usec per loop I tried a few different ways, and I got the same results, until I did: $ python -m timeit -s s = set(range(10, 1000100)) next(iter(s)) 1000 loops, best of 3: 255 usec per loop Now something seems terribly wrong here. next(iter(s)) suddenly jumps up from being 0.3 us, to being more than 200us. Or ~1000x slower. I realize the above has 900k keys, which is big. But 'next(iter(s))' should only touch 1, and, in fact, should always return the *first* entry. My best guess is just that the *first* entry in the internal set table is no longer close to offset 0, which means that 'next(iter(s))' has to evaluate a bunch of table slots before it finds a non-empty entry. Anyway, none of the proposals will really ever be faster than: for x in s: break It is a bit ugly of a construct, but you don't have an attribute lookup, etc. As long as you don't do: for x in s: pass Then it stays nice and fast. John =:- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Thu, Nov 5, 2009 at 11:21 PM, John Arbash Meinel john.arbash.mei...@gmail.com wrote: geremy condra wrote: On Thu, Nov 5, 2009 at 4:09 PM, Alexander Belopolsky alexander.belopol...@gmail.com wrote: On Thu, Nov 5, 2009 at 3:43 PM, Chris Bergstresser ch...@subtlety.com wrote: .. and x = iter(s).next() raises a StopIteration exception. And that's why the documented recipe should probably recommend next(iter(s), default) instead. Especially because iter(s).next() is not even valid code in 3.0. This seems reasonably legible to you? Strikes me as coding by incantation. Also, while I've heard people say that the naive approach is slower, I'm not getting that result. Here's my test: smrt = timeit.Timer(next(iter(s)), s=set(range(100))) smrt.repeat(10) [1.2845709323883057, 0.60247397422790527, 0.59621405601501465, 0.59133195877075195, 0.58387589454650879, 0.56839084625244141, 0.56839680671691895, 0.56877803802490234, 0.56905913352966309, 0.56846404075622559] naive = timeit.Timer(x=s.pop();s.add(x), s=set(range(100))) naive.repeat(10) [0.93139314651489258, 0.53566789627075195, 0.53674602508544922, 0.53608798980712891, 0.53634309768676758, 0.53557991981506348, 0.53578495979309082, 0.53666114807128906, 0.53576493263244629, 0.53491711616516113] Perhaps my test is flawed in some way? Geremy Condra Well, it also uses a fairly trivial set. 'set(range(100))' is going to generally have 0 collisions and everything will hash to a unique bucket. As such, pop ing an item out of the hash is a single val = table[int mask]; table[int mask] = _dummy, and then looking it up again requires 2 table lookups (one finds the _dummy, the next finds that the chain is broken and can rewrite the _dummy.) However, if a set is more full, or has more collisions, or ... then pop() and add() become relatively more expensive. Surprising to me, is that next(iter(s)) was actually slower than .pop() + .add() for 100 node set in my testing: $ alias TIMEIT=python -m timeit -s 's = set(range(100)' $ TIMEIT x = next(iter(s)) 100 loops, best of 3: 0.263 usec per loop $ TIMEIT x = s.pop(); s.add(x) 100 loops, best of 3: 0.217 usec per loop though both are much slower than the fastest we've found: $ TIMEIT for x in s: break 1000 loops, best of 3: 0.0943 usec per loop now, what about a set with *lots* of collisions. Create 100 keys that all hash to the same bucket: aliase TIMEIT=python -m timeit -s 's = set([x*1024*1024 for x in range(100)])' $ TIMEIT x = next(iter(s)) 100 loops, best of 3: 0.257 usec per loop $ TIMEIT x = s.pop(); s.add(x) 100 loops, best of 3: 0.218 usec per loop I tried a few different ways, and I got the same results, until I did: $ python -m timeit -s s = set(range(10, 1000100)) next(iter(s)) 1000 loops, best of 3: 255 usec per loop Now something seems terribly wrong here. next(iter(s)) suddenly jumps up from being 0.3 us, to being more than 200us. Or ~1000x slower. I realize the above has 900k keys, which is big. But 'next(iter(s))' should only touch 1, and, in fact, should always return the *first* entry. My best guess is just that the *first* entry in the internal set table is no longer close to offset 0, which means that 'next(iter(s))' has to evaluate a bunch of table slots before it finds a non-empty entry. Anyway, none of the proposals will really ever be faster than: for x in s: break It is a bit ugly of a construct, but you don't have an attribute lookup, etc. As long as you don't do: for x in s: pass Then it stays nice and fast. John =:- Thanks for the info. Doing a bit more digging it appears that taking the lookup out of the picture puts the pop/add more or less on par with the for x in s: break solution, with the former behaving more predictably and the latter averaging marginally faster times. Either way, the loss in readability isn't worth it to me to get the minor improvement in performance. Given that adding a set.pick method would incur half the lookup cost that pop/add does, I think its reasonable to say that even using the fastest method proposed to implement it won't differ all that greatly from the least efficient proposal, and that its therefore pointless to consider set.pick from an optimisation standpoint. Aesthetics, of course, are another thing altogether. Geremy Condra ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, 5 Nov 2009 08:13:55 pm Michael Foord wrote: There are several partial implementations, including Python inspired languages, but if we are looking at 'major complete implementations' then the current list seems to be: CPython, Jython, IronPython and PyPy. Even Unladen Swallow is a fork (sorry - I mean branch) of CPython rather than a separate implementation. I don't know how mature or active it is, so it may not count as either major or complete, but there's also CLPython: http://common-lisp.net/project/clpython/ -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Thu, Nov 5, 2009 at 5:02 PM, Raymond Hettinger pyt...@rcn.com wrote: Forgot to post the code. It is short, fast, and easy. It is explicit about handing the case with an empty input. And it is specific about which value it returns (always the first iterated value; not an arbitrary one). There's no guessing about what it does. It gets the job done. I'm trying to take this suggestion in the best possible light, which is that you honestly think I didn't read past Chapter 3 of the Python Tutorial, and I am therefore in fact unfamiliar with function definitions. I read Raymond's suggestion rather as a question: why bother with a tedious, multi-year process, when a three-line function will achieve exactly the same? Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 3003 - Python Language Moratorium
On Thu, Nov 5, 2009 at 19:23, Guido van Rossum gu...@python.org wrote: On Thu, Nov 5, 2009 at 3:35 PM, Brett Cannon br...@python.org wrote: On Thu, Nov 5, 2009 at 15:26, Guido van Rossum gu...@python.org wrote: I have come to the conclusion that there are better ways to pre-announce that a module is going to disappear instead of deprecation warnings. What exactly are those better ways? Document as deprecated only? Sorry, I have an existence proof, but no construction. :-) Ideas welcome. Silent deprecations, loud documentation, biweekly home visits, whatever, as long as it doesn't log a message by default. Well, one option is to come up with the equivalent of -3, but for all warnings; the antithesis of -W. And obviously glaring deprecation warnings in the docs (removal has been discussed but always shot down as someone who comes across old code might still need docs for it). The clarification I need is will this in any way influence when modules are removed. If they stay in for the life of a major version then I want it made clear that bug fixes for the code take lower priority over all other code in the standard library. -Brett ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from a setwithoutremoving it
On Fri, 6 Nov 2009 10:52:54 am Yuvgoog Greenle wrote: On Fri, Nov 6, 2009 at 1:17 AM, James Y Knight f...@fuhm.net wrote: Is this thread over yet? Sorry, I just had to point out that pop/add has a side effect that would be apparent on a set that multiple threads access - it loses an item and then gets it back. Sounds like a sleeper race condition that's going to be rare but extremely hard to find if it does occur. Crooked as a gil. Surely Raymond's suggestion also suffers from a similar race condition? for x in set: return x creates a set_iterator. If another thread modifies the original set after the set_iterator is created but before the return, you would get a mysterious and almost impossible to debug RuntimeError. -- Steven D'Aprano ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Retrieve an arbitrary element from asetwithoutremoving it
On Thu, Nov 5, 2009 at 11:43 PM, Martin v. Löwis mar...@v.loewis.de wrote: I read Raymond's suggestion rather as a question: why bother with a tedious, multi-year process, when a three-line function will achieve exactly the same? Because it doesn't achieve exactly the same. What I want is a sane, rational way to describe what I'm doing in the core API, so other programmers learning the language don't spend the amount of time I did perplexed that there was a .pop() and a .remove() and a .discard(), but there wasn't a .pick(). I don't want to have to write the same little helper function in every project to fill a deficiency in the library. I don't want to have to argue about the flaws in solutions with race conditions, or the fact that cheap functions become expensive functions when performed over the network, or that there's a real value in having an atomic operation which throws a sane exception when it fails, or how it's disturbing to my OCD core to have an API which believes: if x in s: s.remove(x) ... is too confusing, so there should be a .discard() method, but ... for x in s: break ... is self-evident and obvious, so there's no need for a .pick(). -- Chris ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
