[Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Mark Shannon
To quote from PEP 424: Rationale = Being able to pre-allocate lists based on the expected size, as estimated by ``__length_hint__``, can be a significant optimization. CPython has been observed to run some code faster than PyPy, purely because of this optimization being present.

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Stefan Behnel
Mark Shannon, 16.07.2012 10:37: If resizing of lists is too slow, then we should reconsider the 9/8 factor and/or look to tweak the resizing code. The thing is that the performance is platform specific. On systems with a fast memory allocator, especially on Linux, the difference is negligible.

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Raymond Hettinger
On Jul 16, 2012, at 1:37 AM, Mark Shannon wrote: To quote from PEP 424: Rationale = Being able to pre-allocate lists based on the expected size, as estimated by ``__length_hint__``, can be a significant optimization. CPython has been observed to run some code faster than PyPy,

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Maciej Fijalkowski
On Mon, Jul 16, 2012 at 11:02 AM, Stefan Behnel stefan...@behnel.de wrote: Mark Shannon, 16.07.2012 10:37: If resizing of lists is too slow, then we should reconsider the 9/8 factor and/or look to tweak the resizing code. The thing is that the performance is platform specific. On systems

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Tim Golden
Speaking of which - I find this bikeshed disgusting. Disgusting? Irritating, perhaps, but why should a PEP -- even one whose purpose is to codify existing practice -- not result in discussions about its subject matter? The final P stands for Proposal, not for Pronouncement. TJG

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Stefan Behnel
Maciej Fijalkowski, 16.07.2012 11:14: On Mon, Jul 16, 2012 at 11:02 AM, Stefan Behnel wrote: Mark Shannon, 16.07.2012 10:37: If resizing of lists is too slow, then we should reconsider the 9/8 factor and/or look to tweak the resizing code. The thing is that the performance is platform

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Victor Stinner
*If* resizing list is so slow, then why not make it faster? A simple solution to speed up such problem is to change the overallocation factor, but it may waste memory. Python tries to be fast and to not waste too much memory. Why is it a significant optimisation? How much slower is it? Where

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Nick Coghlan
On Mon, Jul 16, 2012 at 7:21 PM, Tim Golden m...@timgolden.me.uk wrote: Speaking of which - I find this bikeshed disgusting. Disgusting? Irritating, perhaps, but why should a PEP -- even one whose purpose is to codify existing practice -- not result in discussions about its subject matter?

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Antoine Pitrou
On Mon, 16 Jul 2012 23:23:18 +1000 Nick Coghlan ncogh...@gmail.com wrote: - distinguishing between different reasons for saying don't preallocate any space (i.e. returning zero). I still haven't heard a convincing rationale for this one The point is that zero is a valid value for a length

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Nick Coghlan
On Tue, Jul 17, 2012 at 12:01 AM, Antoine Pitrou solip...@pitrou.net wrote: On Mon, 16 Jul 2012 23:23:18 +1000 Nick Coghlan ncogh...@gmail.com wrote: - distinguishing between different reasons for saying don't preallocate any space (i.e. returning zero). I still haven't heard a convincing

Re: [Python-Dev] Do we need __length_hint__ at all? (Was PEP 0424: A method for exposing a length hint)

2012-07-16 Thread Antoine Pitrou
On Tue, 17 Jul 2012 00:18:55 +1000 Nick Coghlan ncogh...@gmail.com wrote: Given that all standard library containers default to assuming a size of zero (absent information indicating otherwise), and a special value would need to be special cased by *every* client of the API (and almost