Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-08-01 Thread Mark Shannon 

While the idea behind PEP 424 is sound, the text of the PEP is rather vague and 
missing a lot of details.
There was extended discussion on the details, but none of that has appeared in 
the PEP yet.

So Alex, how about adding those details?

Also the rationale is rather poor.
Given that CPython is the reference implementation, PyPy should be compared to 
CPython, not vice-versa.
Reversing PyPy and CPython in the rationale gives:

'''
Being able to pre-allocate lists based on the expected size, as estimated by 
__length_hint__,
can be a significant optimization.
PyPy has been observed to run some code slower than CPython, purely because 
this optimization is absent.
'''

Which is a PyPy bug report, not a rationale for a PEP ;)

Perhaps a better rationale would something along the lines of:

'''
Adding a __length_hint__ method to the iterator protocol allows sequences, 
notably lists,
to be initialised from iterators with only a single resize operation.
This allows sequences to be intialised quickly, yet have a small growth factor, 
reducing memory use.
'''

Cheers,
Mark.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-08-01 Thread Maciej Fijalkowski 
On Wed, Aug 1, 2012 at 10:46 AM, Mark Shannon  wrote:
> While the idea behind PEP 424 is sound, the text of the PEP is rather vague
> and missing a lot of details.
> There was extended discussion on the details, but none of that has appeared
> in the PEP yet.
>
> So Alex, how about adding those details?
>
> Also the rationale is rather poor.
> Given that CPython is the reference implementation, PyPy should be compared
> to CPython, not vice-versa.
> Reversing PyPy and CPython in the rationale gives:
>
> '''
> Being able to pre-allocate lists based on the expected size, as estimated by
> __length_hint__,
>
> can be a significant optimization.
> PyPy has been observed to run some code slower than CPython, purely because
> this optimization is absent.
> '''
>
> Which is a PyPy bug report, not a rationale for a PEP ;)
>
> Perhaps a better rationale would something along the lines of:
>
> '''
> Adding a __length_hint__ method to the iterator protocol allows sequences,
> notably lists,
> to be initialised from iterators with only a single resize operation.
> This allows sequences to be intialised quickly, yet have a small growth
> factor, reducing memory use.
> '''
>

Hi Mark.

It's not about saving memory. It really is about speed. Noone bothered
measuring cpython with length hint disabled to compare, however we did
that for pypy hence the rationale contains it. It's merely to state
"this seems like an important optimization". Since the C-level code
involved is rather similar (it's mostly runtime anyway), it seems
reasonable to draw a conclusion that removing length hint from cpython
would cause slowdown.

Cheers,
fijal
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-08-01 Thread Mark Shannon 

Maciej Fijalkowski wrote:

On Wed, Aug 1, 2012 at 10:46 AM, Mark Shannon  wrote:

While the idea behind PEP 424 is sound, the text of the PEP is rather vague
and missing a lot of details.
There was extended discussion on the details, but none of that has appeared
in the PEP yet.

So Alex, how about adding those details?

Also the rationale is rather poor.
Given that CPython is the reference implementation, PyPy should be compared
to CPython, not vice-versa.
Reversing PyPy and CPython in the rationale gives:

'''
Being able to pre-allocate lists based on the expected size, as estimated by
__length_hint__,

can be a significant optimization.
PyPy has been observed to run some code slower than CPython, purely because
this optimization is absent.
'''

Which is a PyPy bug report, not a rationale for a PEP ;)

Perhaps a better rationale would something along the lines of:

'''
Adding a __length_hint__ method to the iterator protocol allows sequences,
notably lists,
to be initialised from iterators with only a single resize operation.
This allows sequences to be intialised quickly, yet have a small growth
factor, reducing memory use.
'''



Hi Mark.

It's not about saving memory. It really is about speed. Noone bothered
measuring cpython with length hint disabled to compare, however we did
that for pypy hence the rationale contains it. It's merely to state
"this seems like an important optimization". Since the C-level code
involved is rather similar (it's mostly runtime anyway), it seems
reasonable to draw a conclusion that removing length hint from cpython
would cause slowdown.


It is not about making it faster *or* saving memory, but *both*.
Without __length_hint__ there is a trade off between speed and memory use.
You can have speed at the cost of memory by increasing the resize factor.

With __length_hint__ you can get both speed and good memory use.

Cheers,
Mark

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 0424: A method for exposing a length hint

2012-08-01 Thread Maciej Fijalkowski 
On Wed, Aug 1, 2012 at 12:06 PM, Mark Shannon  wrote:
> Maciej Fijalkowski wrote:
>>
>> On Wed, Aug 1, 2012 at 10:46 AM, Mark Shannon  wrote:
>>>
>>> While the idea behind PEP 424 is sound, the text of the PEP is rather
>>> vague
>>> and missing a lot of details.
>>> There was extended discussion on the details, but none of that has
>>> appeared
>>> in the PEP yet.
>>>
>>> So Alex, how about adding those details?
>>>
>>> Also the rationale is rather poor.
>>> Given that CPython is the reference implementation, PyPy should be
>>> compared
>>> to CPython, not vice-versa.
>>> Reversing PyPy and CPython in the rationale gives:
>>>
>>> '''
>>> Being able to pre-allocate lists based on the expected size, as estimated
>>> by
>>> __length_hint__,
>>>
>>> can be a significant optimization.
>>> PyPy has been observed to run some code slower than CPython, purely
>>> because
>>> this optimization is absent.
>>> '''
>>>
>>> Which is a PyPy bug report, not a rationale for a PEP ;)
>>>
>>> Perhaps a better rationale would something along the lines of:
>>>
>>> '''
>>> Adding a __length_hint__ method to the iterator protocol allows
>>> sequences,
>>> notably lists,
>>> to be initialised from iterators with only a single resize operation.
>>> This allows sequences to be intialised quickly, yet have a small growth
>>> factor, reducing memory use.
>>> '''
>>>
>>
>> Hi Mark.
>>
>> It's not about saving memory. It really is about speed. Noone bothered
>> measuring cpython with length hint disabled to compare, however we did
>> that for pypy hence the rationale contains it. It's merely to state
>> "this seems like an important optimization". Since the C-level code
>> involved is rather similar (it's mostly runtime anyway), it seems
>> reasonable to draw a conclusion that removing length hint from cpython
>> would cause slowdown.
>
>
> It is not about making it faster *or* saving memory, but *both*.
> Without __length_hint__ there is a trade off between speed and memory use.
> You can have speed at the cost of memory by increasing the resize factor.

No, you cannot.

if you allocate a huge region, you're not gonna make much of speed,
because at the end you need to copy stuff anyway. Besides large
allocations are slow. With length hint that is correct (sometimes you
can do that) you have a zero-copy scenario
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] HTTPS repositories failing when using selfsigned certs

2012-08-01 Thread Antoine Pitrou 
On Wed, 01 Aug 2012 05:58:06 +0200
Jesus Cea  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> My mercurial clone is , and today I
> can't create a patch from it (in the bug tracker). No explanation in
> the web interface, but checking the sourcecode of the resulting page,
> I see a SSL certificate failure.
> 
> So, looks like bugs.python.org is now verifying repository certificates.
> 
> My certificate is selfsigned and, moreover, it is behind a SNI server,
> so the certificate python.org is getting is a selfsigned "jcea.es"
> certificate.
> 
> What can I do, beside buying a "real" cert?.

Why don't you just use a HTTP URL?

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] HTTPS repositories failing when using selfsigned certs

2012-08-01 Thread Antoine Pitrou 
On Wed, 1 Aug 2012 14:12:54 +0200
Antoine Pitrou  wrote:

> On Wed, 01 Aug 2012 05:58:06 +0200
> Jesus Cea  wrote:
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA1
> > 
> > My mercurial clone is , and today I
> > can't create a patch from it (in the bug tracker). No explanation in
> > the web interface, but checking the sourcecode of the resulting page,
> > I see a SSL certificate failure.
> > 
> > So, looks like bugs.python.org is now verifying repository certificates.
> > 
> > My certificate is selfsigned and, moreover, it is behind a SNI server,
> > so the certificate python.org is getting is a selfsigned "jcea.es"
> > certificate.
> > 
> > What can I do, beside buying a "real" cert?.
> 
> Why don't you just use a HTTP URL?

Whoops, I hadn't seen the P.S. in your e-mail:

> PS: If I try to use the http version of my repository
> (), I get an error: "('invalid token',
> 97)".

In this case the issue with the http version should perhaps be figured
out first.

Regards

Antoine.


-- 
Software development and contracting: http://pro.pitrou.net


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Fix findnocoding.p and pysource.py scripts

2012-08-01 Thread Serhiy Storchaka 

On 01.08.12 21:16, victor.stinner wrote:

http://hg.python.org/cpython/rev/67d36e8ddcfc
changeset:   78375:67d36e8ddcfc
user:Victor Stinner 
date:Wed Aug 01 20:12:51 2012 +0200
summary:
   Fix findnocoding.p and pysource.py scripts

I suppose that these scripts didn't work since Python 3.0.



-line1 = infile.readline()
-line2 = infile.readline()
+with infile:
+line1 = infile.readline()
+line2 = infile.readline()

-if get_declaration(line1) or get_declaration(line2):
-# the file does have an encoding declaration, so trust it
-infile.close()
-return False
+if get_declaration(line1) or get_declaration(line2):
+# the file does have an encoding declaration, so trust it
+infile.close()
+return False


infile.close() is unnecessary here.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Fix findnocoding.p and pysource.py scripts

2012-08-01 Thread Victor Stinner 
2012/8/1 Serhiy Storchaka :
>> changeset:   78375:67d36e8ddcfc
>>Fix findnocoding.p and pysource.py scripts
>>
>
>> -line1 = infile.readline()
>> -line2 = infile.readline()
>> +with infile:
>> +line1 = infile.readline()
>> +line2 = infile.readline()
>>
>> -if get_declaration(line1) or get_declaration(line2):
>> -# the file does have an encoding declaration, so trust it
>> -infile.close()
>> -return False
>> +if get_declaration(line1) or get_declaration(line2):
>> +# the file does have an encoding declaration, so trust it
>> +infile.close()
>> +return False
>
> infile.close() is unnecessary here.

Ah yes correct, I forgot this one. The new changeset 8ace059cdffd removes it.

It is not perfect, there is still a race condition in
looks_like_python() (pysource.py) if KeyboardInterrupt occurs between
the file is opened and the beginning of the "with infile" block, but
it don't think that it is really important ;-)

Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com