Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nick Coghlan
On 3 June 2017 at 08:21, Nathaniel Smith wrote: > On Fri, Jun 2, 2017 at 1:29 PM, Terry Reedy wrote: >> On 6/2/2017 12:21 PM, Barry Warsaw wrote: >>> >>> On Jun 03, 2017, at 02:10 AM, Nick Coghlan wrote: >> >> The benefit of making any backport a private API is that it would mean we wer

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nathaniel Smith
On Fri, Jun 2, 2017 at 1:29 PM, Terry Reedy wrote: > On 6/2/2017 12:21 PM, Barry Warsaw wrote: >> >> On Jun 03, 2017, at 02:10 AM, Nick Coghlan wrote: > > >>> The benefit of making any backport a private API is that it would mean >>> we weren't committing to support that API for general use: it wo

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-06-02 Thread Guido van Rossum
On Fri, Jun 2, 2017 at 1:07 PM, Koos Zevenhoven wrote: > [...] > I suppose it is, especially because there seems to be nothing that > prevents you from getting runtime annotations in the enclosing class/module > ​: > > > number: int > > @call > def number(): > return 42 > Well mypy actually

Re: [Python-Dev] PEP 544: Protocols - second round

2017-06-02 Thread Ivan Levkivskyi
On 1 June 2017 at 00:10, Guido van Rossum wrote: > > On Wed, May 31, 2017 at 2:16 AM, Ivan Levkivskyi > wrote: > >> On 31 May 2017 at 00:58, Guido van Rossum wrote: >> [...] >> >> Thank you for very detailed answers! I have practically nothing to add. >> It seems to me that most of the Kevin's

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Terry Reedy
On 6/2/2017 12:21 PM, Barry Warsaw wrote: On Jun 03, 2017, at 02:10 AM, Nick Coghlan wrote: The benefit of making any backport a private API is that it would mean we weren't committing to support that API for general use: it would be supported *solely* for the use case discussed in the PEP (i.

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-06-02 Thread Koos Zevenhoven
On Fri, Jun 2, 2017 at 8:57 PM, Guido van Rossum wrote: > On Fri, Jun 2, 2017 at 9:41 AM, Koos Zevenhoven wrote: >> >> I still don't understand what would happen with __annotations__. If >> the decorator returns a non-function, one would expect the annotations >> to be in the __annotations__ attr

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Larry Hastings
On 06/02/2017 02:38 AM, Antoine Pitrou wrote: I hope those are not the actual numbers you're intending to use ;-) I still think that allocating more than 1 or 2MB at once would be foolish. Remember this is data that's going to be carved up into (tens of) thousands of small objects. Large objec

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Larry Hastings
On 06/02/2017 12:09 PM, Tim Peters wrote: I should note that Py_ADDRESS_IN_RANGE is my code - this isn't a backhanded swipe at someone else. One minor note. During the development of 3.6, CPython started permitting some C99-isms, including static inline functions. Py_ADDRESS_IN_RANGE was th

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Larry Hastings
On 06/02/2017 02:46 AM, Victor Stinner wrote: I would be curious of another test: use pymalloc for objects larger than 512 bytes. For example, allocate up to 4 KB? In the past, we already changed the maximum size from 256 to 512 to support most common Python objects on 64-bit platforms. Since

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Tim Peters
[Tim] >> While I would like to increase the pool size, it's fraught with >> danger. [Antoine Pitrou ] > What would be the point of increasing the pool size? Apart from being > able to allocate 4KB objects out of it, I mean. > > Since 4KB+ objects are relatively uncommon (I mean we don't allocate

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Antoine Pitrou
On Fri, 2 Jun 2017 13:23:05 -0500 Tim Peters wrote: > > While I would like to increase the pool size, it's fraught with > danger. What would be the point of increasing the pool size? Apart from being able to allocate 4KB objects out of it, I mean. Since 4KB+ objects are relatively uncommon (I

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Tim Peters
[INADA Naoki ] > ... > Since current pool size is 4KB and there is pool_header in pool, > we can't allocate 4KB block from pool. > And if support 1KB block, only 3KB of 4KB can be actually used. > I think 512 bytes / 4KB (1/8) is good ratio. > > Do you mean increase pool size? > > How about adding

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-06-02 Thread Guido van Rossum
On Fri, Jun 2, 2017 at 9:41 AM, Koos Zevenhoven wrote: > On Fri, Jun 2, 2017 at 6:34 PM, Naomi Seyfer wrote: > > Yep, interested in implementing it! I will put implementation time on my > > schedule and tell y'all when it is, for holding myself accountable -- it > > turns out I never do anythin

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nathaniel Smith
On Jun 2, 2017 7:24 AM, "Ben Darnell" wrote: The PEP's rationale is now "This PEP will help facilitate the future adoption of :pep:`543` across all supported Python versions, which will improve security for both Python 2 and Python 3 users." What exactly are these security improvements? My under

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 12:41 PM, Antoine Pitrou wrote: > > On Fri, 2 Jun 2017 12:22:06 -0400 > Donald Stufft wrote: >> >> It’s not just bootstrapping that pip has a problem with for C extensions, it >> also prevents upgrading PyOpenSSL on Windows because having pip import >> PyOpenSSL locks th

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread David Wilson
On Sat, Jun 03, 2017 at 02:10:50AM +1000, Nick Coghlan wrote: > * and figure out some other pip-specific option for ensurepip > bootstrapping (like a *private* MemoryBIO implementation, or falling > back to synchronous mode in requests) Ignoring Ben's assertion regarding the legitimacy of async w

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 12:39 PM, Nick Coghlan wrote: > > On 3 June 2017 at 02:22, Donald Stufft wrote: >> It’s not just bootstrapping that pip has a problem with for C extensions, it >> also prevents upgrading PyOpenSSL on Windows because having pip import >> PyOpenSSL locks the .dll, and we can’

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Antoine Pitrou
On Fri, 2 Jun 2017 12:22:06 -0400 Donald Stufft wrote: > > It’s not just bootstrapping that pip has a problem with for C extensions, it > also prevents upgrading PyOpenSSL on Windows because having pip import > PyOpenSSL locks the .dll, and we can’t delete it or overwrite it until the > pip pr

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-06-02 Thread Koos Zevenhoven
On Fri, Jun 2, 2017 at 6:34 PM, Naomi Seyfer wrote: > Yep, interested in implementing it! I will put implementation time on my > schedule and tell y'all when it is, for holding myself accountable -- it > turns out I never do anything not on my schedule. > I still don't understand what would happ

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nick Coghlan
On 3 June 2017 at 02:22, Donald Stufft wrote: > It’s not just bootstrapping that pip has a problem with for C extensions, it > also prevents upgrading PyOpenSSL on Windows because having pip import > PyOpenSSL locks the .dll, and we can’t delete it or overwrite it until the > pip process exits and

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Donald Stufft
> On Jun 2, 2017, at 12:10 PM, Nick Coghlan wrote: > > On 2 June 2017 at 19:42, Victor Stinner wrote: >> Thanks Cory for the long explanation. Let me try to summarize (tell me >> if I'm wrong). >> >> We have 3 options: >> >> * Do nothing: reject the PEP 546 and let each project handles securi

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Barry Warsaw
On Jun 03, 2017, at 02:10 AM, Nick Coghlan wrote: >* Introduce a dependency from requests onto PyOpenSSL when running in >async mode on Python 2.7 in the general case, and figure out some >other pip-specific option for ensurepip bootstrapping (like a >*private* MemoryBIO implementation, or falling

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Nick Coghlan
On 2 June 2017 at 19:42, Victor Stinner wrote: > Thanks Cory for the long explanation. Let me try to summarize (tell me > if I'm wrong). > > We have 3 options: > > * Do nothing: reject the PEP 546 and let each project handles security > on its own (current status co) > * Write *new* C code, maybe

[Python-Dev] Summary of Python tracker Issues

2017-06-02 Thread Python tracker
ACTIVITY SUMMARY (2017-05-26 - 2017-06-02) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open6025 (+26) closed 36284 (+43) total 42309 (+69) Open issues wit

Re: [Python-Dev] PEP 484 update proposal: annotating decorated declarations

2017-06-02 Thread Naomi Seyfer
Yep, interested in implementing it! I will put implementation time on my schedule and tell y'all when it is, for holding myself accountable -- it turns out I never do anything not on my schedule. On Wed, May 31, 2017 at 3:17 PM, Guido van Rossum wrote: > On Wed, May 31, 2017 at 6:16 AM, Ivan Le

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Ben Darnell
The PEP's rationale is now "This PEP will help facilitate the future adoption of :pep:`543` across all supported Python versions, which will improve security for both Python 2 and Python 3 users." What exactly are these security improvements? My understanding (which may well be incorrect) is that

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Antoine Pitrou
Thanks you for all the explanations. So to summarize my opinion, I'm still -0.5 on this PEP. I would also like to see the issues Jython, Ubuntu et al. have mentioned solved before this is accepted. Regards Antoine. On Fri, 2 Jun 2017 11:42:58 +0200 Victor Stinner wrote: > Thanks Cory for t

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread INADA Naoki
> I would be curious of another test: use pymalloc for objects larger > than 512 bytes. For example, allocate up to 4 KB? Since current pool size is 4KB and there is pool_header in pool, we can't allocate 4KB block from pool. And if support 1KB block, only 3KB of 4KB can be actually used. I think

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Cory Benfield
> On 2 Jun 2017, at 10:42, Victor Stinner wrote: > > Writing new code seems more risky and error-prone than backporting > already "battle-tested" MemoryBIO from master. I also expect that > writing code to validate certificate will be longer than the "100 > lines of C code in (probably)" expecte

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Victor Stinner
I would be curious of another test: use pymalloc for objects larger than 512 bytes. For example, allocate up to 4 KB? In the past, we already changed the maximum size from 256 to 512 to support most common Python objects on 64-bit platforms. Since Python objects contain many pointers: switching fr

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Victor Stinner
Thanks Cory for the long explanation. Let me try to summarize (tell me if I'm wrong). We have 3 options: * Do nothing: reject the PEP 546 and let each project handles security on its own (current status co) * Write *new* C code, maybe using certitude as a starting point, to offload certifcate val

Re: [Python-Dev] The untuned tunable parameter ARENA_SIZE

2017-06-02 Thread Antoine Pitrou
On Thu, 1 Jun 2017 20:21:12 -0700 Larry Hastings wrote: > On 06/01/2017 02:50 AM, Antoine Pitrou wrote: > > Another possible strategy is: allocate several arenas at once (using a > > larger mmap() call), and use MADV_DONTNEED to relinquish individual > > arenas. > > Thus adding a *fourth* layer

Re: [Python-Dev] RFC: Backport ssl.MemoryBIO and ssl.SSLObject to Python 2.7

2017-06-02 Thread Cory Benfield
> On 1 Jun 2017, at 20:59, Steve Dower wrote: > > On 01Jun2017 1010, Nathaniel Smith wrote: >> I believe that for answering this question about the ssl module, it's really >> only Linux users that matter, since pip/requests/everyone else pushing for >> this only want to use ssl.MemoryBIO on Li