Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Nathaniel Smith]

On Tue, May 23, 2017 at 8:34 PM, David Wilson  wrote:
> On Tue, May 23, 2017 at 06:13:17PM -0700, Cory Benfield wrote:
>
>> There are discussions around Requests unvendoring its dependencies
>> thanks to the improved nature of pip. This might be a year of pretty
>> big changes for Requests.
>
> In which case, what is to prevent Requests from just depending on
> pyOpenSSL as usual?

I thought that requests couldn't have any hard dependencies on C
extensions, because pip vendors requests, and pip needs to be
pure-python for bootstrapping purposes? Cory would know better than me
though, so perhaps I'm wrong...

> I'm still writing 2.7 code every day and would love to see it live a
> little longer, but accepting every feature request seems the wrong way
> to go - and MemoryBIO is a hard sell as a security enhancement, it's new
> functionality.

IIUC, the security enhancement is indirect but real: on Windows/MacOS,
Python's dependence on openssl is a security liability, and to get
away from this we need Cory's new library that abstracts over
different TLS implementations. But for applications to take advantage
of this, they need to switch to using the new library. And before they
can switch to using the new library, it needs to work everywhere. And
for the new library to work on python 2 on unix, it needs MemoryBIO's
in the stdlib – ideally using an interface that's as-close-as-possible
to what they look like on python 3, so he doesn't have to implement
totally different backends for py2 and py3, because Cory is already a
hero for trying to make this happen and we don't want to waste any
more of his time than necessary. So the end result is that if you have
Python 2 code doing SSL/TLS on Windows/MacOS, and you want proper
trust handling and prompt security updates, then MemoryBIO support is
actually on the critical path for making that happen.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Victor Stinner]

Sure, make your change and then update libffi!

Victor

Le 23 mai 2017 18:19, "Steve Dower"  a écrit :

> On 23May2017 1212, Victor Stinner wrote:
>
>> 2017-05-22 13:17 GMT-05:00 Steve Dower :
>>
>>> Once the special protection is removed, most of these cases will become
>>> OSError due to the general protection against segmentation faults.
>>>
>>
>> It didn't know that ctypes on Windows had a special protection against
>> programming errors. I'm not aware of such protection Linux. If you
>> call a function with the wrong number of arguments, it's likely to
>> crash or return random data.
>>
>> I guess that the point is to help debugging. But since Python 3.6,
>> faulthandler now registers a Windows exception handler and so it able
>> to dump the Python traceback on any Windows exception:
>> https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable
>>
>> So I think that it's now fine to remove the ctypes protection. Just
>> advice (remind? ;-)) users to enable faulthandler: python3 -X
>> faulthandler, or call faulthandler.enable(). (You might want to use a
>> log file for that on Windows, depends on the use case.)
>>
>
> faulthandler is already recommended in the docs, and the existing SEH
> protection for access violations will remain (since that is independent of
> libffi).
>
> I'll be honest, I have appreciated the functionality in the past, but it
> really isn't good practice and getting rid of it will be an overall
> benefit. Technically even the segfault protection isn't a great idea, since
> you really do end up in an unknown state with regards to memory page
> allocations, but it's better than crashing all the way out.
>
> Cheers,
> Steve
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

Le 23 mai 2017 20:43, "David Wilson"  a écrit :
In which case, what is to prevent Requests from just depending on

pyOpenSSL as usual?


>From what I heard, pyOpenSSL development is slowing down, so I'm not sure
that it's really safe and future-proof (TLS 1.3 anyone?).

I'm still writing 2.7 code every day and would love to see it live a
little longer, but accepting every feature request seems the wrong way
to go - and MemoryBIO is a hard sell as a security enhancement, it's new
functionality.


You are true that they are new features. I disagree on the "accepting every
feature" part: we are talking about two classes and it's restricted to
security. Security matters for me and for practical reasons explained in
thid thread, we need the two classes.

Cory's PEP adds long awaited features (bugfixes?) to TLS, like getting
access to root certificates on macOS and Windows. Not having to provide our
own set of root certificates would make applications hlobally more secure.
It's ttricky to update certificates. It happens that root CA are revoked
after aa break-in or because the CA is no more trusted.

I also understood that getting access to system CA allows admins to
register their company CA and so avoid that users ignore the TLS warning
(unknown CA).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Cory Benfield]

> On 23 May 2017, at 20:34, David Wilson  wrote:
> 
> On Tue, May 23, 2017 at 06:13:17PM -0700, Cory Benfield wrote:
> 
>> There are discussions around Requests unvendoring its dependencies
>> thanks to the improved nature of pip. This might be a year of pretty
>> big changes for Requests.
> 
> In which case, what is to prevent Requests from just depending on
> pyOpenSSL as usual?

Requests currently does not depend on PyOpenSSL on 2.7: it’s simply an optional 
dependency for those who need it.

Cory
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

Le 23 mai 2017 17:49, "Alex Gaynor"  a écrit :

I'm +1 on this, I even wrote the patch: https://bugs.python.org/issue22559
:-) If you're interested in making sure that still applies and tests still
pass, I'd be a big fan.


Oh sorry, I wasn't aware of that one. Sure, will do.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

Le 23 mai 2017 19:46, "Nick Coghlan"  a écrit :
(...) to the RHEL system Python by switching the default

behaviour for new installs to be to verify SSL/TLS certificates
against the system trust store


Awesome! Security, I see security everywhere!

Stay safe,
Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[David Wilson]

On Tue, May 23, 2017 at 06:13:17PM -0700, Cory Benfield wrote:

> There are discussions around Requests unvendoring its dependencies
> thanks to the improved nature of pip. This might be a year of pretty
> big changes for Requests.

In which case, what is to prevent Requests from just depending on
pyOpenSSL as usual?

I'm still writing 2.7 code every day and would love to see it live a
little longer, but accepting every feature request seems the wrong way
to go - and MemoryBIO is a hard sell as a security enhancement, it's new
functionality.


David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Nick Coghlan]

On 24 May 2017 at 10:46, Victor Stinner  wrote:
> Hi,
>
> Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python
> 2.7? I can do the backport.
>
>   https://docs.python.org/dev/library/ssl.html#ssl.MemoryBIO

+1 from me - the last SSL module resync to 2.7 was from 3.4, and
bringing them back closer to feature parity again is genuinely
beneficial in ensuring the Python ecosystem is able to keep up with
evolving network security standards.

Guido requested back when PEP 466 was written that any further
security backports come with their own PEP in order to clearly
communicate what's being backported, and the "What's New in Python 2.7
Maintenance Releases?" section is organised accordingly.

However, the "Why?" section in such a PEP can be a lot shorter than it
was for the original precedent setting one, since it only needs to
describe the benefits of the specific security features being
backported, rather than having to make the case for the idea of
backporting security features in general.

Cheers,
Nick.

P.S. Somewhat related, folks may be interested to know that the
upcoming RHEL 7.4 release finally completes the process of backporting
PEPs 466 & 476 to the RHEL system Python by switching the default
behaviour for new installs to be to verify SSL/TLS certificates
against the system trust store:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7-Beta/html/7.4_Release_Notes/new_features_compiler_and_tools.html

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Steve Dower]

On 23May2017 1212, Victor Stinner wrote:

2017-05-22 13:17 GMT-05:00 Steve Dower :

Once the special protection is removed, most of these cases will become
OSError due to the general protection against segmentation faults.


It didn't know that ctypes on Windows had a special protection against
programming errors. I'm not aware of such protection Linux. If you
call a function with the wrong number of arguments, it's likely to
crash or return random data.

I guess that the point is to help debugging. But since Python 3.6,
faulthandler now registers a Windows exception handler and so it able
to dump the Python traceback on any Windows exception:
https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable

So I think that it's now fine to remove the ctypes protection. Just
advice (remind? ;-)) users to enable faulthandler: python3 -X
faulthandler, or call faulthandler.enable(). (You might want to use a
log file for that on Windows, depends on the use case.)


faulthandler is already recommended in the docs, and the existing SEH 
protection for access violations will remain (since that is independent 
of libffi).


I'll be honest, I have appreciated the functionality in the past, but it 
really isn't good practice and getting rid of it will be an overall 
benefit. Technically even the segfault protection isn't a great idea, 
since you really do end up in an unknown state with regards to memory 
page allocations, but it's better than crashing all the way out.


Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Cory Benfield]

> On 23 May 2017, at 17:58, Victor Stinner  wrote:
> 
> 2017-05-23 19:54 GMT-05:00 Cory Benfield :
>> In the absence of a Python 2.7 backport, Requests is required to basically 
>> use the same solution that Twisted currently does: namely, a mandatory 
>> dependency on PyOpenSSL.
> 
> Last time I looked at requests, it embedded all its dependencies. I
> don't like the idea of embedding PyOpenSSL…

There are discussions around Requests unvendoring its dependencies thanks to 
the improved nature of pip. This might be a year of pretty big changes for 
Requests.

Cory

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

2017-05-23 19:54 GMT-05:00 Cory Benfield :
> In the absence of a Python 2.7 backport, Requests is required to basically 
> use the same solution that Twisted currently does: namely, a mandatory 
> dependency on PyOpenSSL.

Last time I looked at requests, it embedded all its dependencies. I
don't like the idea of embedding PyOpenSSL...

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

Well, things evolved recently. asyncio is getting more popular, TLS
became even more important, and pyOpenSSL is not more evolving (from
what I heard from his maintainer, Hynek). Does Twister still use
pyOpenSSL?

I like the idea of putting more security stuff into Python to ease
maintainance of applications and try to make it as security as
possible. Python has more resources (ex: CI) than some smaller
projects.

Victor

2017-05-23 19:49 GMT-05:00 Alex Gaynor :
> I'm +1 on this, I even wrote the patch: https://bugs.python.org/issue22559
> :-) If you're interested in making sure that still applies and tests still
> pass, I'd be a big fan.
>
> In addition to all the benefits you mentioned, it also substantially reduces
> the diff between 2.7 and 3.x (or at least it did when I originally wrote
> it).
>
> Cheers,
> Alex
>
> On Tue, May 23, 2017 at 8:46 PM, Victor Stinner 
> wrote:
>>
>> Hi,
>>
>> Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python
>> 2.7? I can do the backport.
>>
>>   https://docs.python.org/dev/library/ssl.html#ssl.MemoryBIO
>>
>> Cory Benfield told me that it's a blocking issue for him to implement
>> his PEP 543 -- A Unified TLS API for Python 2.7:
>>
>>   https://www.python.org/dev/peps/pep-0543/
>>
>> And I expect that if a new cool TLS API happens, people will want to
>> use it on Python 2.7-3.6, not only on Python 3.7. Security evolves
>> more quickly that the current Python release process, and people wants
>> to keep their application secure.
>>
>> From what I understood, he wants to first implement an abstract
>> MemoryBIO API (http://sans-io.readthedocs.io/ like API? I'm not sure
>> about that), and then implement a socket/FD based on top of that.
>> Maybe later, some implementations might have a fast-path using
>> socket/FD directly.
>>
>> He described me his PEP and I strongly support it (sorry, I missed it
>> when he posted it on python-dev), but we decided (Guido van Rossum,
>> Christian Heimes, Cory Benfield and me, see the tweet below) to not
>> put this in the stdlib right now, but spend more time on testing it on
>> Twisted, asyncio, requests, etc. So publishing an implementation on
>> PyPI was proposed instead. It seems like we agreed on a smooth plan
>> (or am I wrong, Cory?).
>>
>>   https://twitter.com/VictorStinner/status/865467388141027329
>>
>> I'm quite sure that Twisted will love MemoryBIO on Python 2.7 as well,
>> to implement TLS, especially on Windows using IOCP. Currently,
>> external libraries (C extensions) are required.
>>
>> I'm not sure if the PEP 466 should be amended for that? Is a new PEP
>> really needed? MemoryBIO/SSLObject are tiny. Nick (Coghlan): what do
>> you think?
>>
>>   https://www.python.org/dev/peps/pep-0466/
>>
>> Victor
>
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: D1B3 ADC0 E023 8CA6
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Cory Benfield]

> On 23 May 2017, at 17:46, Victor Stinner  wrote:
> 
> 
> He described me his PEP and I strongly support it (sorry, I missed it
> when he posted it on python-dev), but we decided (Guido van Rossum,
> Christian Heimes, Cory Benfield and me, see the tweet below) to not
> put this in the stdlib right now, but spend more time on testing it on
> Twisted, asyncio, requests, etc. So publishing an implementation on
> PyPI was proposed instead. It seems like we agreed on a smooth plan
> (or am I wrong, Cory?).

Yes, this is correct. There are plans afoot to look at moving Requests to a 
more event-loop-y model, and doing so basically mandates a MemoryBIO. In the 
absence of a Python 2.7 backport, Requests is required to basically use the 
same solution that Twisted currently does: namely, a mandatory dependency on 
PyOpenSSL.

I’m very much +1 on this.

Cory
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Alex Gaynor]

I'm +1 on this, I even wrote the patch: https://bugs.python.org/issue22559
:-) If you're interested in making sure that still applies and tests still
pass, I'd be a big fan.

In addition to all the benefits you mentioned, it also substantially
reduces the diff between 2.7 and 3.x (or at least it did when I originally
wrote it).

Cheers,
Alex

On Tue, May 23, 2017 at 8:46 PM, Victor Stinner 
wrote:

> Hi,
>
> Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python
> 2.7? I can do the backport.
>
>   https://docs.python.org/dev/library/ssl.html#ssl.MemoryBIO
>
> Cory Benfield told me that it's a blocking issue for him to implement
> his PEP 543 -- A Unified TLS API for Python 2.7:
>
>   https://www.python.org/dev/peps/pep-0543/
>
> And I expect that if a new cool TLS API happens, people will want to
> use it on Python 2.7-3.6, not only on Python 3.7. Security evolves
> more quickly that the current Python release process, and people wants
> to keep their application secure.
>
> From what I understood, he wants to first implement an abstract
> MemoryBIO API (http://sans-io.readthedocs.io/ like API? I'm not sure
> about that), and then implement a socket/FD based on top of that.
> Maybe later, some implementations might have a fast-path using
> socket/FD directly.
>
> He described me his PEP and I strongly support it (sorry, I missed it
> when he posted it on python-dev), but we decided (Guido van Rossum,
> Christian Heimes, Cory Benfield and me, see the tweet below) to not
> put this in the stdlib right now, but spend more time on testing it on
> Twisted, asyncio, requests, etc. So publishing an implementation on
> PyPI was proposed instead. It seems like we agreed on a smooth plan
> (or am I wrong, Cory?).
>
>   https://twitter.com/VictorStinner/status/865467388141027329
>
> I'm quite sure that Twisted will love MemoryBIO on Python 2.7 as well,
> to implement TLS, especially on Windows using IOCP. Currently,
> external libraries (C extensions) are required.
>
> I'm not sure if the PEP 466 should be amended for that? Is a new PEP
> really needed? MemoryBIO/SSLObject are tiny. Nick (Coghlan): what do
> you think?
>
>   https://www.python.org/dev/peps/pep-0466/
>
> Victor
>



-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: D1B3 ADC0 E023 8CA6
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Backport ssl.MemoryBIO on Python 2.7?

[Victor Stinner]

Hi,

Would you be ok to backport ssl.MemoryBIO and ssl.SSLObject on Python
2.7? I can do the backport.

  https://docs.python.org/dev/library/ssl.html#ssl.MemoryBIO

Cory Benfield told me that it's a blocking issue for him to implement
his PEP 543 -- A Unified TLS API for Python 2.7:

  https://www.python.org/dev/peps/pep-0543/

And I expect that if a new cool TLS API happens, people will want to
use it on Python 2.7-3.6, not only on Python 3.7. Security evolves
more quickly that the current Python release process, and people wants
to keep their application secure.

>From what I understood, he wants to first implement an abstract
MemoryBIO API (http://sans-io.readthedocs.io/ like API? I'm not sure
about that), and then implement a socket/FD based on top of that.
Maybe later, some implementations might have a fast-path using
socket/FD directly.

He described me his PEP and I strongly support it (sorry, I missed it
when he posted it on python-dev), but we decided (Guido van Rossum,
Christian Heimes, Cory Benfield and me, see the tweet below) to not
put this in the stdlib right now, but spend more time on testing it on
Twisted, asyncio, requests, etc. So publishing an implementation on
PyPI was proposed instead. It seems like we agreed on a smooth plan
(or am I wrong, Cory?).

  https://twitter.com/VictorStinner/status/865467388141027329

I'm quite sure that Twisted will love MemoryBIO on Python 2.7 as well,
to implement TLS, especially on Windows using IOCP. Currently,
external libraries (C extensions) are required.

I'm not sure if the PEP 466 should be amended for that? Is a new PEP
really needed? MemoryBIO/SSLObject are tiny. Nick (Coghlan): what do
you think?

  https://www.python.org/dev/peps/pep-0466/

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Nathaniel Smith]

Well, it sounds like you have an answer to the question of whether the
cpython developers are interested in making official snap releases, but of
course this doesn't stop you making unofficial snap releases, and simple
standalone python distributions can be pretty handy. I just wanted to point
out that virtualenv support is a very important feature of python builds,
and it may be a challenge to support this within the app-as-container
paradigm, because it requires that the python interpreter be able to locate
itself, copy itself to arbitrary directories, and then run the copy. (Or
maybe it's fine, dunno. And python 3 has some changes that might make it
friendlier than python 2 in this regard.)

-n

On May 16, 2017 8:09 AM, "Martin Wimpress" 
wrote:

> Hi all,
>
> I work at Canonical as part of the engineering team developing Ubuntu
> and Snapcraft [1] and I'm a long time Python fan :-)
>
> We've created snaps, a platform that enables projects to directly
> control delivery of software updates to users. This video of a
> lightning talk by dlang developers at DConf2017 [2] shows how they've
> made good use of snaps to distribute their compiler. They found the
> release channels particularly useful so their users can track a
> specific release.
>
> Is there someone here who'd be interested in doing the same for Python?
>
> [1] https://snapcraft.io/
> [2] https://www.youtube.com/watch?v=M-bDzr4gYUU
> [3] https://snapcraft.io/docs/core/install
> [4] https://build.snapcraft.io/
>
> --
> Regards, Martin.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> njs%40pobox.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Victor Stinner]

2017-05-22 13:17 GMT-05:00 Steve Dower :
> Once the special protection is removed, most of these cases will become
> OSError due to the general protection against segmentation faults.

It didn't know that ctypes on Windows had a special protection against
programming errors. I'm not aware of such protection Linux. If you
call a function with the wrong number of arguments, it's likely to
crash or return random data.

I guess that the point is to help debugging. But since Python 3.6,
faulthandler now registers a Windows exception handler and so it able
to dump the Python traceback on any Windows exception:
https://docs.python.org/dev/library/faulthandler.html#faulthandler.enable

So I think that it's now fine to remove the ctypes protection. Just
advice (remind? ;-)) users to enable faulthandler: python3 -X
faulthandler, or call faulthandler.enable(). (You might want to use a
log file for that on Windows, depends on the use case.)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python is Bae

[Steve Holden]

Hi Victor,

Since this request (or at least the bit about branded goods) isn't really
appropriate for python-dev I am taking the liberty of forwarding this email
to the PSF's administrator, who may be better placed to help. I have Bcc'd
python-dev on this reply so they aren't troubled by correspondence on that
topic.

I applaud your efforts to build a Python community from the ground up, and
wish your efforts every success. I imagine there are other developers who
feel similarly.

regards
 Steve

Steve Holden

On Mon, May 22, 2017 at 8:59 PM, Victor Njunu  wrote:

> Dear Python,
>
> My name is Victor Mshindi and I am a student at United States
> International University - Africa(USIU - Africa) studying Information
> Systems Technology and also a member of the growing tech community. I've
> been working with python for 2 years now, currently using it for machine
> learning and web development(Django) and would like to first say thank you
> for this amazing language and the community you've built.
>
> At USIU we realized there is an increasing number of tech students who
> aren't self learning and depending on classes only, we also noticed that
> the community was becoming exclusive for top developers and not passing
> knowledge to new students. As a result we decide to start a python class to
> help beginners and then a django class to help them transition and become
> really good developers. We also hope to grow the python sub community and
> one day hold our very own Nairobi Pycon giving python developers a chance
> to learn and interact with members form around Kenya , outside Kenya and
> hopefully some core python members.
>
> In order to improve the quality of the class we would need to provide
> attendees branded merchandise, snacks & lunch and hopefully a Skype session
> with one of the core members. This will help excite them and welcome them
> in the best way possible to the python community. Therefore I would like to
> request for support from the foundation by providing a Skype session,
> Branded merchandise and if possible financial support to buy lunch & snacks.
>
> Regards,
> Victor Mshindi
> United States International University - Africa
> Student and IT Club President,
> Django Full stack Developer,
> +254-712-619-612
> Email Hygiene scanner powered by Forefront Online Protection for Exchange.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> steve%40holdenweb.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Guido van Rossum]

I think the I inevitable conclusion is"thanks, but no thanks."

On May 23, 2017 8:05 AM, "Martin Wimpress" 
wrote:

> Hi,
>
> On 23/05/17 00:56, Matt Billenstein wrote:
>
>> On Tue, May 16, 2017 at 11:31:42AM +0100, Martin Wimpress wrote:
>>
>>> Is there someone here who'd be interested in doing the same for Python?
>>>
>>
>> Do snaps support Windows and/or MacOS?
>>
>
> You can't install snaps on Windows or macOS. Snaps are currently targeting
> various Linux distros, you can create Linux snaps on Windows using WSL and
> we're working on improving the tooling for snap creation on macOS and
> Windows.
>
> m
>>
>>
> --
> --
> Regards, Martin.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

[Nick Coghlan]

On 23 May 2017 at 18:38, INADA Naoki  wrote:
> Hi, Nick.
>
> I read again and I think PEP 538 is mostly ready for accepted,
> without waiting PEP 540.

Great to hear!

> One remaining my concern is setting LANG.
>
>> Setting LANG to C.UTF-8 ensures that even components that only check the 
>> LANG fallback for their locale settings will still use C.UTF-8 .
> https://www.python.org/dev/peps/pep-0538/#setting-both-lc-ctype-lang-for-utf-8-locale-coercion
>
> I feel setting only LC_CTYPE making PEP 538 simpler.
> Is there any real component using LANG for deciding encoding?
>
> For example, date command refers LC_TIME.
>
> $ LANG=ja_JP.UTF-8 LC_CTYPE=C date
> 2017年  5月 23日 火曜日 17:31:03 JST
>
> $ LANG=ja_JP.UTF-8 LC_CTYPE=C.UTF-8 date  # Coercing only LC_CTYPE
> 2017年  5月 23日 火曜日 17:32:58 JST
>
> $ LANG=C.UTF-8 LC_CTYPE=C.UTF-8 date  # Coercing both of LC_CTYPE and LANG
> Tue May 23 17:31:10 JST 2017
>
> In this case, coercing only LC_CTYPE has less side-effect.
>
> Would you add example demonstrates how coercing LANG helps people?

I'm honestly not sure it does - I think it's an assumption I added to
the PEP early on, and never actually tested. Looking at it more
closely now, all of the interpreter level checks are specifically for
LC_CTYPE, and experimenting with "LANG=C LC_CTYPE=C.UTF-8" indicates
that coercing only LC_CTYPE is still enough to fix the GNU readline
encoding compatibility problem covered in
https://www.python.org/dev/peps/pep-0538/#considering-locale-coercion-independently-of-utf-8-mode

So I'll take another pass through the implementation this weekend, and
simplify it to only set LC_CTYPE regardless of whether it's using
C.UTF-8, C.utf8, or UTF-8 as the target locale. Assuming that doesn't
uncover any hidden problems with the idea, I'll then update the PEP to
match.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Steve Dower]

On 23May2017 0844, Mariatta Wijaya wrote:

+1

My understanding is this is a documentation change, marking it as
deprecated in 3.6.2 and a Misc News entry.
No actual code change.
Correct?


That's correct. I'm thinking just a :note: box in that section, and then 
you can keep working on the 3.7 change. I'll let you file the bugs.


Cheers,
Steve

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Mariatta Wijaya]

+1

My understanding is this is a documentation change, marking it as
deprecated in 3.6.2 and a Misc News entry.
No actual code change.
Correct?



Mariatta Wijaya

On Tue, May 23, 2017 at 8:28 AM, Antoine Pitrou  wrote:

> On Mon, 22 May 2017 11:17:18 -0700
> Steve Dower  wrote:
> >
> > I'd like to propose a highly-accelerated deprecation period for this
> > specific feature, starting in CPython 3.6.2 and being "completed" in
> > 3.7.0, when we will hopefully move onto a newer libffi.
> >
> > In general, the "feature" is a misfeature anyway, since calling a native
> > function with incorrect arguments is unsupported and a very easy way to
> > cause information leakage or code execution vulnerabilities.
>
> Agreed.
>
> > Does anyone have any reasons to oppose this? It already has votes from
> > another Windows expert and the 3.6/3.7 Release Manager, but we wanted to
> > see if anyone has a concern we haven't thought of.
>
> +1 from me.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> mariatta.wijaya%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Deprecate invalid ctypes call protection on Windows

[Antoine Pitrou]

On Mon, 22 May 2017 11:17:18 -0700
Steve Dower  wrote:
> 
> I'd like to propose a highly-accelerated deprecation period for this 
> specific feature, starting in CPython 3.6.2 and being "completed" in 
> 3.7.0, when we will hopefully move onto a newer libffi.
> 
> In general, the "feature" is a misfeature anyway, since calling a native 
> function with incorrect arguments is unsupported and a very easy way to 
> cause information leakage or code execution vulnerabilities.

Agreed.

> Does anyone have any reasons to oppose this? It already has votes from 
> another Windows expert and the 3.6/3.7 Release Manager, but we wanted to 
> see if anyone has a concern we haven't thought of.

+1 from me.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Antoine Pitrou]

On Mon, 22 May 2017 23:56:43 +
Matt Billenstein  wrote:
> On Tue, May 16, 2017 at 11:31:42AM +0100, Martin Wimpress wrote:
> > Is there someone here who'd be interested in doing the same for Python?  
> 
> Do snaps support Windows and/or MacOS?

If you want a package manager that supports installing binaries on
Linux, Windows and MacOS, I suggest you try out conda:
https://conda.io/docs/
Building conda packages uses a tool named conda-build (both are open
source):
https://conda.io/docs/building/recipe.html

Conda powers the Anaconda software distribution, published by Continuum
Analytics.  Many conda-build recipes are available publicly:
https://github.com/ContinuumIO/anaconda-recipes
For example here is the conda-build recipe for Python 3.6.1:
https://github.com/ContinuumIO/anaconda-recipes/blob/master/python-3.6/meta.yaml
There are also a growing number of community-maintained
recipes and packages:
https://conda-forge.github.io/

Disclaimer: I work for Continuum (though not on Conda nor
Anaconda).

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Martin Wimpress]

Hi,

On 23/05/17 00:56, Matt Billenstein wrote:

On Tue, May 16, 2017 at 11:31:42AM +0100, Martin Wimpress wrote:

Is there someone here who'd be interested in doing the same for Python?


Do snaps support Windows and/or MacOS?


You can't install snaps on Windows or macOS. Snaps are currently 
targeting various Linux distros, you can create Linux snaps on Windows 
using WSL and we're working on improving the tooling for snap creation 
on macOS and Windows.



m



--
--
Regards, Martin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Martin Wimpress]

On 23/05/17 09:27, Petr Viktorin wrote:

On 05/23/2017 01:34 AM, Brett Cannon wrote:


On Tue, 16 May 2017 at 08:08 Martin Wimpress 
> 
wrote:


Hi all,

I work at Canonical as part of the engineering team developing Ubuntu
and Snapcraft [1] and I'm a long time Python fan :-)

We've created snaps, a platform that enables projects to directly
control delivery of software updates to users. This video of a
lightning talk by dlang developers at DConf2017 [2] shows how they've
made good use of snaps to distribute their compiler. They found the
release channels particularly useful so their users can track a
specific release.

Is there someone here who'd be interested in doing the same for 
Python?



So the problem with adding Snap is it's yet one more thing for us to 
create at release time. And if we do this for Snap are there 
competitors from e.g. Fedora that people would want supported?


Not Fedora per se, Flatpak [0] (formerly `xdg-app`) is distro-agnostic [1].


Snaps are supported on Fedora 24, 25 and 26 [1] as well as other distros 
[2].





[0] http://flatpak.org/
[1] http://flatpak.org/faq.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/martin.wimpress%40canonical.com 



[1] 
https://insights.ubuntu.com/2017/04/11/snap-support-lands-in-fedora-24-25-26/

[2] https://snapcraft.io/docs/core/install

--
Regards, Martin.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Freddy Rietdijk]

In my opinion the trend of creating these standalone packages is pretty
bad. Developers that choose to support these kind of packages become
responsible not only for the code they distribute, but also for whether the
dependencies that are bundled in such a standalone package are up to date
and secure.

What libraries would have to be bundled? I guess that depends on how
standalone you want to get. In Nixpkgs we get to 130 MB for Python 3.5, if
it would bundle all its dependencies (glibc, openssl, ncurses, some X11
libs, ...). If I am correct you can assume certain libraries are available
on the host system so you could skip bundling them.

Furthermore, there's now at least three competing formats: Snappy, Flatpak
and AppImage.


On Tue, May 16, 2017 at 12:31 PM, Martin Wimpress <
martin.wimpr...@canonical.com> wrote:

> Hi all,
>
> I work at Canonical as part of the engineering team developing Ubuntu
> and Snapcraft [1] and I'm a long time Python fan :-)
>
> We've created snaps, a platform that enables projects to directly
> control delivery of software updates to users. This video of a
> lightning talk by dlang developers at DConf2017 [2] shows how they've
> made good use of snaps to distribute their compiler. They found the
> release channels particularly useful so their users can track a
> specific release.
>
> Is there someone here who'd be interested in doing the same for Python?
>
> [1] https://snapcraft.io/
> [2] https://www.youtube.com/watch?v=M-bDzr4gYUU
> [3] https://snapcraft.io/docs/core/install
> [4] https://build.snapcraft.io/
>
> --
> Regards, Martin.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> freddyrietdijk%40fridh.nl
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 538 (review round 2): Coercing the legacy C locale to a UTF-8 based locale

[INADA Naoki]

Hi, Nick.

I read again and I think PEP 538 is mostly ready for accepted,
without waiting PEP 540.

One remaining my concern is setting LANG.

> Setting LANG to C.UTF-8 ensures that even components that only check the LANG 
> fallback for their locale settings will still use C.UTF-8 .
https://www.python.org/dev/peps/pep-0538/#setting-both-lc-ctype-lang-for-utf-8-locale-coercion

I feel setting only LC_CTYPE making PEP 538 simpler.
Is there any real component using LANG for deciding encoding?

For example, date command refers LC_TIME.

$ LANG=ja_JP.UTF-8 LC_CTYPE=C date
2017年  5月 23日 火曜日 17:31:03 JST

$ LANG=ja_JP.UTF-8 LC_CTYPE=C.UTF-8 date  # Coercing only LC_CTYPE
2017年  5月 23日 火曜日 17:32:58 JST

$ LANG=C.UTF-8 LC_CTYPE=C.UTF-8 date  # Coercing both of LC_CTYPE and LANG
Tue May 23 17:31:10 JST 2017

In this case, coercing only LC_CTYPE has less side-effect.

Would you add example demonstrates how coercing LANG helps people?


On Tue, May 9, 2017 at 8:57 PM, Nick Coghlan  wrote:
> Hi folks,
>
> Enough changes have accumulated in PEP 538 since the start of the
> previous thread that it seems sensible to me to start a new thread
> specifically covering the current design (which aims to address all
> the concerns raised in the previous thread).
>
> I haven't requoted the PEP in full since it's so long, but will
> instead refer readers to the web version:
> https://www.python.org/dev/peps/pep-0538/
>
> I also generated a diff covered the full changes to the PEP text:
>
> * https://gist.github.com/ncoghlan/1067805fe673b3735ac854e195747493/revisions
> (this is the diff covering the last few days of changes
>
> Summarising the key technical changes:
>
> * to make the runtime behaviour independent of whether or not locale
> coercion took place, stdin and stderr now always have
> "surrogateescape" as their error handler in the potential coercion
> target locales. This means Python will behave the same way regardless
> of whether the locale gets set externally (e.g. by a parent Python
> process or a container image definition) or implicitly during CLI
> startup
> * for the full locales, the interpreter now sets LC_CTYPE and LANG,
> *not* LC_ALL. This means LC_ALL is once again a full locale override,
> and also means that CPython won't inadvertently interfere with other
> locale categories like LC_MONETARY, LC_NUMERIC, etc
> * the reference implementation has been refactored so the bulk of the
> new code lives in the shared library and is exposed to the linker via
> a couple of underscore prefixed API symbols
> (_Py_LegacyLocaleDetected() and _Py_CoerceLegacyLocale()). While the
> current PEP still keeps them private, it would be straightforward to
> make them public for use in embedding applications if we decided we
> wanted to do so.
> * locale coercion and warnings are now enabled by default on all
> platforms that use the autotools-based build chain - the assumption
> that some platforms didn't need them turned out to be incorrect
>
> In addition to being updated to cover the above changes, the Rationale
> section of the PEP has also been updated to explain why it doesn't
> propose setting PYTHONIOENCODING, and to walk through some examples of
> the problems with GNU readlines compatibility when the current locale
> isn't set correctly.
>
> The essential related changes to the reference implementation can be seen 
> here:
>
> * Always set "surrogateescape" for coercion target locales,
> independently of whether or not coercion occurred:
> https://github.com/ncoghlan/cpython/commit/188e7807b6d9e49377aacbb287c074e5cabf70c5
> * Stop setting LC_ALL:
> https://github.com/python/peps/commit/2f530ce0d1fd24835ac0c6f984f40db70482a18f
>
> (There are also some smaller cleanup commits that can be seen by
> browsing that branch on GitHub)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Petr Viktorin]

On 05/23/2017 01:34 AM, Brett Cannon wrote:


On Tue, 16 May 2017 at 08:08 Martin Wimpress 
> 
wrote:


Hi all,

I work at Canonical as part of the engineering team developing Ubuntu
and Snapcraft [1] and I'm a long time Python fan :-)

We've created snaps, a platform that enables projects to directly
control delivery of software updates to users. This video of a
lightning talk by dlang developers at DConf2017 [2] shows how they've
made good use of snaps to distribute their compiler. They found the
release channels particularly useful so their users can track a
specific release.

Is there someone here who'd be interested in doing the same for Python?


So the problem with adding Snap is it's yet one more thing for us to 
create at release time. And if we do this for Snap are there competitors 
from e.g. Fedora that people would want supported?


Not Fedora per se, Flatpak [0] (formerly `xdg-app`) is distro-agnostic [1].


[0] http://flatpak.org/
[1] http://flatpak.org/faq.html
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Snap Python for simple distribution across multiple Linux distros

[Matt Billenstein]

On Tue, May 16, 2017 at 11:31:42AM +0100, Martin Wimpress wrote:
> Is there someone here who'd be interested in doing the same for Python?

Do snaps support Windows and/or MacOS?

m

-- 
Matt Billenstein
m...@vazor.com
http://www.vazor.com/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com