Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Paul Moore
On 20 February 2013 04:07, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/19/2013 09:37 PM, Paul Moore wrote: On 20 February 2013 00:54, Fred Drake f...@fdrake.net wrote: I'd posit that anything successful will no longer need to be added to the

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Chris Jerdonek
On Wednesday, February 20, 2013 at 2:48 AM, Chris Jerdonek wrote: I meant that bringing distlib into http://hg.python.org/cpython/ would give it more visibility to core devs and others that already keep an eye on python-checkins (the mailing list). And I think seeing the Sphinx-processed

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread rahul garg
Date: Tue, 19 Feb 2013 12:48:02 -0600 Subject: Re: [Python-Dev] Built with VS2012 Express for desktop From: br...@python.org To: rahulg...@live.ca CC: python-dev@python.org On Tue, Feb 19, 2013 at 12:37 PM, rahul garg rahulg...@live.ca wrote: Hi. I downloaded Python 3.3 source,

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread Oleg Broytman
On Wed, Feb 20, 2013 at 08:23:19AM +0100, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 19 Feb 2013 20:37:36 -0800 Eli Bendersky eli...@gmail.com wrote: On Tue, Feb 19, 2013 at 10:50 AM, Oleg Broytman p...@phdru.name wrote: Oleg, lately I have the feeling you're getting too automatic

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread M.-A. Lemburg
On 20.02.2013 03:37, Paul Moore wrote: On 20 February 2013 00:54, Fred Drake f...@fdrake.net wrote: I'd posit that anything successful will no longer need to be added to the standard library, to boot. Packaging hasn't done well there. distlib may be the exception, though. Packaging tools

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread Antoine Pitrou
Le Wed, 20 Feb 2013 13:09:13 +0400, Oleg Broytman p...@phdru.name a écrit : On Wed, Feb 20, 2013 at 08:23:19AM +0100, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 19 Feb 2013 20:37:36 -0800 Eli Bendersky eli...@gmail.com wrote: On Tue, Feb 19, 2013 at 10:50 AM, Oleg Broytman

Re: [Python-Dev] PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Antoine Pitrou
Le Tue, 19 Feb 2013 19:54:21 -0500, Fred Drake f...@fdrake.net a écrit : On Tue, Feb 19, 2013 at 6:19 PM, Donald Stufft donald.stu...@gmail.com wrote: Let's not add anything to the stdlib till it has real world usage. Doing otherwise is putting the cart before the horse. I'd posit that

Re: [Python-Dev] Built with VS2012 Express for desktop

2013-02-20 Thread Oleg Broytman
On Wed, Feb 20, 2013 at 10:54:06AM +0100, Antoine Pitrou solip...@pitrou.net wrote: Le Wed, 20 Feb 2013 13:09:13 +0400, Oleg Broytman p...@phdru.name a ??crit : On Wed, Feb 20, 2013 at 08:23:19AM +0100, Antoine Pitrou solip...@pitrou.net wrote: On Tue, 19 Feb 2013 20:37:36 -0800 Eli

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread M.-A. Lemburg
On 20.02.2013 00:16, Daniel Holth wrote: On Tue, Feb 19, 2013 at 5:10 PM, M.-A. Lemburg m...@egenix.com wrote: On 19.02.2013 23:01, Daniel Holth wrote: On Tue, Feb 19, 2013 at 4:34 PM, M.-A. Lemburg m...@egenix.com wrote: On 19.02.2013 14:40, Nick Coghlan wrote: On Tue, Feb 19, 2013 at

[Python-Dev] The last Python 2.6 security release

2013-02-20 Thread Barry Warsaw
I just updated PEP 361 (Python 2.6 release schedule). Python 2.6 is in security maintenance, source only release mode. Official support for Python 2.6 expires on October 1 2013, and I would like to do one last release[1], i.e. 2.6.9 as close to that date as possible. I know of issue 16248, but

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 17:25, schrieb Benjamin Peterson: Are these going to become patches for Python, too? I'm working on it. The patches need to be discussed as they break backward compatibility and AFAIK XML standards, too. ___ Python-Dev mailing list

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Maciej Fijalkowski
On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes christ...@python.org wrote: Am 20.02.2013 17:25, schrieb Benjamin Peterson: Are these going to become patches for Python, too? I'm working on it. The patches need to be discussed as they break backward compatibility and AFAIK XML standards,

Re: [Python-Dev] [Python-checkins] cpython (3.3): Rebuild importlib.h after the changes introduced in 0f65bf6063ca.

2013-02-20 Thread Eric Snow
On Wed, Feb 20, 2013 at 1:16 PM, ezio.melotti python-check...@python.org wrote: http://hg.python.org/cpython/rev/9d00c79b27e1 changeset: 82280:9d00c79b27e1 branch: 3.3 parent: 82278:96b4acb253f8 user:Ezio Melotti ezio.melo...@gmail.com date:Wed Feb 20 21:42:46

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Skip Montanaro
I'm working on it. The patches need to be discussed as they break backward compatibility and AFAIK XML standards, too. That's not very good. XML parsers are supposed to parse XML according to standards. Is the goal to have them actually do that, or just address DDOS issues? Having read

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Carl Meyer
On 02/20/2013 01:53 PM, Skip Montanaro wrote: That's not very good. XML parsers are supposed to parse XML according to standards. Is the goal to have them actually do that, or just address DDOS issues? Having read through Christian's mail and several of his references, it seems to me that

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 21:17, schrieb Maciej Fijalkowski: On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes christ...@python.org wrote: Am 20.02.2013 17:25, schrieb Benjamin Peterson: Are these going to become patches for Python, too? I'm working on it. The patches need to be discussed as they break

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 22:02, schrieb Carl Meyer: Also, despite the title of this thread, the vulnerabilities include fetching of external DTDs and entities (per standard), which opens up attacks that are worse than just denial-of-service. In our initial Django release advisory we carelessly lumped the

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Greg Ewing
Carl Meyer wrote: An XML parser that follows the XML standard is never safe to expose to untrusted input. Does the XML standard really mandate that a conforming parser must blindly download any DTD URL given to it from the real live internet? Somehow I doubt that. -- Greg

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread R. David Murray
On Thu, 21 Feb 2013 11:35:23 +1300, Greg Ewing greg.ew...@canterbury.ac.nz wrote: Carl Meyer wrote: An XML parser that follows the XML standard is never safe to expose to untrusted input. Does the XML standard really mandate that a conforming parser must blindly download any DTD URL

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Fred Drake
On Wed, Feb 20, 2013 at 5:45 PM, R. David Murray rdmur...@bitdance.com wrote: (Wikipedia says: Programs for reading documents may not be required to read the external subset., which would seem to confirm that.) Validating parsers are required to read the external subset; this doesn't apply to

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Carl Meyer
On 02/20/2013 03:35 PM, Greg Ewing wrote: Carl Meyer wrote: An XML parser that follows the XML standard is never safe to expose to untrusted input. Does the XML standard really mandate that a conforming parser must blindly download any DTD URL given to it from the real live internet?

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 23:45, schrieb R. David Murray: I don't believe it does. The DTD URL is, if I remember correctly, specified as an identifier. The fact that you can often also download the DTD from the location specified by the identifier is a secondary effect. But, it's been a *long* time

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Wed, 20 Feb 2013 22:55:57 +0100 Christian Heimes christ...@python.org wrote: Am 20.02.2013 21:17, schrieb Maciej Fijalkowski: On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes christ...@python.org wrote: Am 20.02.2013 17:25, schrieb Benjamin Peterson: Are these going to become patches

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Donald Stufft
On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A single 1 kB XML document can kill virtually any machine, even servers with more than hundred GB RAM. Assuming an attacker can inject arbitrary XML.

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 20.02.2013 23:56, schrieb Fred Drake: While I'd hate to make XML processing more painful than it often is, there's no injunction not to be reasonable. Security concerns and resource limits are cross-cutting concerns, so it's not wrong to provide safe defaults. Doing so *will* be backward

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Christian Heimes
Am 21.02.2013 00:08, schrieb Antoine Pitrou: Not everyone is a security nuts. But, but, but ... it's fun to be paranoid! You get so many new potential enemies. :) Jerry Fletcher ___ Python-Dev mailing list Python-Dev@python.org

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft donald.stu...@gmail.com wrote: On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A single 1 kB XML document can kill virtually any machine, even servers

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread PJ Eby
On Wed, Feb 20, 2013 at 5:30 AM, M.-A. Lemburg m...@egenix.com wrote: The wording in the PEP alienates the egg format by defining an incompatible new standard for the location of the metadata file: This isn't a problem, because there's not really a use case at the moment for eggs to include a

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Donald Stufft
On Wednesday, February 20, 2013 at 6:23 PM, Christian Heimes wrote: We can add a function to the XML package tree that enables all restrictions: * limit expansion depths of nested entities * limit total amount of expanded chars * disable external entity expansion * optionally force expat to

Re: [Python-Dev] [Distutils] PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread PJ Eby
On Tue, Feb 19, 2013 at 6:42 AM, Nick Coghlan ncogh...@gmail.com wrote: Nothing in the PEP is particularly original - almost all of it is either stolen from other build and packaging systems, or is designed to provide a *discoverable* alternative to existing setuptools/distribute/pip practices

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Jesse Noller
On Feb 20, 2013, at 6:22 PM, Antoine Pitrou solip...@pitrou.net wrote: On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft donald.stu...@gmail.com wrote: On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2013 06:22 PM, Antoine Pitrou wrote: On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft donald.stu...@gmail.com wrote: On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Donald Stufft
On Wednesday, February 20, 2013 at 6:22 PM, Antoine Pitrou wrote: On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufft donald.stu...@gmail.com (mailto:donald.stu...@gmail.com) wrote: On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a

[Python-Dev] Postponing acceptance of PEP 426

2013-02-20 Thread Nick Coghlan
The feedback I have received (both on-list and in response to some off-list queries to specific people) tells me that PEP 426 isn't quite ready for acceptance yet. Things I'll be working on or facilitating over the next few weeks: - documenting an overall transition plan to put the new metadata

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Steven D'Aprano
On 21/02/13 10:22, Antoine Pitrou wrote: On Wed, 20 Feb 2013 18:21:22 -0500 Donald Stufftdonald.stu...@gmail.com wrote: On Wednesday, February 20, 2013 at 6:08 PM, Antoine Pitrou wrote: It's not a distributed DoS issue, it's a severe DoS vulnerabilities. A single 1 kB XML document can kill

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Nick Coghlan
On Thu, Feb 21, 2013 at 9:49 AM, Tres Seaver tsea...@palladion.com wrote: Two words: hash randomization. If it applies to one, it applies to the other. Agreed. Christian's suggested approach sounds sane to me: - make it possible to enable safer behaviour globally in at least 2.7 and 3.3 (and

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Vinay Sajip
M.-A. Lemburg mal at egenix.com writes: The suggestion to have the metadata available on PyPI doesn't have anything to do with security. It's about being able to determine compatibility and select the right distribution file for download. The metadata also helps in creating dependency

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Vinay Sajip
Chris Jerdonek chris.jerdonek at gmail.com writes: Maybe this is already stated somewhere, but is there a plan for when distlib will be brought into the repository? Is there a reason not to do it now? It seems it would have more visibility that way (e.g. people could see it as part of the

Re: [Python-Dev] Fwd: PEP 426 is now the draft spec for distribution metadata 2.0

2013-02-20 Thread Vinay Sajip
Paul Moore p.f.moore at gmail.com writes: Understood - that's why I suggested that distlib reach a point where it's stable as an external package and supported on (some) older versions. I'm hoping for an experience more like unittest2 than packaging/distutils2. Currently, distlib runs on

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Fred Drake
On Wed, Feb 20, 2013 at 7:38 PM, Nick Coghlan ncogh...@gmail.com wrote: Christian's suggested approach sounds sane to me: Definitely. A strong +1 from me, FWIW these days. -Fred -- Fred L. Drake, Jr.fred at fdrake.net A storm broke loose in my mind. --Albert Einstein

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Barry Warsaw
On Feb 21, 2013, at 10:38 AM, Nick Coghlan wrote: - make it possible to enable safer behaviour globally in at least 2.7 and 3.3 (and perhaps in 2.6 and 3.2 security releases as well) I want to be fairly conservative with 2.6.9. -Barry ___ Python-Dev

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2013 09:08 PM, Barry Warsaw wrote: On Feb 21, 2013, at 10:38 AM, Nick Coghlan wrote: - make it possible to enable safer behaviour globally in at least 2.7 and 3.3 (and perhaps in 2.6 and 3.2 security releases as well) I want to be

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Barry Warsaw
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Feb 20, 2013, at 11:35 PM, Tres Seaver wrote: I believe that the same rationale should apply as that for adding hash randomization in 2.6.8: this is at least as bad a vulnerability, with many more vectors of attack. Except that I really want

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Stefan Behnel
Maciej Fijalkowski, 20.02.2013 21:17: On Wed, Feb 20, 2013 at 8:24 PM, Christian Heimes wrote: Am 20.02.2013 17:25, schrieb Benjamin Peterson: Are these going to become patches for Python, too? I'm working on it. The patches need to be discussed as they break backward compatibility and AFAIK

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Wed, 20 Feb 2013 18:45:10 -0500 Donald Stufft donald.stu...@gmail.com wrote: No software you run on your computer grabs data from someone you don't trust and it all validates that even though you trust them they haven't been exploited? What the hell do you mean exactly? There are other

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Thu, 21 Feb 2013 11:37:47 +1100 Steven D'Aprano st...@pearwood.info wrote: It's easy to forget that malware existed long before the Internet. The internet is just a transmission vector, it is not the source of malicious files. The source of malicious files is *other people*, and unless

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Thu, 21 Feb 2013 10:38:07 +1000 Nick Coghlan ncogh...@gmail.com wrote: On Thu, Feb 21, 2013 at 9:49 AM, Tres Seaver tsea...@palladion.com wrote: Two words: hash randomization. If it applies to one, it applies to the other. Agreed. Christian's suggested approach sounds sane to me: -

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/21/2013 01:53 AM, Antoine Pitrou wrote: On Thu, 21 Feb 2013 11:37:47 +1100 Steven D'Aprano st...@pearwood.info wrote: It's easy to forget that malware existed long before the Internet. The internet is just a transmission vector, it is not

Re: [Python-Dev] XML DoS vulnerabilities and exploits in Python

2013-02-20 Thread Antoine Pitrou
On Thu, 21 Feb 2013 02:29:08 -0500 Tres Seaver tsea...@palladion.com wrote: Antoine, A single, small,, malicious XML file can kill a machine (not just the process parsing it) by sucking all available RAM. We are talking hard lockup, reboot-to-fix-it sorts of DOC here. Sure, but in many