Re: [Python-Dev] Python and the Unicode Character Database
Antoine Pitrou writes: Le vendredi 03 décembre 2010 à 13:58 +0900, Stephen J. Turnbull a écrit : Antoine Pitrou writes: The legacy format argument looks like a red herring to me. When converting from a format to another it is the programmer's job to his/her job right. Uhmm, the argument *for* this feature proposed by several people is that Python's numeric constructors do it (right) so that the programmer doesn't have to. As far as I understand, Alexander was talking about a legacy pre-unicode text format. We don't have to support this. *I* didn't say we *should* support it. I'm saying that *others'* argument for not restricting the formats accepting by string to number converters to something well-defined and AFAIK universally understood by users (developers of Python programs *and* end-users) is that we *already* support this. Alexander, Martin, and I are basically just pointing out that no, the support we have via the built-in numeric constructors is incomplete and nonconforming. We feel that is a bug to be fixed by (1) implementing the definition as currently found in the documents, and (2) moving the non-ASCII support to another module (or, as a compromise, supporting non-ASCII digits via an argument to the built-ins -- that was my proposal, I don't know if Alexander or Martin would find it acceptable). Given that some committers (MAL, you?) don't even consider that accepting and converting a string containing digits from multiple scripts as a single number is a bug, I'd really rather that this bug/feature not be embedded in the interpreter. I suppose that as a built-in rather than syntax, technically it doesn't fall under the moratorium, but it makes me nervous ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transform() and untransform() methods, and the codec registry
Alexander Belopolsky writes: In fact, once the language moratorium is over, I will argue that str.encode() and byte.decode() should deprecate encoding argument and just do UTF-8 encoding/decoding. Hopefully by that time most people will forget that other encodings exist. (I can dream, right?) It's just a dream. There's a pile of archival material, often on R/O media, out there that won't be transcoded any more quickly than the inscriptions on Tutankhamun's tomb. Remember, Python is a language used to implement such translations. It's not an application. I think it would be reasonable to make UTF-8 the *default* encoding on all platforms, except for internal OS functions, where Windows will presumably continue to use UTF-16 and *nix distros will probably continue to agree to disagree about whether on-disk format is NFD or NFC (and the Python language as yet doesn't know about NFC v. NFD, although the library does). In the discussion of PEP 263, I proposed that the external encoding of Python scripts themselves be fixed as UTF-8, and other encodings would have to be pretranslated by an appropriate codec. That was shouted down by the European contingent, who wanted to continue using Latin-1 and Latin-2 without codecs or a wrapper to call them transparently. However, this time around you might get a more sympathetic hearing. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
Martin v. Löwis, 02.12.2010 21:24: Since discussion has trailed off without any blocking objections, I'm accepting PEP 384. Martin, you may mark the PEP accepted and proceed with merging the implementation for the beta on Saturday. Thanks! will do (I'll also take into consideration the proposed changes). Note that the PEP states under Signature Changes: A number of functions currently expect a specific struct, even though callers typically have PyObject* available. These have been changed to expect PyObject* as the parameter; this will cause warnings in applications that currently explicitly cast to the parameter type. That's not true for C++ compilation where I now get an error - 3 failing tests in Cython's test suite tonight due to the new PyEval_EvalCode() signature. I'm not really complaining (the API fixes are long overdue), just leaving a comment that what a compiler considers a warning or error pretty much depends on compiler, platform and configuration. So the above statement can be considered euphemistic. Stefan ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
Am I still missing something? Apparently. The hole C API would break if objects would move in memory. Since they have to stay at fixed addresses, it's easy enough to use the address as ID. There is no problem to be solved here. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] python 2 for building python 3
To build the trunk Python 3k, it seems that Python 2 has to be installed. This is because Objects/typeslots.py is called in the Makefile, and this file uses Python 2 print statement syntax, which fails if $(PYTHON) is actually Python 3. Can anyone reproduce this? Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
I'm not really complaining (the API fixes are long overdue), just leaving a comment that what a compiler considers a warning or error pretty much depends on compiler, platform and configuration. No no no. It does *not* depend on compiler, platform, or configuration. It *only* depends on the programming language. So for C++, it's an API change; for C, it's not. Still, I can point that out. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
I'm afraid I don't follow you. Unless you're suggesting some sort of esoteric object system whereby objects *don't* have identity (e.g. where objects are emergent properties of some sort of distributed, non-localised information), any object naturally has an identity -- itself. Not in Java or C#. It is in these languages possible to determine whether to references refer to the same object. However, objects don't naturally have a distinct identification (be it an integer or something else). If you really want to associate unique numbers with objects in these languages, the common approach is to put them into an identity dictionary as keys. It seems counter-productive to me to bother with an identity function which doesn't meet that constraint. If id(x) == id(y) implies nothing about x and y (they may, or may not, be the same object) then what's the point? See James' explanation: it would be possible to use this as the foundation of an identity hash table. Why would you bother using that function when you could just use x == y instead? Because in a hash table, you also need a hash value. Regards, Martn ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
Am 04.12.2010 09:48, schrieb Eli Bendersky: To build the trunk Python 3k, it seems that Python 2 has to be installed. This is because Objects/typeslots.py is called in the Makefile, and this file uses Python 2 print statement syntax, which fails if $(PYTHON) is actually Python 3. This shouldn't be necessary, as typeslots.inc is also checked into subversion, and should have a newer time stamp than typeslots.inc (perhaps not currently, but that is the plan, anyway). In any case, I now made the script 2-vs-3-agnostic. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
On Dec 4, 2010, at 11:13, Martin v. Löwis mar...@v.loewis.de wrote: Am 04.12.2010 09:48, schrieb Eli Bendersky: To build the trunk Python 3k, it seems that Python 2 has to be installed. This is because Objects/typeslots.py is called in the Makefile, and this file uses Python 2 print statement syntax, which fails if $(PYTHON) is actually Python 3. This shouldn't be necessary, as typeslots.inc is also checked into subversion, and should have a newer time stamp than typeslots.inc (perhaps not currently, but that is the plan, anyway). In any case, I now made the script 2-vs-3-agnostic. Regards, Martin I had the error pop out in a real build, so the time stamp is probably not a 100% guarantee. Therefore I think it's good that you made it version agnostic. This should be probably be a requirement for all scripts that are part of the build. Thanks, Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
Martin v. Löwis wrote: I'm afraid I don't follow you. Unless you're suggesting some sort of esoteric object system whereby objects *don't* have identity (e.g. where objects are emergent properties of some sort of distributed, non-localised information), any object naturally has an identity -- itself. Not in Java or C#. It is in these languages possible to determine whether to references refer to the same object. However, objects don't naturally have a distinct identification (be it an integer or something else). Surely even in Java or C#, objects have an *identity* even if the language doesn't provide a way to query their distinct *identification*. An object stored at one memory location is distinct from any other object stored at different memory locations at that same moment in time, regardless of whether or not the language gives you a convenient label for that identity. Even if that memory location can change during the lifetime of the object, at any one moment, object X is a different object from every other object. The fact that we can even talk about this object versus that object implies that objects have identity. To put it in Python terms, if the id() function were removed, it would no longer be possible to get the unique identification associated with an object, but you could still compare the identity of two objects using `is`. Of course, I'm only talking about objects. In Java there are values which aren't objects, such as ints and floats. That's irrelevant for our discussion, because Python has no such values. If you really want to associate unique numbers with objects in these languages, the common approach is to put them into an identity dictionary as keys. It seems counter-productive to me to bother with an identity function which doesn't meet that constraint. If id(x) == id(y) implies nothing about x and y (they may, or may not, be the same object) then what's the point? See James' explanation: it would be possible to use this as the foundation of an identity hash table. I'm afraid James' explanation didn't shed any light on the question to me. It seems to me that Java's IdentityHashValue[sic -- I think the correct function name is actually IdentityHashCode] is equivalent to Python's hash(), not to Python's id(), and claiming it is related to identity is misleading and confusing. I don't think I'm alone here -- it seems to me that even among Java programmers, the same criticisms have been raised: http://bugs.sun.com/bugdatabase/view_bug.do?bug%5Fid=6321873 http://deepakjha.wordpress.com/2008/07/31/interesting-fact-about-identityhashcode-method-in-javalangsystem-class/ Like hash(), IdentityHashCode doesn't make any promises about identity at all. Two distinct objects could have the same hash code, and a further test is needed to distinguish them. Why would you bother using that function when you could just use x == y instead? Because in a hash table, you also need a hash value. Well, sure, in a hash table you need a hash value. But I was talking about an id() function. So is that it? Is IdentityHashValue (or *Code, as the case may be) just a longer name for hash()? -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python and the Unicode Character Database
Le samedi 04 décembre 2010 à 17:13 +0900, Stephen J. Turnbull a écrit : Antoine Pitrou writes: Le vendredi 03 décembre 2010 à 13:58 +0900, Stephen J. Turnbull a écrit : Antoine Pitrou writes: The legacy format argument looks like a red herring to me. When converting from a format to another it is the programmer's job to his/her job right. Uhmm, the argument *for* this feature proposed by several people is that Python's numeric constructors do it (right) so that the programmer doesn't have to. As far as I understand, Alexander was talking about a legacy pre-unicode text format. We don't have to support this. *I* didn't say we *should* support it. I'm saying that *others'* argument for not restricting the formats accepting by string to number converters to something well-defined and AFAIK universally understood by users (developers of Python programs *and* end-users) is that we *already* support this. As far as I can parse your sentence, I think you are mistaken. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Fri, 3 Dec 2010 18:11:57 -0500 James Y Knight f...@fuhm.net wrote: On Dec 3, 2010, at 5:52 PM, Martin v. Löwis wrote: Am 03.12.2010 23:48, schrieb Éric Araujo: But I'm not interested at all in having it in distutils2. I want the Python build itself to use it, and alas, I can't because of the freeze. You can’t in 3.2, true. Neither can you in 3.1, or any previous version. If you implement it in distutils2, you have very good chances to get it for 3.3. Isn’t that a win? It is, unfortunately, a very weak promise. Until distutils2 is integrated in Python, I probably won't spend any time on it. It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then. +1 from me. Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
On Sat, 4 Dec 2010 15:06:45 +1100 Cameron Simpson c...@zip.com.au wrote: On 03Dec2010 18:15, James Y Knight f...@fuhm.net wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | gc is implementation specific. CPython uses ref counting + cycle | gc. A constraint on all implementations is that objects have a fixed, | unique id during their lifetime. CPython uses the address as the id, so | it cannot move objects. Other implementations do differently. Compacting | gc requires an id to current address table or something. | | It's somewhat unfortuante that python has this constraint, instead of | the looser: objects have a fixed id during their lifetime, which is | much easier to implement, and practically as useful. Python doesn't have the constraint you imagine; it _does_ have objects have a fixed id during their lifetime. _CPython_ has this constraint because it uses the address as the id, which is free and needs no papping or extra storage. Well, the main reason CPython has this constraint is that the C API mandates constant object pointers. That we can then reuse the pointer for id() is only a consequence of this. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
On Sat, 4 Dec 2010 11:20:57 +0200 Eli Bendersky eli...@gmail.com wrote: I had the error pop out in a real build, so the time stamp is probably not a 100% guarantee. Therefore I think it's good that you made it version agnostic. This should be probably be a requirement for all scripts that are part of the build. Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). It would be better if this didn't change. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On Sat, 4 Dec 2010 10:10:44 +0100 (CET) gregory.p.smith python-check...@python.org wrote: Author: gregory.p.smith Date: Sat Dec 4 10:10:44 2010 New Revision: 87010 Log: issue7213 + issue2320: Cause a DeprecationWarning if the close_fds argument is not passed to subprocess.Popen as the default value will be changing in a future Python to the safer and more often desired value of True. That doesn't seem to be a good idea under Windows, is it? (“Note that on Windows, you cannot set *close_fds* to true and also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.”) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
Surely even in Java or C#, objects have an *identity* even if the language doesn't provide a way to query their distinct *identification*. When people said the id of an object should this or that, they always meant identification, not identity (perhaps except for you). Nobody would propose that the identity (in your wording) of two distinct objects might be the same, or might change over time. I'm afraid James' explanation didn't shed any light on the question to me. It seems to me that Java's IdentityHashValue[sic -- I think the correct function name is actually IdentityHashCode] is equivalent to Python's hash() No, it's not. java.lang.Object.hashCode() is equivalent to Python's hash(). In particular, for both of these, the following requirement holds: object that *compare* equal must hash equal. This is not the case for the identity hash code, or Python's id function: objects that compare equal may still have different ids, or different java identity hash codes. not to Python's id(), and claiming it is related to identity is misleading and confusing. No, it's right on spot. This is *not* about the regular value hash, but about the identity hash. Like hash(), IdentityHashCode doesn't make any promises about identity at all. It sure does: calling identityHashCode on the very same object twice will always return the same value - even after a garbage collection. So if two references refer to the same object, calling identityHashCode on both of them gives the same value. This is similar to the relationship between equals and hashCode. Two distinct objects could have the same hash code, and a further test is needed to distinguish them. Correct. However, references with different identity hash codes will never refer to the same object. Why would you bother using that function when you could just use x == y instead? Because in a hash table, you also need a hash value. Well, sure, in a hash table you need a hash value. But I was talking about an id() function. So is that it? Is IdentityHashValue (or *Code, as the case may be) just a longer name for hash()? Of course not. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
Steven D'Aprano writes: Martin v. Löwis wrote: It seems counter-productive to me to bother with an identity function which doesn't meet that constraint. If id(x) == id(y) implies nothing about x and y (they may, or may not, be the same object) then what's the point? See James' explanation: it would be possible to use this as the foundation of an identity hash table. I'm afraid James' explanation didn't shed any light on the question to me. It seems to me that Java's IdentityHashValue[sic -- I think the correct function name is actually IdentityHashCode] is equivalent to Python's hash(), not to Python's id(), and claiming it is related to identity is misleading and confusing. Not quite equivalent. Python's hash() obeys a certain additional constraint (that numeric values that compare equal have the same hash), and IdentityHashValue presumably won't work at all on numbers in Java (since they aren't objects). Bikeshedding aside, I have to agree with you: there's no point in calling such a function id() or IdentityAnything(). Everything you need to know (including both the fact that it could be an efficient way to search for a unique object, and the possible non-uniqueness of the object assigned that code) is contained in the word hash. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
On Sat, Dec 4, 2010 at 1:37 PM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 4 Dec 2010 15:06:45 +1100 Cameron Simpson c...@zip.com.au wrote: On 03Dec2010 18:15, James Y Knight f...@fuhm.net wrote: | On Dec 3, 2010, at 6:04 PM, Terry Reedy wrote: | gc is implementation specific. CPython uses ref counting + cycle | gc. A constraint on all implementations is that objects have a fixed, | unique id during their lifetime. CPython uses the address as the id, so | it cannot move objects. Other implementations do differently. Compacting | gc requires an id to current address table or something. | | It's somewhat unfortuante that python has this constraint, instead of | the looser: objects have a fixed id during their lifetime, which is | much easier to implement, and practically as useful. Python doesn't have the constraint you imagine; it _does_ have objects have a fixed id during their lifetime. _CPython_ has this constraint because it uses the address as the id, which is free and needs no papping or extra storage. Well, the main reason CPython has this constraint is that the C API mandates constant object pointers. That we can then reuse the pointer for id() is only a consequence of this. Regards Antoine. Yes. I think further discussion of Java-vs-Python makes no sense (and it's proven that C-API-less python can exist and be compatible) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou solip...@pitrou.net wrote: Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). Are you sure about this? I remember needing an existing Python to building Python 2.7 on a new python-less install of FreeBSD a couple of months ago. But maybe that was just an issue with timestamps on files. I'll see if I can reproduce. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
Le samedi 04 décembre 2010 à 13:23 +, Mark Dickinson a écrit : On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou solip...@pitrou.net wrote: Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). Are you sure about this? I remember needing an existing Python to building Python 2.7 on a new python-less install of FreeBSD a couple of months ago. I remember compiling a Python 2.6 (2.5?) on a machine with no Python installed. But that was long ago. (of course, this is difficult to check on a standard development machine :-)) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson dicki...@gmail.com wrote: On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou solip...@pitrou.net wrote: Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). Are you sure about this? I remember needing an existing Python to building Python 2.7 on a new python-less install of FreeBSD a couple of months ago. But maybe that was just an issue with timestamps on files. I'll see if I can reproduce. With a fresh checkout of the release27-maint branch on an Ubuntu 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a './configure make' fails with: [...] checking for socklen_t... yes checking for build directories... done configure: creating ./config.status config.status: creating Makefile.pre config.status: creating Modules/Setup.config config.status: creating Misc/python.pc config.status: creating Modules/ld_so_aix config.status: creating pyconfig.h creating Modules/Setup creating Modules/Setup.local creating Makefile ./Parser/asdl_c.py -h ./Include ./Parser/Python.asdl /usr/bin/env: python: No such file or directory make: *** [Include/Python-ast.h] Error 127 I think this is the same problem that I saw on the FreeBSD VM. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
2010/12/4 Mark Dickinson dicki...@gmail.com: On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson dicki...@gmail.com wrote: On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou solip...@pitrou.net wrote: Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). Are you sure about this? I remember needing an existing Python to building Python 2.7 on a new python-less install of FreeBSD a couple of months ago. But maybe that was just an issue with timestamps on files. I'll see if I can reproduce. With a fresh checkout of the release27-maint branch on an Ubuntu 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a './configure make' fails with: [...] checking for socklen_t... yes checking for build directories... done configure: creating ./config.status config.status: creating Makefile.pre config.status: creating Modules/Setup.config config.status: creating Misc/python.pc config.status: creating Modules/ld_so_aix config.status: creating pyconfig.h creating Modules/Setup creating Modules/Setup.local creating Makefile ./Parser/asdl_c.py -h ./Include ./Parser/Python.asdl /usr/bin/env: python: No such file or directory make: *** [Include/Python-ast.h] Error 127 I think this is the same problem that I saw on the FreeBSD VM. You have to touch Include/Python-ast.h and Python/Python-ast.c. We do this for release tarballs. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
Le samedi 04 décembre 2010 à 13:39 +, Mark Dickinson a écrit : On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson dicki...@gmail.com wrote: On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou solip...@pitrou.net wrote: Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). Are you sure about this? I remember needing an existing Python to building Python 2.7 on a new python-less install of FreeBSD a couple of months ago. But maybe that was just an issue with timestamps on files. I'll see if I can reproduce. With a fresh checkout of the release27-maint branch on an Ubuntu 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a './configure make' fails with: How about with the release tarball? Perhaps SVN doesn't get timestamps right. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
On Sat, Dec 4, 2010 at 1:40 PM, Benjamin Peterson benja...@python.org wrote: You have to touch Include/Python-ast.h and Python/Python-ast.c. We do this for release tarballs. Ah, that makes sense. Thanks. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
On Sat, Dec 4, 2010 at 15:41, Antoine Pitrou solip...@pitrou.net wrote: Le samedi 04 décembre 2010 à 13:39 +, Mark Dickinson a écrit : On Sat, Dec 4, 2010 at 1:23 PM, Mark Dickinson dicki...@gmail.com wrote: On Sat, Dec 4, 2010 at 11:41 AM, Antoine Pitrou solip...@pitrou.net wrote: Er, normally you don't need *any* Python installed to build Python (be it 3.x or 2.x). Are you sure about this? I remember needing an existing Python to building Python 2.7 on a new python-less install of FreeBSD a couple of months ago. But maybe that was just an issue with timestamps on files. I'll see if I can reproduce. With a fresh checkout of the release27-maint branch on an Ubuntu 64-bit VM, with /usr/bin/python renamed to /usr/bin/python_not_here, a './configure make' fails with: How about with the release tarball? Perhaps SVN doesn't get timestamps right. My original problem was that I was re-running 'make' on a svn py3k branch checkout, which already had a compiled ./python exe in its root (Python 3.2, of course). Since the script Objects/typeslots.py (which Martin checked in just yesterday) required Python 2, this failed, although python on my machine actually refers to 2.6.5. The failure then happened since in the root of the Python build, python referred to the local Python 3 executable. Relying on timestamps sounds a bit too brittle. I think it would be nice if: 1. Parts of the Makefile that use Python checked if Python is installed and generate a useful error if not. 2. All Python scripts that are part of the build should be 2-vs-3 version agnostic for the time being (= until Python 2 is distant history, which won't happen soon) Eli ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
Quotes are out of order. Martin v. Löwis writes: No, it's not. java.lang.Object.hashCode() is equivalent to Python's hash(). In particular, for both of these, the following requirement holds: object that *compare* equal must hash equal. Aha. I see, now. That is indeed a different concept. Mutable objects retain their identities across mutations. Two distinct objects could have the same hash code, and a further test is needed to distinguish them. Correct. However, references with different identity hash codes will never refer to the same object. Why is useful to expose an identity hash? AFAICS it is *only* useful in building an identity hash table. If so, why not just provide id() or the is operator or both and be done with it? This is different from exposing the value hash AFAICS, where it may be possible to optimize value hashing of composite objects by computing their hashes from the hashes of subobjects, perhaps omitting some subobjects that don't affect equality (eg, Emacs strings may carry display properties that are ignored when they are compared as strings). With identity, I don't see much point in leaving that up to the programmer. In implementations with stable object addresses, the address is a fine implementation of identity_hash() -- perfect, in fact. Otherwise, you either have to omit mutable members from the computation or store the hashcode in the object. In the former situation, you will likely get very coarse hashing, and the set of immutable members (if any ;-) can be computed by the compiler. In the latter, you may as well store an id directly and avoid the whole hash concept for that type (except as a space optimization, but that again implies coarse hashing). [F]or the identity hash code, or Python's id function: objects that compare equal may still have different ids, or different java identity hash codes. ISTM that's no problem, you have to do an extra comparison for identity when the codes are equal either way. Of course the identity hash code is a more precise optimization, but that doesn't make hash() unusable for this purpose (cf Sun's built-in implementation of IdentityHashCode that always returns 2). The problem presumably lies in the fact that there are id()-ifiable objects that, because they are mutable, either are unhashable (Python hash) or would have an unstable value hash (Lisp sxhash). Ie, AFAICS def identity_hash (thing): Wrapper for hash() suitable for building an identity hash table. try: return hash(thing) except TypeError: return 0 would be a fine implementation for Python.wink Like hash(), IdentityHashCode doesn't make any promises about identity at all. It sure does: calling identityHashCode on the very same object twice will always return the same value - even after a garbage collection. Well, if you want to put it that way, so does hash(). I read Steven as placing more emphasis on like hash() than on at all. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Fri, Dec 03, 2010 at 11:52:41PM +0100, Martin v. Löwis wrote: Am 03.12.2010 23:48, schrieb Éric Araujo: But I'm not interested at all in having it in distutils2. I want the Python build itself to use it, and alas, I can't because of the freeze. You can’t in 3.2, true. Neither can you in 3.1, or any previous version. If you implement it in distutils2, you have very good chances to get it for 3.3. Isn’t that a win? It is, unfortunately, a very weak promise. Until distutils2 is integrated in Python, I probably won't spend any time on it. At the language summit it was proposed and seemed generally accepted (maybe I took silence as consent... it's been almost a year now) that bold new modules (and bold rewrites of existing modules since it fell out of the distutils/2 discussion) should get implemented in a module on pypi before being merged into the python stdlib. If you wouldn't want to work on any of those modules until they were actually integrated into Python, it sounds like you disagree with that as a general practice? -Toshio pgpBIM4lN9FET.pgp Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
On Sat, Dec 4, 2010 at 12:27 PM, Antoine Pitrou solip...@pitrou.net wrote: On Fri, 3 Dec 2010 18:11:57 -0500 James Y Knight f...@fuhm.net wrote: On Dec 3, 2010, at 5:52 PM, Martin v. Löwis wrote: Am 03.12.2010 23:48, schrieb Éric Araujo: But I'm not interested at all in having it in distutils2. I want the Python build itself to use it, and alas, I can't because of the freeze. You can’t in 3.2, true. Neither can you in 3.1, or any previous version. If you implement it in distutils2, you have very good chances to get it for 3.3. Isn’t that a win? It is, unfortunately, a very weak promise. Until distutils2 is integrated in Python, I probably won't spend any time on it. It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then. +1 from me. +1 too. Just to clarify my position in a few sentences: - I was told not to progressively change distutils - the PEPs for my changes were accepted under the condition that the changes would be made in distutils2 - PEP 384 is yet another accepted PEP, why the rule would change - because it's a small feature ? - I have tons of small features I had to revert and push back in distutils2 If Martin changes make it into Distutils1, I am going to ask that a collection of small features that I have should go there too, because they should not break things Regards Tarek -- Tarek Ziadé | http://ziade.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
Le 04/12/2010 17:55, Tarek Ziadé a écrit : On Sat, Dec 4, 2010 at 12:27 PM, Antoine Pitrou solip...@pitrou.net wrote: It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then. +1 from me. +1 too. +1, and I take responsibility to propose a merge plan to python-dev after 3.2 is released and carry it out. Regards ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 4 Dec 2010 10:10:44 +0100 (CET) gregory.p.smith python-check...@python.org wrote: Author: gregory.p.smith Date: Sat Dec 4 10:10:44 2010 New Revision: 87010 Log: issue7213 + issue2320: Cause a DeprecationWarning if the close_fds argument is not passed to subprocess.Popen as the default value will be changing in a future Python to the safer and more often desired value of True. That doesn't seem to be a good idea under Windows, is it? (“Note that on Windows, you cannot set *close_fds* to true and also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.”) Regardless of what platform you are on I think the warning makes sense, it was just worded too strongly. It is asking people to be explicit with close_fds. Explicitly setting close_fds=False when that is desired is good. I modified the docs and warning message to just say that the default close_fds behavior will change in the future without specifying exactly what it will be. It could make sense for the default to be a soft-True on windows that changes behavior if any of the std handles are set if that matches what users expect and find easiest. We may want to reconsider its entire future in the face of the new pass_fds parameter, etc. Those are 3.3 decisions. -gps ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On 4 December 2010 18:14, Gregory P. Smith g...@krypto.org wrote: On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 4 Dec 2010 10:10:44 +0100 (CET) gregory.p.smith python-check...@python.org wrote: Author: gregory.p.smith Date: Sat Dec 4 10:10:44 2010 New Revision: 87010 Log: issue7213 + issue2320: Cause a DeprecationWarning if the close_fds argument is not passed to subprocess.Popen as the default value will be changing in a future Python to the safer and more often desired value of True. That doesn't seem to be a good idea under Windows, is it? (“Note that on Windows, you cannot set *close_fds* to true and also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.”) Regardless of what platform you are on I think the warning makes sense, it was just worded too strongly. It is asking people to be explicit with close_fds. Explicitly setting close_fds=False when that is desired is good. I modified the docs and warning message to just say that the default close_fds behavior will change in the future without specifying exactly what it will be. It could make sense for the default to be a soft-True on windows that changes behavior if any of the std handles are set if that matches what users expect and find easiest. We may want to reconsider its entire future in the face of the new pass_fds parameter, etc. Those are 3.3 decisions. This sounds like omitting the close_fds parameter is now considered ill-advised, if not outright wrong. That's silly - I certainly never specify close_fds, expecting the module to do the correct thing if I don't know/care enough to say. I use Popen as a convenience function, so ignoring low-level details is the whole point in my opinion (and close_fds *is* a low-level detail for what I do, on Windows). At the very least, the whole of the section Replacing Older Functions with the subprocess Module (with a couple of small exceptions) is in need of updating, as it is recommending bad practice (not specifying close_fds) at the moment... OK, I'm exaggerating a touch here. But can someone point me at the discussion of this change? Or if there hasn't been one, can we have one (i.e., to start the ball rolling, can someone justify the change, please). Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] gc ideas -- sparse memory
Why is useful to expose an identity hash? AFAICS it is *only* useful in building an identity hash table. If so, why not just provide id() or the is operator or both and be done with it? That's precisely James' point: Java provides the identity hash *instead* of the id() function (i.e. it does not have an equivalent of id()). Doing so gives greater liberties in implementing Java. For example, an implementation that would only use the type and not the instance for identity hash would still be conforming (as would one that always returns 0). With identity, I don't see much point in leaving that up to the programmer. In implementations with stable object addresses, the address is a fine implementation of identity_hash() -- perfect, in fact. Of course. James' complaint is that Python-the-language mandates support for an id() function, though - a requirement that even implementations that don't have stable object addresses now must support. If Python mandated only identity hash properties of the id() function, alternative implementations could be simplified. ISTM that's no problem, you have to do an extra comparison for identity when the codes are equal either way. Of course the identity hash code is a more precise optimization, but that doesn't make hash() unusable for this purpose (cf Sun's built-in implementation of IdentityHashCode that always returns 2). That's only the case if the hash() result is guaranteed not to change. In some applications, it may be desirable to have that as an absolute guarantee (rather than just being a convention). No, you can't substitute identity hash with hash: the value hash of an object may change over time, whereas the identity hash must not. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 384 accepted
At the language summit it was proposed and seemed generally accepted (maybe I took silence as consent... it's been almost a year now) that bold new modules (and bold rewrites of existing modules since it fell out of the distutils/2 discussion) should get implemented in a module on pypi before being merged into the python stdlib. If you wouldn't want to work on any of those modules until they were actually integrated into Python, it sounds like you disagree with that as a general practice? No - I just don't feel responsible for projects outside of CPython (unless I have an interest in the project as a user, i.e. to scratch my own itches). I have no use for a distutils2 package whatsoever - to install Python packages, I would either use the builtin distutils (preferred), or setuptools, or distribute (if the package author requires setuptools). I agree that new functionality should stand on its own merits first. And if it fails that test (i.e. it doesn't get much attraction from users when stand-alone), it shouldn't be included into Python at all. Now, distutils2 is different: it's *not* new functionality. So perhaps yes: I disagree with the principle that bold rewrites should be developed independently. Such work would be much better carried out in a branch - it will never stand on its own. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] python 2 for building python 3
1. Parts of the Makefile that use Python checked if Python is installed and generate a useful error if not. 2. All Python scripts that are part of the build should be 2-vs-3 version agnostic for the time being (= until Python 2 is distant history, which won't happen soon) I think this is asked too much, unless you want to also contribute code for that (which I really consider unnecessary). Users who build from the source repository just need to get their build environment right. It's unfortunate that subversion doesn't provide relative ordering of files after an update, perhaps Mercurial is better with that. For a release, the time stamps in the tar file will do fine. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On Sat, Dec 4, 2010 at 11:28 AM, Paul Moore p.f.mo...@gmail.com wrote: On 4 December 2010 18:14, Gregory P. Smith g...@krypto.org wrote: On Sat, Dec 4, 2010 at 3:45 AM, Antoine Pitrou solip...@pitrou.net wrote: On Sat, 4 Dec 2010 10:10:44 +0100 (CET) gregory.p.smith python-check...@python.org wrote: Author: gregory.p.smith Date: Sat Dec 4 10:10:44 2010 New Revision: 87010 Log: issue7213 + issue2320: Cause a DeprecationWarning if the close_fds argument is not passed to subprocess.Popen as the default value will be changing in a future Python to the safer and more often desired value of True. That doesn't seem to be a good idea under Windows, is it? (“Note that on Windows, you cannot set *close_fds* to true and also redirect the standard handles by setting *stdin*, *stdout* or *stderr*.”) Regardless of what platform you are on I think the warning makes sense, it was just worded too strongly. It is asking people to be explicit with close_fds. Explicitly setting close_fds=False when that is desired is good. I modified the docs and warning message to just say that the default close_fds behavior will change in the future without specifying exactly what it will be. It could make sense for the default to be a soft-True on windows that changes behavior if any of the std handles are set if that matches what users expect and find easiest. We may want to reconsider its entire future in the face of the new pass_fds parameter, etc. Those are 3.3 decisions. This sounds like omitting the close_fds parameter is now considered ill-advised, if not outright wrong. That's silly - I certainly never specify close_fds, expecting the module to do the correct thing if I don't know/care enough to say. I use Popen as a convenience function, so ignoring low-level details is the whole point in my opinion (and close_fds *is* a low-level detail for what I do, on Windows). At the very least, the whole of the section Replacing Older Functions with the subprocess Module (with a couple of small exceptions) is in need of updating, as it is recommending bad practice (not specifying close_fds) at the moment... OK, I'm exaggerating a touch here. But can someone point me at the discussion of this change? Or if there hasn't been one, can we have one (i.e., to start the ball rolling, can someone justify the change, please). Paul. Making the change was intended to force the discussion. I'm glad that worked. :) I don't like the thought of requiring people to specify close_fds either but the default is a bad one (see http://bugs.python.org/issue7213 and http://bugs.python.org/issue2320) so we should change it. The real question is how should we go about doing the change? This warning asking people to always specify close_fds=True is heavy handed. Other places in the stdlib and docs no doubt still need to be updated to reflect it, etc. Options that seem worthy of debate: A) The DeprecationWarning that exists in py3k as of today. B) Remove the DeprecationWarning, simply document that people should be explicit about it if they care at all and that the default will change in 3.3. C) Never change close_fds behavior. we're stuck with it for life. D) Deprecate close_fds so that it becomes a legacy no-op. the new pass_fds flag could evolve into this. this will break existing code that depends on close_fds one way or another. I'm in 100% agreement that forcing people to pass close_fds in makes the API less friendly (granted, people already find it unfriendly but why make it worse?). Option B seems the most friendly to me. Option D is always on the table but I haven't planned out what a future without it should look like. I prefer requiring people who need open file descriptors to pass them in; a semaphore for all fds could be created and pass_fds=ALL becomes the new close_fds=False (I think one of the bugs suggests this). -gps ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python and the Unicode Character Database
On Fri, Dec 3, 2010 at 12:10 AM, Alexander Belopolsky
alexander.belopol...@gmail.com wrote:
..
I don't think decimal module should support non-European decimal
digits. The only place where it can make some sense is in int()
because here we have a fighting chance of producing a reasonable
definition. The motivating use case is conversion of numerical data
extracted from text using simple '\d+' regex matches.
It turns out, this use case does not quite work in Python either:
re.compile(r'\s+(\d+)\s+').match(' \u2081\u2082\u2083 ').group(1)
'₁₂₃'
int(_)
Traceback (most recent call last):
File stdin, line 1, in module
UnicodeEncodeError: 'decimal' codec can't encode character '\u2081' in
position 0: invalid decimal Unicode string
This may actually be a bug in Python regex implementation because
Unicode standard seems to recommend that '\d' be interpreted as gc =
Decimal_Number (Nd):
http://unicode.org/reports/tr18/#Compatibility_Properties
I actually wonder if Python's re module can claim to provide even
Basic Unicode Support.
http://unicode.org/reports/tr18/#Basic_Unicode_Support
Here is how I would do it:
1. String x of non-European decimal digits is only accepted in
int(x), but not by int(x, 0) or int(x, 10).
2. If x contains one or more non-European digits, then
(a) all digits must be from the same block:
def basepoint(c):
return ord(c) - unicodedata.digit(c)
all(basepoint(c) == basepoint(x[0]) for c in x) - True
(b) and '+' or '-' sign is not alowed.
3. A character c is a digit if it matches '\d' regex. I think this
means unicodedata.category(c) - 'Nd'.
Condition 2(b) is important because there is no clear way to define
what is acceptable as '+' or '-' using Unicode character properties
and not all number systems even support local form of negation. (It
is also YAGNI.)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
DeprecationWarnings that show up in a lot of existing code are controversial and have caused pain in the past. I'd like to leave this on for 3.2 beta1 and see how things go. We can remove the warning if it is deemed too noisy during any betas. (case study: the md5 and sha module DeprecationWarnings are loathed around the world as those modules were never going to be removed in 2.x and 2to3 has a fixer for code that uses them) My understanding is that DeprecationWarnings are now off by default in part because of issues like the above. So they should only be seen by programmers who turn them of for the purpose of seeing them so they can future-proof their code before it fails. As a result, there have been several added recently in efforts to clean up the APis. The real issue is changing the default. If allowed, should it be switched in one release or should there be an intermediate release with no default. Which are you planning? My feeling is that we should be able to correct bad-enough mistakes. Code can easily be made portable across all 3.x versions by given an explicit value and not depending on defaults. I think there should be an intermediate version to cause overt failure of code that is not upgraded, because the programmer neglected either to turn DeprecationWarnings on for test runs or to read the News releases. Terry J. Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python and the Unicode Character Database
I actually wonder if Python's re module can claim to provide even Basic Unicode Support. Do you really wonder? Most definitely it does not. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On 4 December 2010 20:13, Gregory P. Smith g...@krypto.org wrote: This sounds like omitting the close_fds parameter is now considered ill-advised, if not outright wrong. [...] Making the change was intended to force the discussion. I'm glad that worked. :) :-) I don't like the thought of requiring people to specify close_fds either but the default is a bad one (see http://bugs.python.org/issue7213 and http://bugs.python.org/issue2320) so we should change it. Both of these issues seem to be for Unix (based on the fact that they use grep and cat as executables - I haven't dug very deeply into this). I work on Windows only, so I'm happy to take the experts' opinions on that OS. Is there an issue on Windows? If not, and given how different FD inheritance is on Windows, I'd argue that in the absence of bug reports, there's no need to change behaviour on Windows. The real question is how should we go about doing the change? This warning asking people to always specify close_fds=True is heavy handed. Other places in the stdlib and docs no doubt still need to be updated to reflect it, etc. Options that seem worthy of debate: A) The DeprecationWarning that exists in py3k as of today. Given that the option appears not to be getting deprecated (just the default changed), this seems wrong. I know DeprecationWarnings are off by default. If someone switches them on, they'd expect to have to address them. But what should they do in this case? Unless they have thought hard about FD inheritance, and then explicitly decided to use the default because it matches what they want (as opposed to specifying it explicitly, and documenting their intent clearly), nothing. Or they use the default expecting it to be good enough, and suddenly have to worry if they are relying on something that's changing (I have no feel for how likely it is that they'll be affected by the change, other than to say that personally, none of my code would be). That seems to me like a bad use of a DeprecationWarning, as a non-trivial amount of the time there's nothing to do. B) Remove the DeprecationWarning, simply document that people should be explicit about it if they care at all and that the default will change in 3.3. That seems reasonable to me. But I can see that the backward compatibility rules might apply here. On the other hand, given the bug reports, surely this is a bug fix? C) Never change close_fds behavior. we're stuck with it for life. Sounds harsh. But look at it another way - if compatibility rules mean that we can't change the default, the reported bugs need to be fixed another way. Or behaviour needs to be documented more explicitly so that the bugs can be downgraded to user error caused by not reading the (new, improved) documentation. Compatibility rules are to protect working programs. What proportion of programs using the default are working and what proportion are wrong? It seems to me that most are working, so compatibility applies. D) Deprecate close_fds so that it becomes a legacy no-op. the new pass_fds flag could evolve into this. this will break existing code that depends on close_fds one way or another. This works for me, as I never specify close_fds. I'm in 100% agreement that forcing people to pass close_fds in makes the API less friendly (granted, people already find it unfriendly but why make it worse?). I'd say making the parameter mandatory is highly unfriendly - to the point of making the API feel like an advanced one, not suitable for simple cases like replacing os.system. Option B seems the most friendly to me. Option D is always on the table but I haven't planned out what a future without it should look like. I prefer requiring people who need open file descriptors to pass them in; a semaphore for all fds could be created and pass_fds=ALL becomes the new close_fds=False (I think one of the bugs suggests this). In my view: 1. Don't change Windows behaviour unless there's a demonstrable issue. The Popen code is split into 2 distinct paths internally anyway, so that shouldn't be a problem. If documenting the behaviour becomes awkward should Unix behaviour change and Windows doesn't, that indicates to me that there's a problem with the behaviour (if it's hard to explain...). 2. Users who don't specify close_fds because they don't care (as opposed to because they explicitly want the current default, but choose not to say so - I appreciate that this is a fine distinction, difficult to police in practice) should see no change in behaviour, apart from identified bugs being fixed. If you're not hitting a bug, you should see no change at all. Keep the interface straightforward for people who don't know (or don't want to worry about) the subtleties of FD inheritance. The original goal was for subprocess to replace os.system, os.popen, os.spawn, etc. That's never quite happened because subprocess is just a little bit too conceptually complex for those basic tasks.
Re: [Python-Dev] PEP 384 accepted
On 04/12/2010 11:27, Antoine Pitrou wrote: On Fri, 3 Dec 2010 18:11:57 -0500 James Y Knightf...@fuhm.net wrote: On Dec 3, 2010, at 5:52 PM, Martin v. Löwis wrote: Am 03.12.2010 23:48, schrieb Éric Araujo: But I'm not interested at all in having it in distutils2. I want the Python build itself to use it, and alas, I can't because of the freeze. You can’t in 3.2, true. Neither can you in 3.1, or any previous version. If you implement it in distutils2, you have very good chances to get it for 3.3. Isn’t that a win? It is, unfortunately, a very weak promise. Until distutils2 is integrated in Python, I probably won't spend any time on it. It seems like it'd be a good idea to start integrating distutils2 into python trunk immediately after the 3.2 branch is cut, then. +1 from me. And from me. Michael Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On 4 December 2010 22:51, Terry Reedy tjre...@udel.edu wrote: My feeling is that we should be able to correct bad-enough mistakes. That's fair. Is this a bad-enough mistake? From a brief reading of the 2 bug reports, combined with my own trouble-free experience with Popen (always leaving close_fds as default), it doesn't seem so, to me. If someone provides evidence that this is a serious bug in the API, then that's a different matter. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On 4 December 2010 23:07, Paul Moore p.f.mo...@gmail.com wrote: Is there an issue on Windows? If not, and given how different FD inheritance is on Windows, I'd argue that in the absence of bug reports, there's no need to change behaviour on Windows. Actually, from the error message I just got: ValueError: close_fds is not supported on Windows platforms if you redirect stdin/stdout/stderr So on Windows, for the issues mentioned (both of which involve redirected stdin/out/err), leaving the current default appears to be not only advisable, but required. I suspect an os-dependent default would be ugly to document... Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r87010 - in python/branches/py3k: Doc/library/subprocess.rst Lib/subprocess.py Lib/test/test_subprocess.py
On 12/4/2010 3:07 PM, Paul Moore wrote: The original goal was for subprocess to replace os.system, os.popen, os.spawn, etc. That's never quite happened because subprocess is just a little bit too conceptually complex for those basic tasks. Is that way? I didn't find it particularly hard to learn, given the cheat sheet of techniques for doing the replacements. However, it is a bit deficient in providing non-blocking handling primitives for actually communicating with interactive spawned programs. subprocess.communicate provides one technique, which works for an extremely limited set of circumstances: I've offered some alternatives in http://bugs.python.org/issue10482 that greatly expand the types of communications that can be achieved without deadlock. Of course, it is still a small fraction of the possible circumstances, and doesn't really handle the hard case of feeding a program and concurrently analyzing its output to determine its future feedings: that still requires a complex, explicitly threaded and synchronized program. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87070 - python/branches/py3k/Lib/test/test_shutil.py
On Sat, Dec 4, 2010 at 20:04, hirokazu.yamamoto
python-check...@python.orgwrote:
Author: hirokazu.yamamoto
Date: Sun Dec 5 03:04:16 2010
New Revision: 87070
Log:
Now can reproduce the error on AMD64 Windows Server 2008
even where os.symlink is not supported.
Modified:
python/branches/py3k/Lib/test/test_shutil.py
Modified: python/branches/py3k/Lib/test/test_shutil.py
==
--- python/branches/py3k/Lib/test/test_shutil.py(original)
+++ python/branches/py3k/Lib/test/test_shutil.pySun Dec 5 03:04:16
2010
@@ -271,25 +271,33 @@
shutil.rmtree(src_dir)
shutil.rmtree(os.path.dirname(dst_dir))
-@unittest.skipUnless(hasattr(os, symlink),
- Missing symlink implementation)
+@unittest.skipUnless(hasattr(os, 'link'), 'requires os.link')
def test_dont_copy_file_onto_link_to_itself(self):
# bug 851123.
os.mkdir(TESTFN)
src = os.path.join(TESTFN, 'cheese')
dst = os.path.join(TESTFN, 'shop')
try:
-f = open(src, 'w')
-f.write('cheddar')
-f.close()
-
-if hasattr(os, link):
-os.link(src, dst)
-self.assertRaises(shutil.Error, shutil.copyfile, src, dst)
-with open(src, 'r') as f:
-self.assertEqual(f.read(), 'cheddar')
-os.remove(dst)
+with open(src, 'w') as f:
+f.write('cheddar')
+os.link(src, dst)
+self.assertRaises(shutil.Error, shutil.copyfile, src, dst)
+with open(src, 'r') as f:
+self.assertEqual(f.read(), 'cheddar')
+os.remove(dst)
+finally:
+shutil.rmtree(TESTFN, ignore_errors=True)
+@unittest.skipUnless(hasattr(os, symlink),
+ Missing symlink implementation)
+def test_dont_copy_file_onto_symlink_to_itself(self):
+# bug 851123.
+os.mkdir(TESTFN)
+src = os.path.join(TESTFN, 'cheese')
+dst = os.path.join(TESTFN, 'shop')
+try:
+with open(src, 'w') as f:
+f.write('cheddar')
# Using `src` here would mean we end up with a symlink pointing
# to TESTFN/TESTFN/cheese, while it should point at
# TESTFN/cheese.
@@ -299,10 +307,7 @@
self.assertEqual(f.read(), 'cheddar')
os.remove(dst)
finally:
-try:
-shutil.rmtree(TESTFN)
-except OSError:
-pass
+shutil.rmtree(TESTFN, ignore_errors=True)
@unittest.skipUnless(hasattr(os, symlink),
Missing symlink implementation)
___
Python-checkins mailing list
python-check...@python.org
http://mail.python.org/mailman/listinfo/python-checkins
I created #10540 for this issue, but the patch I have on there is just a bad
hack. I need to fix os.path.samefile for hard links, which might be easier
if we keep st_ino data in stat structures on Windows.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87070 - python/branches/py3k/Lib/test/test_shutil.py
On 2010/12/05 11:08, Brian Curtin wrote: I created #10540 for this issue, but the patch I have on there is just a bad hack. I need to fix os.path.samefile for hard links, which might be easier if we keep st_ino data in stat structures on Windows. MSDN says, http://msdn.microsoft.com/en-us/library/aa363788%28v=VS.85%29.aspx In the NTFS file system, a file keeps the same file ID until it is deleted. Hard link can be live in NTFS only, so maybe we can say st_ino data is valid until hard links are established. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87070 - python/branches/py3k/Lib/test/test_shutil.py
I missed it, st_dev is not set yet. When I set st_dev with dwVolumeSerialNumber, it was sometimes negative. Is it OK? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
