[Python-Dev] Weekly Python Patch/Bug Summary
Patch / Bug Summary ___ Patches : 386 open ( +4) / 3089 closed (+10) / 3475 total (+14) Bugs: 889 open ( +9) / 5636 closed (+12) / 6525 total (+21) RFE : 212 open ( +1) / 201 closed ( +0) / 413 total ( +1) New / Reopened Patches __ db4.4 support for bsddb (2006-03-02) CLOSED http://python.org/sf/1441660 opened by Jérome Laheurte Faster list comprehensions (2006-03-03) CLOSED http://python.org/sf/1442442 opened by Collin Winter Add support for the If-Modified-Since header (2006-03-03) http://python.org/sf/1442867 opened by Shannon -jj Behrens PyLong_FromString optimization (2006-03-04) http://python.org/sf/1442927 opened by Alan McIntyre Incremental codecs for CJKCodecs (2006-03-05) http://python.org/sf/1443155 opened by Hye-Shik Chang Remove bad PREDICT in ceval.c (2006-03-04) http://python.org/sf/1443159 opened by Collin Winter gc.get_count() and other changes (2006-03-05) CLOSED http://python.org/sf/1443865 opened by Barry A. Warsaw email/charset.py convert() patch (2006-03-06) http://python.org/sf/1443875 opened by Tokio Kikuchi Various minor fixes for NULL checks, etc. (2006-03-06) http://python.org/sf/1444030 opened by Hye-Shik Chang Make itertools.tee participate in GC (2006-03-06) http://python.org/sf/1444398 opened by Thomas Wouters kwdargs for compile/__import__ (2006-03-07) http://python.org/sf/1444529 opened by Thomas Wouters New with semantics (2006-03-08) http://python.org/sf/1445739 opened by Guido van Rossum Patches Closed __ db4.4 support for bsddb (2006-03-02) http://python.org/sf/1441660 closed by nnorwitz Inconsistency in os' function naming (2006-03-01) http://python.org/sf/1440928 closed by gvanrossum PEP 357 --- adding nb_index (2006-02-21) http://python.org/sf/1436368 closed by gvanrossum Faster list comprehensions (2006-03-03) http://python.org/sf/1442442 closed by nnorwitz The email package needs an application type (2006-02-04) http://python.org/sf/1424065 closed by bwarsaw Size of time_t changed to 64 bits in MSVC++ 2005 CRT (2006-02-24) http://python.org/sf/1437769 closed by loewis gc.get_count() and other changes (2006-03-05) http://python.org/sf/1443865 closed by bwarsaw StreamReader.readline with size reading multiple lines (2005-12-13) http://python.org/sf/1379332 closed by doerwalter Make property use the getter's __doc__ if no doc argument (2006-02-18) http://python.org/sf/1434038 closed by gbrandl New / Reopened Bugs ___ yet another svn head compiler change (2006-03-02) http://python.org/sf/1441486 opened by Michael Hudson socket read() can cause MemoryError in Windows (2006-03-02) http://python.org/sf/1441530 opened by taukki Omission in docs for urllib2.urlopen() (2006-03-02) http://python.org/sf/1441864 opened by Kent Johnson A (treaded) Python crash only on dual core machines (2006-03-02) CLOSED http://python.org/sf/1441884 opened by Robert Kiendl Multiple simultaneous sendtos on AF_UNIX socket broken. (2006-03-02) http://python.org/sf/1441984 opened by Brian Brunswick Traceback in pydoc (2006-03-02) CLOSED http://python.org/sf/1442012 opened by Thomas Heller IDLE shell window gets very slow when displaying long lines (2006-03-03) http://python.org/sf/1442493 opened by Heiko Selber os.listdir doesn't release GIL (2006-02-15) CLOSED http://python.org/sf/1432525 reopened by gvanrossum docs for os.statvfs miss f_bsize parameter (2006-03-03) CLOSED http://python.org/sf/1442767 opened by Chris Cogdon handling comments with markupbase and HTMLParser (2006-03-03) http://python.org/sf/1442874 opened by Daniel Bad Coroutine link (2006-03-04) CLOSED http://python.org/sf/1442937 opened by Paddy McCarthy Pickle protocol 2 fails on private slots. (2006-03-05) http://python.org/sf/1443328 opened by Daniele Varrazzo locale.getpreferredencoding() dies when setlocale fails (2006-03-05) http://python.org/sf/1443504 opened by Catherine Devlin email 3.0+ stops parsing headers prematurely (2006-03-05) http://python.org/sf/1443866 opened by Mark Sapiro os.open() Documentation (2006-03-06) http://python.org/sf/1444104 opened by Thomas Guettler subprocess test cases fail with noexec-mounted /tmp (2006-03-06) http://python.org/sf/108 opened by Wummel Pointer freed twice in Py_InitializeEx() (2006-03-07) http://python.org/sf/1444893 opened by athorp getpass.getpass queries on STDOUT not STERR (*nix) (2006-03-07) http://python.org/sf/1445068 opened by Jon Lasser embedding Python causes memory leaks (2006-03-08) http://python.org/sf/1445210 opened by Andrew Trevorrow install fails on hard link (2006-03-08)
Re: [Python-Dev] conditional expressions - add parens?
Morel Xavier wrote: Steve Holden wrote: Contrast with the bleeding obvious: level = 0 if absolute_import in self.futures: level = -1 regards Steve The issue that spawned the necessity of a ternary operator in the first place was that this syntax is not usable at all in quite a few situations like, say, list comprehensions... And of course the example I criticized was a list comprehension (not). The fact that ternary operators improve some features isn't an excuse for using them when they actually decrease readability. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd www.holdenweb.com Love me, love my blog holdenweb.blogspot.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
On 3/9/06, Nick Coghlan [EMAIL PROTECTED] wrote: Steven Elliott wrote: I'm interested in how builtins could be more efficient. I've read over some of the PEPs having to do with making global variables more efficient (search for global): http://www.python.org/doc/essays/pepparade.html But I think the problem can be simplified by focusing strictly on builtins. Unfortunately, builtins can currently be shadowed in the module global namespace from outside the module (via constructs like import mod; mod.str = my_str). Unless/until that becomes illegal, focusing solely on builtins doesn't help - the difficulties lie in optimising builtin access while preserving the existing name shadowing semantics. Is there any practical way of detecting and flagging constructs like the above (remotely shadowing a builtin in another module)? I can't see a way of doing it (but I know very little about this area...). If it *is* possible, I'd say it's worth implementing at least a warning sooner rather than later - the practice seems questionable at best, and any progress towards outlawing it would help in work on optimising builtins. Paul. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] quit() on the prompt
Guido van Rossum wrote: We seem to have a consensus. Is anybody working on a patch yet? http://python.org/sf/1446372 Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
On Thu, 2006-03-09 at 12:00 +, Paul Moore wrote: On 3/9/06, Nick Coghlan [EMAIL PROTECTED] wrote: Steven Elliott wrote: I'm interested in how builtins could be more efficient. I've read over some of the PEPs having to do with making global variables more efficient (search for global): http://www.python.org/doc/essays/pepparade.html But I think the problem can be simplified by focusing strictly on builtins. Unfortunately, builtins can currently be shadowed in the module global namespace from outside the module (via constructs like import mod; mod.str = my_str). Unless/until that becomes illegal, focusing solely on builtins doesn't help - the difficulties lie in optimising builtin access while preserving the existing name shadowing semantics. Is there any practical way of detecting and flagging constructs like the above (remotely shadowing a builtin in another module)? I can't see a way of doing it (but I know very little about this area...). It may be possible to flag it, or it may be possible it make it work. In my post I mentioned one special case that needs to be addressed (assigning to __builtins__). What Nick mentioned in his post (import mod; mod.str = my_str) is another special case that needs to be addressed. If we can assume that all pyc files are compiled with the same set of default bulitins (which should be assured by the by the version in the pyc file) then there are two ways that things like mod.str = my_str could be handled. I believe that currently mod.str = my_str alters the module's global hash table (f-f_globals in the code). One way of handling it is to alter STORE_ATTR (op code for assigning to mod.str) to always check to see if the key being assigned is one of the default builtins. If it is, then the module's indexed array of builtins is assigned to. Alternatively if we also wanted to optimize mod.str = my_str then there could be a new opcode like STORE_ATTR that would take an index into the array of builtins instead of an index into the names. PEP 280, which Nick mentioned, talks about a cells, a hybrid data structure that can do both hash table lookups and lookups by index efficiently. That's great, but I'm curious if additional gains can be made be focusing just on builtins. -- --- | Steven Elliott | [EMAIL PROTECTED] | --- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
At 08:50 AM 3/9/2006 -0600, Steven Elliott wrote: I believe that currently mod.str = my_str alters the module's global hash table (f-f_globals in the code). One way of handling it is to alter STORE_ATTR (op code for assigning to mod.str) to always check to see if the key being assigned is one of the default builtins. If it is, then the module's indexed array of builtins is assigned to. It's not the opcode that would change, it's the C function referenced by the module type's tp_setattro function slot. This has already been attempted before, in order to implement a warning for this behavior. You might want to research that, because the patch ended up being backed out for some reason. I think it ended up being too strict of a check to be accepted for Python 2.4. If some version of it could come back, however, it's possible we could use this in Python 2.5 to allow -O compilation to assume that unshadowed builtins are static, making it potentially possible to convert some of them to specialized bytecodes, or perhaps a BUILTIN_OP n bytecode, where 'n' is a combination of an operation selector and the number of arguments to be taken from the stack. The determination of unshadowed would have to be conservative, in that 'from foo import *' might shadow a builtin, so using 'import *' would disable optimization of all builtins. However, if that were not present, and there's no statically detectable assignment shadowing a particular builtin, that builtin could be optimized. (Assuming -O, of course.) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] ctypes is in SVN now.
ctypes is in SVN now, and the buildbot is green, after I disabled a test that dumped core on sparc solaris. The crash was apparently caused Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Thomas Heller [EMAIL PROTECTED] wrote: Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time. The goal of this work would exactly fit your need: make the creation of extension modules trivially easy (just as easy as adding the modules to the main python.dll). My personal goal, in fact, is to move many of those builtin extension modules from python.dll out into their own .pyd files where they'd belong (were not for this technical annoyance of being forced to use the settings dialog in MSVC). -- Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Giovanni Bajo wrote: Thomas Heller [EMAIL PROTECTED] wrote: Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time. The goal of this work would exactly fit your need: make the creation of extension modules trivially easy (just as easy as adding the modules to the main python.dll). My personal goal, in fact, is to move many of those builtin extension modules from python.dll out into their own .pyd files where they'd belong (were not for this technical annoyance of being forced to use the settings dialog in MSVC). Ideally this would be integrated with distutils because the setup-script has most of the information that's needed. OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough. Oh, IIRC the pywin32 setup script has code that is able to parse MSVC6 project files and create a distutils Extension instance from them - maybe this can help you. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Thomas Heller [EMAIL PROTECTED] wrote: I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time. Ideally this would be integrated with distutils because the setup-script has most of the information that's needed. OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough. I fear this is an orthogonal change. Alas, using distutils to build core extensions is not something I'm ready to tackle at the moment. I was just thiking of something much more naive like using a free tool to build .vcproj/.sln (see www.cmake.org for instance) from a script description. With such a tool, it would be very easy to build a .pyd file around a self-contained .c file (like _heapqmodule.c, etc.), it would mostly be a couple of line changes in the script file, and then re-run the tool executable to regenerate vcproj/sln. OTOH, both the tool executable (cmake.exe in this example) and the current version of the generated vcproj/sln files would be committed in SVN under PCbuild, so to have a minimal impact on developer habits. -- Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
[Steven Elliott] As you probably know each access of a builtin requires two hash table lookups. First, the builtin is not found in the list of globals. It is then found in the list of builtins. If someone really cared about the double lookup, they could flatten a level by starting their modules with: from __builtin__ import * However, we don't see people writing this kind of code. That could mean that the double lookup hasn't been a big concern. Why not have a means of referencing the default builtins with some sort of index the way the LOAD_FAST op code currently works? FWIW, there was a PEP proposing a roughly similar idea, but the PEP encountered a great deal of resistance: http://www.python.org/doc/peps/pep-0329/ When it comes time to write your PEP, it would helpful to highlight how it differs from PEP 329 (i.e. implemented through the compiler rather than as a bytecode hack, etc.). Perhaps what I'm suggesting isn't feasible for reasons that have already been discussed. But it seems like it should be possible to make while True as efficient as while 1. That is going to be difficult as long as it is legal to write: True = 0 Raymond ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Giovanni Bajo wrote: Thomas Heller [EMAIL PROTECTED] wrote: I discussed with Martin a bit about the opportunity of generating .vcproj files with a script-driven tool. I'm going to try and setup something as soon as I find some spare time. Ideally this would be integrated with distutils because the setup-script has most of the information that's needed. OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough. I fear this is an orthogonal change. Alas, using distutils to build core extensions is not something I'm ready to tackle at the moment. I was just thiking of something much more naive like using a free tool to build .vcproj/.sln (see www.cmake.org for instance) from a script description. With such a tool, it would be very easy to build a .pyd file around a self-contained .c file (like _heapqmodule.c, etc.), it would mostly be a couple of line changes in the script file, and then re-run the tool executable to regenerate vcproj/sln. OTOH, both the tool executable (cmake.exe in this example) and the current version of the generated vcproj/sln files would be committed in SVN under PCbuild, so to have a minimal impact on developer habits. While I don't understand *why* you would want to do this (building Python interpreters with a customized list of builtin modules could be a reason for this, I've had thought about something like that for py2exe a long time ago), I've clicked myself through the MSVC dialog and I'm happy now because the two .vcproj files are ready. I'll commit them and let others (Martin, probably) extend them for 64-bit Windows versions ;-). Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Thomas Heller [EMAIL PROTECTED] wrote: Ideally this would be integrated with distutils because the setup-script has most of the information that's needed. OTOH, extensions could be built with distutils even for core Python, and even on Windows. For extensions that are *not* builtin or not included with Python itself distutils works good enough. I fear this is an orthogonal change. Alas, using distutils to build core extensions is not something I'm ready to tackle at the moment. I was just thiking of something much more naive like using a free tool to build .vcproj/.sln (see www.cmake.org for instance) from a script description. With such a tool, it would be very easy to build a .pyd file around a self-contained .c file (like _heapqmodule.c, etc.), it would mostly be a couple of line changes in the script file, and then re-run the tool executable to regenerate vcproj/sln. OTOH, both the tool executable (cmake.exe in this example) and the current version of the generated vcproj/sln files would be committed in SVN under PCbuild, so to have a minimal impact on developer habits. While I don't understand *why* you would want to do this (building Python interpreters with a customized list of builtin modules could be a reason for this, I've had thought about something like that for py2exe a long time ago), That's exactly the reason: packaged executables. I'm sure there is still some weird encoding in world, with 2Mb of cute codec data tables, which is only waiting for people to find out and merge it into python.dll -- because it's harder to maintain it as an external .pyd with the current PCbuild. That's unreasonable IMVHO. CJK codecs *ought* to be distributed as an external .pyd. -- Giovanni Bajo ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
Steven Elliott [EMAIL PROTECTED] wrote: I'm interested in how builtins could be more efficient. I've read over some of the PEPs having to do with making global variables more efficient (search for global): http://www.python.org/doc/essays/pepparade.html But I think the problem can be simplified by focusing strictly on builtins. Whether or not it is a good idea, it is not wholly uncommon for users to manipulate variables in the builtin or other module namespaces. From very simple import hooks replacing __import__, to module.QUIT = 1, these things happen, and I don't think that making them produce a warning as suggested by Paul, necessarily helps us. I personally think the problem can be simplified (and solved) by applying decorators where such builtin lookups could/should be more efficient. Raymond has a recipe for doing such optimization over at the Python cookbook: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940 Toss that little guy into a decorators module; if people want their runtime calling to be fast (at the expense of a slightly increased compilation time), they'll start using it. - Josiah ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
* Paul Moore wrote: If it *is* possible, I'd say it's worth implementing at least a warning sooner rather than later - the practice seems questionable at best, and any progress towards outlawing it would help in work on optimising builtins. /lurk FWIW, this practice is very handy for unit tests. For example, I often shadow builtins like ``file`` for my tests with a mock placed from the outside into the global namespace. nd lurk -- Das Verhalten von Gates hatte mir bewiesen, dass ich auf ihn und seine beiden Gefährten nicht zu zählen brauchte -- Karl May, Winnetou III Im Westen was neues: http://pub.perlig.de/books.html#apache2 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
On 3/9/06, Thomas Heller [EMAIL PROTECTED] wrote: ctypes is in SVN now, and the buildbot is green,Well, only by accident; Neal's amd64 machine has been offline, or you would have seen yellow blocks for warnings: ctypes has yet to be Py_ssize_t'ed. Do you want to do that yourself, or do you want me to submit a patch? (Or I could just check it in ;) after I disabled a test that dumped coreon sparc solaris.The crash was apparently caused Ohh, mystery. Do we get to guess what it was apparently caused by? :)-- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
At 08:51 AM 3/9/2006 -0800, Raymond Hettinger wrote: Perhaps what I'm suggesting isn't feasible for reasons that have already been discussed. But it seems like it should be possible to make while True as efficient as while 1. That is going to be difficult as long as it is legal to write: True = 0 In that case, it can obviously be determined statically that the builtin is shadowed, and optimization is impossible for that builtin in that module. What *is* difficult is the somemodule.True = 0 case, and the somemodule.__dict__['True']=0 case, and the globals()['True'] = 0 case, and other such fiddling. Personally, I think that such shenanigans (in the case where somemodule doesn't already have a module-level binding for the builtin) should not be allowed to change the semantics of optimized functions using that dictionary as a global namespace. I think this is a reasonable implementation limit on dynamicity, although to be conservative I think we should only do this with -O in Python 2.5, if we do it at all. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
Phillip J. Eby wrote: I think this is a reasonable implementation limit on dynamicity, Absolutely. although to be conservative I think we should only do this with -O in Python 2.5, if we do it at all. Or with a __future__ directive? Just ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
[Tim Peters] Added: python/trunk/Tools/scripts/svneol.py (contents, props changed) Log: Simple utility to add svn:eol-style to text files under SVN control. Like reindent.py, I expect to run this mindlessly from time to time, checking in whatever it happens to do ;-) [Thomas Heller] Should 'sln' and 'vcproj' be added to the extensions list? I think these are text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, so the script would not change that. I don't know whether they're text files in the SVN eol-style native sense. The corresponding file types under VC 6 were not: in SVN terms, they should have svn:eol-style set to CRLF (Visual Studio required \r\n line ends for some inscrutable reason -- it would not tolerate \n line ends, and this mattered when, e.g., some Linux-head tried to run Visual Studio from a Linux-native checkout of Python; CRLF was still required, and we faked that under CVS by calling those files binary; SVN offers finer control; I don't know how picky the .NET 2003 MSDev is about this). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
Tim Peters wrote: [Tim Peters] Added: python/trunk/Tools/scripts/svneol.py (contents, props changed) Log: Simple utility to add svn:eol-style to text files under SVN control. Like reindent.py, I expect to run this mindlessly from time to time, checking in whatever it happens to do ;-) [Thomas Heller] Should 'sln' and 'vcproj' be added to the extensions list? I think these are text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, so the script would not change that. I don't know whether they're text files in the SVN eol-style native sense. The corresponding file types under VC 6 were not: in SVN terms, they should have svn:eol-style set to CRLF (Visual Studio required \r\n line ends for some inscrutable reason -- it would not I know. In CVS, they had to be added using the -kb flag, IIRC. tolerate \n line ends, and this mattered when, e.g., some Linux-head tried to run Visual Studio from a Linux-native checkout of Python; CRLF was still required, and we faked that under CVS by calling those files binary; SVN offers finer control; I don't know how picky the .NET 2003 MSDev is about this). I remember that someone has tested that the .vcproj files actually did work even if they had the wrong line endings, so it should be ok. AFAIK, noone tried that with the .sln file - anyway, I have removed the svn:mem-type property and the script then set svn:eol-style native. Maybe a different set of svn properties were better, I have to learn about them. The problem that I saw is that 'svn diff' refused to provide a diff when pcbuild.sln had changed - I would consider this diff useful, both in commit-emails as well as run on the command line before checking in. The Linux-head is the release manager, prepaing the release on a unixish platform, in combination with the windows guy who downloads the source distribution and tries to compile that on Windows. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
[Thomas Heller] ... And I never had tried it before on a sparc machine - all the intel and ppc processors seem to have no problems with it. Pentiums don't enforce natural alignment restrictions, but run much slower on unaligned access (varying by specific chip model, and generally more heavily penalized as time goes on). In the good old days, Pentium was one of dozens of competing architectures, and was the oddball in catering to unaligned access. Now it's eternal backward compatibility with an early implementation accident. Most other architectures never catered to unaligned access, or did so only at the cost of generating an interrupt so that kernel-mode software could fake unaligned access. Bottom line is that unaligned access isn't portable and never was, and even on architectures where it works it can be extremely expensive to use it. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
[Thomas Heller] ... I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) Under VC 6 I generally just renamed a copy of a similar existing project file and used a text editor to change the paths. I haven't tried that under the 7.1 MSDev. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Tim Peters wrote: [Thomas Heller] ... And I never had tried it before on a sparc machine - all the intel and ppc processors seem to have no problems with it. Pentiums don't enforce natural alignment restrictions, but run much slower on unaligned access (varying by specific chip model, and generally more heavily penalized as time goes on). In the good old days, Pentium was one of dozens of competing architectures, and was the oddball in catering to unaligned access. Now it's eternal backward compatibility with an early implementation accident. Most other architectures never catered to unaligned access, or did so only at the cost of generating an interrupt so that kernel-mode software could fake unaligned access. Bottom line is that unaligned access isn't portable and never was, and even on architectures where it works it can be extremely expensive to use it. I'm old enough to know this, but thanks anyway. I'm not so speed paranoid to care about these nanoseconds, maybe timeit can tell if there's a measurable difference. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Tim Peters wrote: [Thomas Heller] ... I could do that myself, but only for x86, while other .pyd files also have build settings for Itanium and x86_64. (I find it always very painful to click through the settings dialog in MSVC - isn't there any other way to create these files?) Under VC 6 I generally just renamed a copy of a similar existing project file and used a text editor to change the paths. I haven't tried that under the 7.1 MSDev. That's what I did when I created _ctypes_test.vcproj from _ctypes.vcproj. I had to insert a new guid though. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
[Thomas Heller] ... The Linux-head is the release manager, prepaing the release on a unixish platform, in combination with the windows guy who downloads the source distribution and tries to compile that on Windows. That sounds like an artificial (process-created) problem, then. Since a release should be made from a tag, there doesn't seem a real reason to make the Windows release wait for a source-release tarball to get made (the Windows dweeb should be able to do a native checkout of the tag, and do their stuff in parallel -- that's how I used to do Windows releases, and it worked fine -- the only thing the Windows release serialized on was waiting for the release's final docs to get cut). Setting project files to some flavor of text mode makes sense for your other reasons regardless (like getting useful diffs). ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
Tim Peters wrote: [Thomas Heller] ... The Linux-head is the release manager, prepaing the release on a unixish platform, in combination with the windows guy who downloads the source distribution and tries to compile that on Windows. That sounds like an artificial (process-created) problem, then. Since a release should be made from a tag, there doesn't seem a real reason to make the Windows release wait for a source-release tarball to get made (the Windows dweeb should be able to do a native checkout of the tag, and do their stuff in parallel -- that's how I used to do Windows releases, and it worked fine -- the only thing the Windows release serialized on was waiting for the release's final docs to get cut). I did it the same way for the 2.3 releases that I made. But I did not mean the official binary Windows installer, I mean someone who downloads the official source tarball, and compiles that himself. There was at least one 2.3 release where one could not compile with MSVC6 from the source tarball because of wrong line endings. Setting project files to some flavor of text mode makes sense for your other reasons regardless (like getting useful diffs). Sure. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Thomas Heller wrote: ctypes is in SVN now, and the buildbot is green, after I disabled a test that dumped core on sparc solaris. The crash was apparently caused Missing are .vcproj files for Windows, both for the _ctypes.pyd extension and the _ctypes_test.pyd extension needed for testing. IIRC, Martin promised to create them - is this offer still valid? It is. I usually create the vcproj files by copying an existing one, and replacing everything that matters in a text editor. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
Tim Peters wrote: Should 'sln' and 'vcproj' be added to the extensions list? I think these are text-files too. Although PCBuild\pcbuild.sln has a binary mime-type property, so the script would not change that. I don't know whether they're text files in the SVN eol-style native sense. Last I tried, VS 2003 would tolerate \n line endings in vcproj files (these being plain XML files). I don't remember whether it would tolerate them in .sln files (as these are *not* XML files); this is why the .sln file is set to binary. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Generated code in test_ast.py
Grant Olson wrote: I’m finishing up a patch for bug 1441408. I had to change the asdl definitions which in turn caused a failure in test_ast.py. A comment in the file indicates that EVERYTHING BELOW IS GENERATED # and has a definition for Slice() that is no longer valid. Can anyone tell me how this is generated? Should it be automatically generated when I run asdl_c.py? You run it with -g, then manually put the output into the test case. I'm developing on Windows. I know in the past some of the autogenerated code from the ast-branch worked better via make. Would this have fixed itself on a linux box, or is this a manual process? It's a manual process. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r42929 - python/trunk/Tools/scripts/svneol.py
[Martin v. Löwis] Last I tried, VS 2003 would tolerate \n line endings in vcproj files (these being plain XML files). I don't remember whether it would tolerate them in .sln files (as these are *not* XML files); this is why the .sln file is set to binary. FYI, I tried changing pcbuild.sln to use plain \n, and it worked fine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
Steven Elliott wrote: One way of handling it is to alter STORE_ATTR (op code for assigning to mod.str) to always check to see if the key being assigned is one of the default builtins. If it is, then the module's indexed array of builtins is assigned to. As long as you're going to all that trouble, it doesn't seem like it would be much harder to treat all global names that way, instead of just a predefined set. The compiler already knows all of the names that are used as globals in the module's code. That's great, but I'm curious if additional gains can be made be focusing just on builtins. As long as builtins can be shadowed, I can't see how to make any extra use of the fact that it's a builtin. A semantic change would be needed, such as forbidding shadowing of builtins, or at least forbidding this from outside the module. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
Raymond Hettinger wrote: That is going to be difficult as long as it is legal to write: True = 0 BTW, are there any plans to make True and False hard constants in 3.0 (like None is now)? Maybe also others like Ellipsis, NotImplemented, etc. Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
On 3/9/06, Thomas Heller [EMAIL PROTECTED] wrote: Thomas Wouters wrote: On 3/9/06, Thomas Heller [EMAIL PROTECTED] wrote: ctypes is in SVN now, and the buildbot is green, Well, only by accident; Neal's amd64 machine has been offline, or you would have seen yellow blocks for warnings: ctypes has yet to be Py_ssize_t'ed. DoDo compiler warnings create yellow blocks?There are other warnings on other machinesas well.OTOH, testing ctypes (see below) can cause core dumps, these should create orange blinking blocks ;-)I thought they did, but I guess I was wrong :) Or maybe just the header turns orange (not yellow.) you want to do that yourself, or do you want me to submit a patch? (Or I could just check it in ;)You can do it faster then me, I assume - so go ahead and check it in.I'll backport it to the upstream ctypes CVS repository. I'm not sure if I'll be faster. It's quite a lot of work to make it completely Py_ssize_t-aware (but worth it, IMHO; ints just don't cut it on 64-bit platforms. :) I have a simple patch to silence the warnings, but it doesn't support indexing beyond int-size and such. I'm also going through all of the code and changing most appropriate things into Py_ssize_t's, and that would be much nicer. I'll probably have some XXX markers left when I'm done, though. Also, how stable should the C API be? Does the C code have any direct users besides Python? -- Thomas Wouters [EMAIL PROTECTED]Hi! I'm a .signature virus! copy me into your .signature file to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
On 3/9/06, Greg Ewing [EMAIL PROTECTED] wrote: BTW, are there any plans to make True and False hard constants in 3.0 (like None is now)? Maybe also others like Ellipsis, NotImplemented, etc. I don't think we should make any of these keywords. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Making builtins more efficient
At 12:46 PM 3/10/2006 +1300, Greg Ewing wrote: Steven Elliott wrote: One way of handling it is to alter STORE_ATTR (op code for assigning to mod.str) to always check to see if the key being assigned is one of the default builtins. If it is, then the module's indexed array of builtins is assigned to. As long as you're going to all that trouble, it doesn't seem like it would be much harder to treat all global names that way, instead of just a predefined set. The compiler already knows all of the names that are used as globals in the module's code. But knowing that an operation is a builtin allows for the possibility of invoking the equivalent C operation directly in the interpreter (e.g. via a jump table), thus letting us translate something like len(foo) from: LOAD_GLOBAL len LOAD_FAST foo CALL_FUNCTION 1 into something like: LOAD_FAST foo BUILTIN_OP len, 1 And, the BUILTIN_OP could invoke a C function passing in a pointer to the arguments on the stack, so as to bypass tuple allocation, and the C function would use PySequence_Length() rather than going through the Python calling convention to PyObject_Call() the 'len' object. Now, whether that'll actually speed real programs up much, I don't know. But there are certainly a lot of things being cut out of the process using this approach, including an opcode fetch/decode, two dictionary lookups (one failing, one successful), and perhaps some tuplefying (only for C funcs w argcount1, since those builtins don't need an argtuple IIRC). And, even if it does speed things up a good deal, there's still a question of whether it should be done, when some real systems hack modules' builtins for testing. However, if BUILTIN_OP were to verify at runtime that __builtins__ is the interpreter standard builtins (i.e., the restricted-mode check), then it could dynamically choose to do the slower operation lookup. That would allow you to hack a module's builtins by giving it a fresh __builtins__ dictionary to implement that kind of monkeypatching. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] unicodedata.c no longer compiles on Windows
Compiling... unicodedata.c \Code\python\Modules\unicodedata.c(74) : error C2133: 'unicodedata_functions' : unknown size It's griping about this: /* Forward declaration */ static PyMethodDef unicodedata_functions[]; ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes is in SVN now.
Thomas Wouters wrote: (about ctypes and Py_ssize_t) you want to do that yourself, or do you want me to submit a patch? (Or I could just check it in ;) You can do it faster then me, I assume - so go ahead and check it in. I'll backport it to the upstream ctypes CVS repository. I'm not sure if I'll be faster. It's quite a lot of work to make it completely Py_ssize_t-aware (but worth it, IMHO; ints just don't cut it on 64-bit platforms. :) I have a simple patch to silence the warnings, but it doesn't support indexing beyond int-size and such. I'm also going through all of the code and changing most appropriate things into Py_ssize_t's, and that would be much nicer. I'll probably have some XXX markers left when I'm done, though. Also, how stable should the C API be? Does the C code have any direct users besides Python? Apart from a few functions that are used as foreign functions from ctypes itself (memmove and memset currently), the C api is not used from the outside. The important thing when changing something in the ctypes sources is to run the unittests (*) on as many platforms and Python versions as possible - currently I do this on all the sourceforge compile farm machines that have either Python = 2.3, or where I have been able to compile Python myself. Plus WindowsXP 32 and OSX 10.4. This is for the ctypes upstream version, of course. If your patch is going to change more than a few int - Py_ssize_t replacements it would probably be less work to do it in ctypes CVS instead of Python SVN. Maintaining code in multiple repositories is no fun ;) (*) ctypes unittests are run by 'python ctypes/test/runtests.py', for example. Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Developing/patching ctypes (was: Re: integrating ctypes into python)
[CC to python-dev again] Hye-Shik Chang wrote: On 3/10/06, Thomas Heller [EMAIL PROTECTED] wrote: Hye-Shik Chang wrote: On 3/9/06, Thomas Heller [EMAIL PROTECTED] wrote: I've now committed ctypes 0.9.9.4 into SVN. It seems to build, at least on the buildbots that are online. Good work! It's all green on my machines (FreeBSD amd64/sparc64). But gcc outputs some warnings: 1) libffi/include/ffi.h.in: void (*fn)() - void (*void)(void) This is not the only file that would have to be changed. I wonder why they don't change it in gcc itself. This *is* the only change to calm gcc warnings down except Py_ssize_t stuff. :-) 2) many of Py_ssize_t / int prototype mismatches. (eg. sq_repeat) Could we make ctypes Py_ssize_t clean? 3) ctypes doesn't support Py_ssize_t nor ssize_t yet. Because major portion of our pythonapi uses it now, it would be better to have it. This should be done in ctypes CVS, and the result imported into python SVN after that. At least ctypes (the Python module) has an c_size_t type now ;-). I'll do it in this weekend. I saw that branch_1_0 is merged into HEAD. Which branch are you using for Python integration at now? I did 'cvs export -r release_0_9_9_4' from the SF repository and then imported this into python SVN external/ctypes-0.9.9.4. The 'release_0_9_9_4' tag is on the 'branch_1_0' branch. I moved on a little bit on that branch, later merged everything into SF CVS HEAD, and continued to work there. We'll have to coordinate this. Different people working in different repositories will become a maintaince nightmare, so there should be an official 'master' repository for ctypes. Question for python-dev: Would it be a solution to move the 'official' ctypes development into Python SVN external/ctypes, or would this be considered abuse? Another location in SVN could be used as well, if external is though to contain only vendor drops... Thomas ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
