Re: [Python-Dev] Idle, site.py, and the release candidates
On Sun, Mar 31, 2013 at 12:34 PM, Terry Jan Reedy tjre...@udel.edu wrote: While trying to test the patch for http://bugs.python.org/**issue5492 http://bugs.python.org/issue5492 on Windows, I discovered that quit() and exit() in the Idle Shell are now disabled, it seems, for all versions on all systems rather than just sometimes on Linux. The problem is a change in idlelib that invalidated an assumption made in site.py. Revs 81718-81721 for http://bugs.python.org/**issue9290 http://bugs.python.org/issue9290 changed idlelib.PyShell.PseudoFile (line 1277 in 3.3) to subclass io.TextIOBase, which subclasses IOBase. This gave PseudoFile and its subclasses a .fileno instance method attribute that raises io.UnsupportedOperation: fileno. This is not a bug since the doc for io.IOBase.fileno says: Return the underlying file descriptor (an integer) of the stream if it exists. An OSError is raised if the IO object does not use a file descriptor. (the particular error raised is not an issue here). This is the code for Quitter.__call__ in site.py (line 368 in 3.3): def __call__(self, code=None): # Shells like IDLE catch the SystemExit, but listen when # stdin wrapper is closed. try: fd = -1 if hasattr(sys.stdin, fileno): fd = sys.stdin.fileno() if fd != 0: # Don't close stdin if it wraps fd 0 sys.stdin.close() except: pass raise SystemExit(code) The incorrect assumption is that if sys.stdin.fileno exits but raises, the call did not come from a shell that needs .close called. I do not know enough about other circumstances in which stdin.fileno would do something other than return 0 to be sure of what the proper fix would be. (I increasingly dislike bare excepts as they hide the thinking and knowledge of the original programmer. What exception was expected that should be passed away?) The other problem is that making *two* function calls inside a broad try/except is almost always a terrible idea. It seems to me that the intended logic is more like this: try: # Close stdin if it wraps any fd other than 0 close_stdin = (sys.stdin.fileno() != 0) except (AttributeError, OSError, io.UnsupportedOperation): # Also close stdin if it doesn't expose a file descriptor close_stdin = True if close_stdin: try: sys.stdin.close() except Exception: pass raise SystemExit(code) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Idle, site.py, and the release candidates
On 3/31/2013 2:39 AM, Nick Coghlan wrote: On Sun, Mar 31, 2013 at 12:34 PM, Terry Jan Reedy tjre...@udel.edu mailto:tjre...@udel.edu I do not know enough about other circumstances in which stdin.fileno would do something other than return 0 to be sure of what the proper fix would be. (I increasingly dislike bare excepts as they hide the thinking and knowledge of the original programmer. What exception was expected that should be passed away?) The other problem is that making *two* function calls inside a broad try/except is almost always a terrible idea. That too. I could not decide which function an exception was expected from. It seems to me that the intended logic is more like this: try: # Close stdin if it wraps any fd other than 0 close_stdin = (sys.stdin.fileno() != 0) except (AttributeError, OSError, io.UnsupportedOperation): # Also close stdin if it doesn't expose a file descriptor close_stdin = True if close_stdin: try: sys.stdin.close() except Exception: pass raise SystemExit(code) There are 4 possible situations for sys.stdin: 1. No .fileno 2. .fileno raises 3. .fileno() != 0 4. lfileno() == 0 I believe the current code calls .close for 1 and raises SystemExit for 2-4. Your code calls for 1-3 and raises for 4. I suspect that is correct. For an rc patch, the safest temporary patch would be to start .__call__ with if sys.stdin.__name__ == 'PseudoInputFile': sys.stdin.close() I would have to check that the name is correct as seen in the user process (cannot at moment). The deeper problem, I think, is that none of sys.version, sys.hexversion, sys._version, sys.platform tell a program that it is running under Idle. It usually does not matter but there are a few situations in which it does. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Idle, site.py, and the release candidates
On Sat, 30 Mar 2013 22:34:32 -0400 Terry Jan Reedy tjre...@udel.edu wrote: I do not know enough about other circumstances in which stdin.fileno would do something other than return 0 to be sure of what the proper fix would be. (I increasingly dislike bare excepts as they hide the thinking and knowledge of the original programmer. You should learn to use the power of version control: http://docs.python.org/devguide/faq.html#how-do-i-find-out-who-edited-or-what-revision-changed-a-line-last $ hg ann Lib/site.py will tell you that the lines you mention come from the following changeset: $ hg log -r 86358b43c8bb -vp changeset: 44390:86358b43c8bb parent: 44385:5670104acd39 user:Christian Heimes christ...@cheimes.de date:Mon Dec 31 03:07:24 2007 + files: Lib/site.py description: Don't close sys.stdin with quit() if sys.stdin wraps fd 0. Otherwise it will raise a warning: Lib/io.py:1221: RuntimeWarning: Trying to close unclosable fd diff --git a/Lib/site.py b/Lib/site.py --- a/Lib/site.py +++ b/Lib/site.py @@ -247,7 +247,12 @@ # Shells like IDLE catch the SystemExit, but listen when their # stdin wrapper is closed. try: -sys.stdin.close() +fd = -1 +if hasattr(sys.stdin, fileno): +fd = sys.stdin.fileno() +if fd != 0: +# Don't close stdin if it wraps fd 0 +sys.stdin.close() except: pass raise SystemExit(code) In this case the original motivation seems obsolete: import sys sys.stdin.fileno() 0 sys.stdin.close() That said, if IDLE users expect those global functions, perhaps IDLE should define its own ones rather than rely on site.py. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Semantics of __int__(), __index__()
Hi all, I was looking into http://bugs.python.org/issue17576 and I found that the semantics of __int__() and __index__() are not precisely defined in the documentation and that the implementation (CPython 3.4a) has some odd behaviour. Defining two classes: class Int1(int): def __init__(self, val=0): print(new %s % self.__class__) class Int2(Int1): def __int__(self): return self and two instances i1 = Int1() i2 = Int2() we get the following behaviour: type(int(i1)) class 'int' I would have expected 'Int1' type(int(i2)) new class '__main__.Int2' class '__main__.Int2' Why is a new Int2 being created? operator.index does similar things. So, 1. Should type(int(x)) be exactly int, or is any subclass OK? 2. Should type(index(x)) be exactly int, or is any subclass OK? 3. Should int(x) be defined as int_check(x.__int__())? 4. Should operator.index(x) be defined as index_check(x.__index__())? where: def int_check(x): if is_int(x): return x else: raise TypeError(...) def index_check(x): if is_index(x): return x else: raise TypeError(...) The definition of is_int(x) and is_index(x) follow from the answers to 1 and 2. I had previously assumed (and would expect) that the answers were: 1. Any subclass is OK 2. Ditto 3. Yes 4. Yes Which means that def is_int(x): return int in type(x).__mro__ is_index = is_int Cheers, Mark. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On Sun, Mar 31, 2013 at 2:29 PM, Mark Shannon m...@hotpy.org wrote: class Int1(int): def __init__(self, val=0): print(new %s % self.__class__) class Int2(Int1): def __int__(self): return self and two instances i1 = Int1() i2 = Int2() we get the following behaviour: type(int(i1)) class 'int' I would have expected 'Int1' Wouldn't that remove the one obvious way to get an 'int' from an 'Int1'? 1. Should type(int(x)) be exactly int, or is any subclass OK? 2. Should type(index(x)) be exactly int, or is any subclass OK? 3. Should int(x) be defined as int_check(x.__int__())? 4. Should operator.index(x) be defined as index_check(x.__index__())? For (1), I'd say yes, it should be exactly an int, so my answer to (3) is no. As written, int_check would do the wrong thing for bools, too: I definitely want int(True) to be 1, not True. For (2) and (4), it's not so clear. Are there use-cases for an __index__ return value that's not directly of type int? I can't think of any offhand. Mark ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Idle, site.py, and the release candidates
On 3/31/2013 3:52 AM, Terry Reedy wrote: For an rc patch, the safest temporary patch would be to start .__call__ with if sys.stdin.__name__ == 'PseudoInputFile': sys.stdin.close() I would have to check that the name is correct as seen in the user process (cannot at moment). In addition, idlelib.PyShell.PseudoInputFile needs a .close method +def close(self): +self.shell.close() + http://bugs.python.org/issue17585 -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Idle, site.py, and the release candidates
On 3/31/2013 6:01 AM, Antoine Pitrou wrote: That said, if IDLE users expect those global functions, perhaps IDLE should define its own ones rather than rely on site.py. I thought of that. Idle would have to check the beginning of every statement before sending it to the user process, which would do the same thing. I personally think exit/quit should eventually go, as ^d/^z or [x] button are as easy. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Idle, site.py, and the release candidates
On Sun, 31 Mar 2013 11:38:07 -0400 Terry Jan Reedy tjre...@udel.edu wrote: On 3/31/2013 6:01 AM, Antoine Pitrou wrote: That said, if IDLE users expect those global functions, perhaps IDLE should define its own ones rather than rely on site.py. I thought of that. Idle would have to check the beginning of every statement before sending it to the user process, which would do the same thing. I personally think exit/quit should eventually go, as ^d/^z or [x] button are as easy. I never use them myself, but they are more discoverable than keyboard shortcuts, and hence can be useful for beginners or casual users. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] py2.7: dictobject not properly resizing
So I did a bit of benchmarking and attached is the code I used. With downsizing happening when ma_used * 2 = ma_filled, or the following for the condition in question: if (mp-ma_used = n_used || (mp-ma_fill*3 (mp-ma_mask+1)*2 mp-ma_used*2 mp-ma_fill)) I see marginally faster performance with downsizing. I chose a factor of 2x because it will ensure downsizings before the ma_fill load factor comes into play and will also not cause a resize on the next insert. Using the vanilla python2.7.3 code, there are never any resize events where the new size is smaller than the old size. Cheers, Micha - http://micha.gd/ http://github.com/mynameisfiber/ test.py Description: Binary data ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
and two instances i1 = Int1() i2 = Int2() we get the following behaviour: type(int(i1)) class 'int' I would have expected 'Int1' type(float(i1)) type 'float' type(float(i2)) class 'float' isinstance(int(i1), int) True isinstance(int(i2), int) new class '__main__.Int2' True isinstance(float(i1), float) True isinstance(float(i2), float) True why is printing new class '__main__.Int2'? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (2.7): Add an itertools recipe showing how to use t.__copy__().
Upcomping - upcoming On Mar 31, 2013 2:38 AM, raymond.hettinger python-check...@python.org wrote: http://hg.python.org/cpython/rev/1026b1d47f30 changeset: 83037:1026b1d47f30 branch: 2.7 parent: 83034:e044d22d2f61 user:Raymond Hettinger pyt...@rcn.com date:Sat Mar 30 23:37:57 2013 -0700 summary: Add an itertools recipe showing how to use t.__copy__(). files: Doc/library/itertools.rst | 12 1 files changed, 12 insertions(+), 0 deletions(-) diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -828,6 +828,18 @@ indices = sorted(random.randrange(n) for i in xrange(r)) return tuple(pool[i] for i in indices) + def tee_lookahead(t, i): + Inspect the i-th upcomping value from a tee object + while leaving the tee object at its current position. + + Raise an IndexError if the underlying iterator doesn't + have enough values. + + + for value in islice(t.__copy__(), i, None): + return value + raise IndexError(i) + Note, many of the above recipes can be optimized by replacing global lookups with local variables defined as default values. For example, the *dotproduct* recipe can be written as:: -- Repository URL: http://hg.python.org/cpython ___ Python-checkins mailing list python-check...@python.org http://mail.python.org/mailman/listinfo/python-checkins ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] mingw32 port
Matthias Klose wrote: [No, I'm not interested in the port myself] patches for a mingw32 port are floating around on the web and the python bug tracker, although most of them as a set of patches in one issue addressing several things, and which maybe outdated for the trunk. at least for me re-reading a big patch in a new version is more work than having the patches in different issues. So proposing to break down the patches in independent ones, dealing with: - mingw32 support (excluding any cross-build support). tested with a native build with srcdir == builddir. The changes for distutils mingw32 support should be a separate patch. Who could review these? Asked Martin, but didn't get a reply yet. - patches to cross-build for mingw32. - patches to deal with a srcdir != builddir configuration, where the srcdir is read-only (please don't mix with the cross-build support). All work should be done on the tip/trunk. So ok to close issue16526, issue3871, issue3754 and suggest in the reports to start over with more granular changes? I post first part of split of issue3871 . It is related only to build of interpreter core. Second part consist of 23 patches related to build of standard extensions . 3 of them are reported as separate issue by other users. As prerequisite is modernization of cygwincompiler.py - ref issue12641. I will post after 2-3 weeks remaining 20 granular updates. Third part is now with only tree updates related to installation (new). Unlike issue3871 will be supported posix installation scheme as all users refuse to use windows scheme. Matthias Roumen ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] A bit about the GIL
Hi, I know this may be tiresome by now, so feel free to ignore, but I'd like to share with the list an idea about the GIL, more specifically the reference counting mechanism. Simply put, make the reference counter a sharded one. That is, separate it into several subcounters, in this case one for each thread. The logic would then be something like this: - when increasing the refcount, a thread writes only to its own subcounter, creating one first if necessary. - similarly, when decreasing the refcount, there is no need to access other subcounters until that subcounter reaches zero. - when a subcounter gets to zero, delete it, and read the other subcounters to check if it was the last one. - delete the object only if there are no more subcounters. Contention could then be reduced to a minimum, since a thread only needs to read other subcounters when its own reaches zero or wants the total value. Depending on the implementation it might help with false sharing too, as subcounters may or may not be in the same cache-line. Unfortunately, in a crude test of mine there is already a severe performance degradation, and that is without rwlocks. I've used a basic linked list, and changed the INCREF/DECREF macros to functions to accommodate the extra logic so it may not be the best approach (too many dereferences). Does this makes sense to anyone? Thanks, Alfredo PS: At the very least, it might be another reason to keep the GIL. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A bit about the GIL
On Mon, 01 Apr 2013 01:14:11 +0200, =?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?= asol...@icai.es wrote: Simply put, make the reference counter a sharded one. That is, separate it into several subcounters, in this case one for each thread. It seems to me this has a family resemblance to some of the stuff Trent is doing in his parallel Python work. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A bit about the GIL
On Mon, Apr 1, 2013 at 11:49 AM, R. David Murray rdmur...@bitdance.comwrote: On Mon, 01 Apr 2013 01:14:11 +0200, =?UTF-8?Q?Alfredo_Solano_Mart=C3=ADnez?= asol...@icai.es wrote: Simply put, make the reference counter a sharded one. That is, separate it into several subcounters, in this case one for each thread. It seems to me this has a family resemblance to some of the stuff Trent is doing in his parallel Python work. It's also a thread for python-ideas, not python-dev. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
