Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-05 Thread Terry Reedy
On 1/4/2011 6:39 PM, Guido van Rossum wrote: So, that significantly weakens the argument that this optimization will break unit tests, since I am happy to promise never to optimize these builtins, and any other builtins intended for I/O. This is one comprehensible rule rather than a list of

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-05 Thread Maciej Fijalkowski
How about not changing semantics and still making this optimization possible? PyPy already has CALL_LIKELY_BUILTIN which checks whether builtins has been altered (by keeping a flag on the module dictionary) and if not, loads a specific builtin on top of value stack. From my current experience, I

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-05 Thread Guido van Rossum
On Wed, Jan 5, 2011 at 5:27 AM, Maciej Fijalkowski fij...@gmail.com wrote: How about not changing semantics and still making this optimization possible? PyPy already has CALL_LIKELY_BUILTIN which checks whether builtins has been altered (by keeping a flag on the module dictionary) and if not,

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Michael Foord
On 04/01/2011 01:02, Guido van Rossum wrote: On Mon, Jan 3, 2011 at 9:52 AM, David Malcolmdmalc...@redhat.com wrote: On Sun, 2011-01-02 at 19:18 -0800, Guido van Rossum wrote: On Sun, Jan 2, 2011 at 5:50 PM, Alex Gaynoralex.gay...@gmail.com wrote: No, it's singularly impossible to prove

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Nick Coghlan
On Tue, Jan 4, 2011 at 8:49 PM, Michael Foord fuzzy...@voidspace.org.uk wrote: I think someone else pointed this out, but replacing builtins externally to a module is actually common for testing. In particular replacing the open function, but also other builtins, is often done temporarily to

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Guido van Rossum
On Tue, Jan 4, 2011 at 2:49 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: I think someone else pointed this out, but replacing builtins externally to a module is actually common for testing. In particular replacing the open function, but also other builtins, is often done temporarily to

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Nick Coghlan
On Wed, Jan 5, 2011 at 1:58 AM, Guido van Rossum gu...@python.org wrote: On Tue, Jan 4, 2011 at 2:49 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: I think someone else pointed this out, but replacing builtins externally to a module is actually common for testing. In particular replacing

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Alex Gaynor
On Tue, Jan 4, 2011 at 10:20 AM, Nick Coghlan ncogh...@gmail.com wrote: On Wed, Jan 5, 2011 at 1:58 AM, Guido van Rossum gu...@python.org wrote: On Tue, Jan 4, 2011 at 2:49 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: I think someone else pointed this out, but replacing builtins

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Reid Kleckner
On Tue, Jan 4, 2011 at 8:21 AM, Alex Gaynor alex.gay...@gmail.com wrote: Ugh, I can't be the only one who finds these special cases to be a little nasty? Special cases aren't special enough to break the rules. Alex +1, I don't think nailing down a few builtins is that helpful for optimizing

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Barry Warsaw
On Jan 04, 2011, at 10:21 AM, Alex Gaynor wrote: Ugh, I can't be the only one who finds these special cases to be a little nasty? Special cases aren't special enough to break the rules. Yeah, I agree. Still it would be interesting to see what kind of performance improvement this would result

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Michael Foord
On 04/01/2011 16:54, Barry Warsaw wrote: On Jan 04, 2011, at 10:21 AM, Alex Gaynor wrote: Ugh, I can't be the only one who finds these special cases to be a little nasty? Special cases aren't special enough to break the rules. Yeah, I agree. Still it would be interesting to see what kind of

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Lukas Lueg
Doesnt this all boil down to being able to monitor PyDict for changes to it's key-space? The keys are immutable anyway so the instances of PyDict could manage a opaque value (in fact, a counter) that changes every time a new value is written to any key. Once we get a reference out of the dict, we

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Neil Schemenauer
Barry Warsaw ba...@python.org wrote: On Jan 04, 2011, at 10:21 AM, Alex Gaynor wrote: Ugh, I can't be the only one who finds these special cases to be a little nasty? Special cases aren't special enough to break the rules. Yeah, I agree. Still it would be interesting to see what kind of

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Steven D'Aprano
Guido van Rossum wrote: On Tue, Jan 4, 2011 at 2:49 AM, Michael Foord fuzzy...@voidspace.org.uk wrote: I think someone else pointed this out, but replacing builtins externally to a module is actually common for testing. In particular replacing the open function, but also other builtins, is

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Guido van Rossum
On Tue, Jan 4, 2011 at 1:50 PM, Steven D'Aprano st...@pearwood.info wrote: I've been known to monkey-patch builtins in the interactive interpreter and in test code. One example that comes to mind is that I had some over-complicated recursive while loop (!), and I wanted to work out the Big Oh

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Daniel Stutzbach
On Tue, Jan 4, 2011 at 9:33 AM, Lukas Lueg lukas.l...@googlemail.comwrote: The keys are immutable anyway so the instances of PyDict could manage a opaque value (in fact, a counter) that changes every time a new value is written to any key. Once we get a reference out of the dict, we can can

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Guido van Rossum
On Tue, Jan 4, 2011 at 2:15 PM, Daniel Stutzbach stutzb...@google.com wrote: On Tue, Jan 4, 2011 at 9:33 AM, Lukas Lueg lukas.l...@googlemail.com wrote: The keys are immutable anyway so the instances of PyDict could manage a opaque value (in fact, a counter) that changes every time a new

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Steven D'Aprano
Guido van Rossum wrote: On Tue, Jan 4, 2011 at 1:50 PM, Steven D'Aprano st...@pearwood.info wrote: I've been known to monkey-patch builtins in the interactive interpreter and in test code. One example that comes to mind is that I had some over-complicated recursive while loop (!), and I wanted

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Lukas Lueg
I very much like the fact that python has *very* little black magic revealed to the user. Strong -1 on optimizing picked builtins in a picked way. 2011/1/4 Steven D'Aprano st...@pearwood.info: Guido van Rossum wrote: On Tue, Jan 4, 2011 at 2:49 AM, Michael Foord fuzzy...@voidspace.org.uk

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Guido van Rossum
On Tue, Jan 4, 2011 at 3:13 PM, Steven D'Aprano st...@pearwood.info wrote: Guido van Rossum wrote: But why couldn't you edit the source code? Because there was no source code -- I was experimenting in the interactive interpreter. I could have just re-created the function by using the readline

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Guido van Rossum
On Tue, Jan 4, 2011 at 2:13 PM, Lukas Lueg lukas.l...@googlemail.com wrote: I very much like the fact that python has *very* little black magic revealed to the user. Strong -1 on optimizing picked builtins in a picked way. That's easy for you to say. -- --Guido van Rossum (python.org/~guido)

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Michael Foord
On 04/01/2011 23:29, Guido van Rossum wrote: On Tue, Jan 4, 2011 at 3:13 PM, Steven D'Apranost...@pearwood.info wrote: Guido van Rossum wrote: But why couldn't you edit the source code? Because there was no source code -- I was experimenting in the interactive interpreter. I could have just

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Michael Foord
On 04/01/2011 23:29, Guido van Rossum wrote: On Tue, Jan 4, 2011 at 3:13 PM, Steven D'Apranost...@pearwood.info wrote: Guido van Rossum wrote: But why couldn't you edit the source code? Because there was no source code -- I was experimenting in the interactive interpreter. I could have just

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-04 Thread Guido van Rossum
On Tue, Jan 4, 2011 at 3:36 PM, Michael Foord fuzzy...@voidspace.org.uk wrote: The only examples I could find from a quick search (using the patch decorator from my mock module which is reasonably well used) patch __builtins__.open and raw_input.

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-03 Thread Glyph Lefkowitz
On Jan 2, 2011, at 10:18 PM, Guido van Rossum wrote: On Sun, Jan 2, 2011 at 5:50 PM, Alex Gaynor alex.gay...@gmail.com wrote: No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong. True.

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-03 Thread Guido van Rossum
On Sun, Jan 2, 2011 at 9:36 PM, Terry Reedy tjre...@udel.edu wrote: On 1/2/2011 10:18 PM, Guido van Rossum wrote: My proposed way out of this conundrum has been to change the language semantics slightly so that global names which (a) coincide with a builtin, and (b) have no explicit

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-03 Thread Guido van Rossum
On Mon, Jan 3, 2011 at 6:12 AM, Glyph Lefkowitz gl...@twistedmatrix.com wrote: Wouldn't this optimization break things like mocking out 'open' for testing via 'module.open = fakeopen'?  I confess I haven't ever wanted to change 'len' but that one seems pretty useful. I am explicitly

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-03 Thread David Malcolm
On Sun, 2011-01-02 at 19:18 -0800, Guido van Rossum wrote: On Sun, Jan 2, 2011 at 5:50 PM, Alex Gaynor alex.gay...@gmail.com wrote: No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong.

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Victor Stinner
Le mercredi 29 décembre 2010 à 14:20 +0100, Martin v. Löwis a écrit : Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x.

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Ned Batchelder
On 1/2/2011 8:17 AM, Victor Stinner wrote: Le mercredi 29 décembre 2010 à 14:20 +0100, Martin v. Löwis a écrit : Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true;

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Alex Gaynor
Cesare Di Mauro cesare.di.mauro at gmail.com writes: 2010/12/28 Lukas Lueg lukas.lueg at googlemail.com Consider the following code: def foobar(x):    for i in range(5):        x[i] = i The bytecode in python 2.7 is the following:  2           0 SETUP_LOOP              30 (to 33)

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Guido van Rossum
On Sun, Jan 2, 2011 at 5:50 PM, Alex Gaynor alex.gay...@gmail.com wrote: No, it's singularly impossible to prove that any global load will be any given value at compile time.  Any optimization based on this premise is wrong. True. My proposed way out of this conundrum has been to change the

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Terry Reedy
On 1/2/2011 10:18 PM, Guido van Rossum wrote: My proposed way out of this conundrum has been to change the language semantics slightly so that global names which (a) coincide with a builtin, and (b) have no explicit assignment to them in the current module, would be fair game for such

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Cesare Di Mauro
2011/1/1 Ned Batchelder n...@nedbatchelder.com On 12/31/2010 12:51 PM, Cesare Di Mauro wrote: Aggressive optimizations can be enabled with explicit options, in order to leave normal debugger-prone code. I wish the Python compiler would adopt a strategy of being able to disable

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Cesare Di Mauro
2011/1/3 Alex Gaynor alex.gay...@gmail.com No, it's singularly impossible to prove that any global load will be any given value at compile time. Any optimization based on this premise is wrong. Alex That's your opinion, but I have very different ideas. Of course we can't leave the

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-02 Thread Nick Coghlan
On Mon, Jan 3, 2011 at 3:36 PM, Terry Reedy tjre...@udel.edu wrote: I believe this amounts to saying 1) Python code executes in three scopes (rather than two): global builtin, modular (misleadingly call global), and local. This much is a possible viewpoint today. 2) A name that is not an

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2011-01-01 Thread Ned Batchelder
On 12/31/2010 12:51 PM, Cesare Di Mauro wrote: 2010/12/31 s...@pobox.com mailto:s...@pobox.com Another example. I can totally remove the variable i, just using the stack, so a debugger (or, in general, having the tracing enabled) cannot even find something to change about

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread Maciej Fijalkowski
OK, but is it mandatory? For example, in the above code, I can unroll the loop because I found that range is the usual built-in, 5 is a low-enough constant, How do you know xrange is xrange and not something else? Cheers, fijal ___ Python-Dev

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread Maciej Fijalkowski
On Fri, Dec 31, 2010 at 12:00 PM, Maciej Fijalkowski fij...@gmail.com wrote: OK, but is it mandatory? For example, in the above code, I can unroll the loop because I found that range is the usual built-in, 5 is a low-enough constant, How do you know xrange is xrange and not something else?

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread Cesare Di Mauro
2010/12/31 Maciej Fijalkowski fij...@gmail.com On Fri, Dec 31, 2010 at 12:00 PM, Maciej Fijalkowski fij...@gmail.com wrote: OK, but is it mandatory? For example, in the above code, I can unroll the loop because I found that range is the usual built-in, 5 is a low-enough constant, How

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread Ethan Furman
Cesare Di Mauro wrote: 2010/12/29 Martin v. Löwis wrote: Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x. Another

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread skip
Another example. I can totally remove the variable i, just using the stack, so a debugger (or, in general, having the tracing enabled) cannot even find something to change about it. Ethan -1 Ethan Debugging is challenging enough as it is -- why would you want to Ethan

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread Cesare Di Mauro
2010/12/31 Ethan Furman et...@stoneleaf.us Cesare Di Mauro wrote: 2010/12/29 Martin v. Löwis wrote: Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-31 Thread Cesare Di Mauro
2010/12/31 s...@pobox.com Another example. I can totally remove the variable i, just using the stack, so a debugger (or, in general, having the tracing enabled) cannot even find something to change about it. Ethan -1 Ethan Debugging is challenging enough as it is --

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-30 Thread Cesare Di Mauro
2010/12/28 Lukas Lueg lukas.l...@googlemail.com Consider the following code: def foobar(x): for i in range(5): x[i] = i The bytecode in python 2.7 is the following: 2 0 SETUP_LOOP 30 (to 33) 3 LOAD_GLOBAL 0 (range)

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-30 Thread Cesare Di Mauro
2010/12/29 Martin v. Löwis mar...@v.loewis.de Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x. Regards, Martin OK,

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-29 Thread Martin v. Löwis
Am 28.12.2010 18:08, schrieb Lukas Lueg: Also, the load_fast in lne 22 to reference x could be taken out of the loop as x will always point to the same object That's not true; a debugger may change the value of x. Regards, Martin ___ Python-Dev

[Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-28 Thread Lukas Lueg
Consider the following code: def foobar(x): for i in range(5): x[i] = i The bytecode in python 2.7 is the following: 2 0 SETUP_LOOP 30 (to 33) 3 LOAD_GLOBAL 0 (range) 6 LOAD_CONST 1 (5) 9

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-28 Thread Benjamin Peterson
2010/12/28 Lukas Lueg lukas.l...@googlemail.com: Consider the following code: def foobar(x):    for i in range(5):        x[i] = i The bytecode in python 2.7 is the following:  2           0 SETUP_LOOP              30 (to 33)              3 LOAD_GLOBAL              0 (range)            

Re: [Python-Dev] Possible optimization for LOAD_FAST ?

2010-12-28 Thread Georg Brandl
Am 28.12.2010 18:24, schrieb Benjamin Peterson: 2010/12/28 Lukas Lueg lukas.l...@googlemail.com: Consider the following code: def foobar(x): for i in range(5): x[i] = i The bytecode in python 2.7 is the following: 2 0 SETUP_LOOP 30 (to 33)