Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-30 Thread Terry Reedy
On 5/29/2010 6:20 AM, Colin H wrote: Perhaps the next step is to re-open the issue? If it is seen as a bug, it would be great to see a fix in 2.6+ - For the purpose of bugfix releases, a 'bug' is a discrepancy between doc and behavior. Every new feature is seen as a 'design bug' by someone.

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-29 Thread Colin H
Perhaps the next step is to re-open the issue? If it is seen as a bug, it would be great to see a fix in 2.6+ - a number of options which will not break backward compatibility have been put forward - cheers, Colin On Thu, May 27, 2010 at 9:05 PM, Reid Kleckner r...@mit.edu wrote: On Thu, May

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-29 Thread Nick Coghlan
On 29/05/10 20:20, Colin H wrote: Perhaps the next step is to re-open the issue? If it is seen as a bug, it would be great to see a fix in 2.6+ - a number of options which will not break backward compatibility have been put forward - cheers, A new feature request requesting a closure mode for

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Colin H
I needed to make a small modification to the workaround - I wasn't able to delete from 'stuff', as the definitions in exec()'d code won't run - they're relying on that being present at runtime. In practice the overhead of doing this is quite noticeable if you run your code like this a lot, and

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Scott Dial
On 5/27/2010 7:14 AM, Colin H wrote: def define_stuff(user_code): context = {...} stuff = {} stuff.update(context) exec(user_code, stuff) return_stuff = {} return_stuff.update(stuff) del return_stuff['__builtins__'] for key in context: if key in return_stuff and

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Colin H
Yep fair call - was primarily modifying Guido's example to make the point about not being able to delete from the globals returned from exec - cheers, Colin On Thu, May 27, 2010 at 2:09 PM, Scott Dial scott+python-...@scottdial.com wrote: On 5/27/2010 7:14 AM, Colin H wrote: def

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Colin H
Just to put a couple of alternatives on the table that don't break existing code - not necessarily promoting them, or suggesting they would be easy to do - 1. modify exec() to take an optional third argument - 'scope_type' - if it is not supplied (but locals is), then it runs as class namespace -

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Nick Coghlan
On 27/05/10 10:38, Guido van Rossum wrote: On Wed, May 26, 2010 at 5:12 PM, Nick Coghlanncogh...@gmail.com wrote: Lexical scoping only works for code that is compiled as part of a single operation - the separation between the compilation of the individual string and the code defining that

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Nick Coghlan
On 27/05/10 13:13, Greg Ewing wrote: The way that functions get access to names in enclosing local scopes is by having them passed in as cells, but that mechanism is only available for optimised local namespaces, not ones implemented as dicts. I believe exec already includes the tapdancing

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Reid Kleckner
On Thu, May 27, 2010 at 11:42 AM, Nick Coghlan ncogh...@gmail.com wrote: However, attaining the (sensible) behaviour Colin is requesting when such top level variable references exist would actually be somewhat tricky. Considering Guido's suggestion to treat two argument exec like a function

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Colin H
This option sounds very promising - seems right to do it at the compile stage - i.e. compile(code_str, name, closure) as you have suggested. If there were any argument against, it would be that the most obvious behaviour (function namespace) is the hardest to induce, but the value in knowing

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-27 Thread Colin H
By hardest to induce I mean the default compile exec(code_str, {}, {}) would still be class namespace, but it's pretty insignificant. On Fri, May 28, 2010 at 12:32 AM, Colin H hawk...@gmail.com wrote: This option sounds very promising - seems right to do it at the compile stage - i.e.

[Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Colin H
Hi, issue991196 was closed being described as intentional. I've added a comment in that issue which argues that this is a serious bug (also aserted by a previous commenter - Armin Rigo), because it creates a unique, undocumented, oddly behaving scope that doesn't apply closures correctly. At

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Mark Dickinson
On Wed, May 26, 2010 at 10:15 AM, Colin H hawk...@gmail.com wrote:   issue991196 was closed being described as intentional.  I've added a comment in that issue which argues that this is a serious bug (also aserted by a previous commenter - Armin Rigo), because it creates a unique,

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Nick Coghlan
On 26/05/10 19:48, Mark Dickinson wrote: This is a long way from my area of expertise (I'm commenting here because it was me who sent Colin here in the first place), and it's not clear to me whether this is a bug, and if it is a bug, how it could be resolved. What would the impact be of having

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Thomas Wouters
On Wed, May 26, 2010 at 11:48, Mark Dickinson dicki...@gmail.com wrote: On Wed, May 26, 2010 at 10:15 AM, Colin H hawk...@gmail.com wrote: issue991196 was closed being described as intentional. I've added a comment in that issue which argues that this is a serious bug (also aserted by a

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Michael Foord
On 26/05/2010 13:51, Nick Coghlan wrote: On 26/05/10 19:48, Mark Dickinson wrote: This is a long way from my area of expertise (I'm commenting here because it was me who sent Colin here in the first place), and it's not clear to me whether this is a bug, and if it is a bug, how it could be

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Nick Coghlan
On 26/05/10 23:08, Michael Foord wrote: On 26/05/2010 13:51, Nick Coghlan wrote: On 26/05/10 19:48, Mark Dickinson wrote: This is a long way from my area of expertise (I'm commenting here because it was me who sent Colin here in the first place), and it's not clear to me whether this is a bug,

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Colin H
Thanks for the details on why the observed behaviour occurs - very clear. My only query would be why this is considered correct? Why is it running as a class namespace, when it is not a class? Is there any reason why this is not considered a mistake? Slightly concerned that this is being

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Terry Reedy
Mark Dickinson wrote (with interactice prompts removed) code = \ y = 3 def f(): return y . f() exec code in {} # works fine exec code in {}, {} # dies with a NameError Traceback (most recent call last): File stdin, line 1, in module File string, line 4, in module File string, line

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Colin H
The changes to the docs will definitely help in understanding why this behaves as it does. I would like like to take one last stab though at justifying why this behaviour isn't correct - will leave it alone if these arguments don't stack up :) Appreciate the input and discussion. Terry Jan Reedy

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Greg Ewing
Mark Dickinson wrote: code = \ ... y = 3 ... def f(): ... return y ... f() ... exec code in {} # works fine exec code in {}, {} # dies with a NameError Seems to me the whole idea of being able to specify separate global and local scopes for top-level code is screwy in the first

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Michael Foord
On 27/05/2010 00:38, Greg Ewing wrote: Mark Dickinson wrote: code = \ ... y = 3 ... def f(): ... return y ... f() ... exec code in {} # works fine exec code in {}, {} # dies with a NameError Seems to me the whole idea of being able to specify separate global and local scopes for

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Guido van Rossum
Let me quickly jump in before someone actually deletes the feature. Nick Coghlan and Thomas Wouters had it right; there is still a use case. Don't kill it -- documenting it better is of course fine. It *might* be possible to add a closure to the definition of f in the case where globals !=

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Nick Coghlan
On 27/05/10 06:07, Colin H wrote: In original Python, the snippet would have given an error whether you thought of it as being in a class or function context, which is how anyone who knew Python then would have expected. Consistency is not a bug. When nested function namespaces were

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Colin H
Mark Dickinson wrote: Seems to me the whole idea of being able to specify separate global and local scopes for top-level code is screwy in the first place. Are there any use cases for it? Maybe the second scope argument to exec() should be deprecated? It is running as class namespace that

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Guido van Rossum
On Wed, May 26, 2010 at 5:12 PM, Nick Coghlan ncogh...@gmail.com wrote: On 27/05/10 06:07, Colin H wrote: In original Python, the snippet would have given an error whether you thought of it as being in a class or function context, which is how anyone who knew Python then would have expected.

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Guido van Rossum
On Wed, May 26, 2010 at 5:37 PM, Colin H hawk...@gmail.com wrote: Mark Dickinson wrote: Seems to me the whole idea of being able to specify separate global and local scopes for top-level code is screwy in the first place. Are there any use cases for it? Maybe the second scope argument to

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Colin H
Hi Guido, Thanks for the possible workaround - unfortunately 'stuff' will contain a whole stack of things that are not in 'context', and were not defined in 'user_code' - things that python embeds - a (very small) selection - {..., 'NameError': type 'exceptions.NameError', 'BytesWarning':

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Guido van Rossum
On Wed, May 26, 2010 at 5:53 PM, Colin H hawk...@gmail.com wrote:   Thanks for the possible workaround - unfortunately 'stuff' will contain a whole stack of things that are not in 'context', and were not defined in 'user_code' - things that python embeds - a (very small) selection - {...,

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Colin H
Of course :) - I need to pay more attention. Your workaround should do the trick. It would make sense if locals could be used for this purpose, but the workaround doesn't add so much overhead in most situations. Thanks for the help, much appreciated, Colin On Thu, May 27, 2010 at 2:05 AM, Guido

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Greg Ewing
On 27/05/10 11:33, Michael Foord wrote: On 27/05/2010 00:38, Greg Ewing wrote: Maybe the second scope argument to exec() should be deprecated? Sounds good to me, certainly ends the confusion over this undoubtedly unintuitive behaviour. :-) Although it's a fair point that it can be useful

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Greg Ewing
On 27/05/10 12:37, Colin H wrote: This is a major use case for exec() - defining code from strings (e.g. enabling you to store python code in the database), and using it at runtime. It seems to me this must have been the point of locals in the first place. I suspect that originally it just

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Greg Ewing
On 27/05/10 12:38, Guido van Rossum wrote: the compiler normally uses syntactic clues to decide whether to generate code using closures, in particular, the presence of nested functions. Well, the compiler could be passed a flag indicating that the code is being compiled for an exec statement.

Re: [Python-Dev] variable name resolution in exec is incorrect

2010-05-26 Thread Greg Ewing
Another approach to all this might be to generalise the mechanism by which a lookup of the globals falls back to a lookup of __builtins__. If this were done recursively, then the stuff could be attached to the globals dict, e.g. stuff['__builtins__'] = __builtins__ g = dict(__builtins__ =