Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
Ah yes, I see what you mean:
class Test():
x = 1
print (x) # Prints 1
print([x+i for i in range(1,3)]) # NameError (x)
Anyway, I apologise for posting to Python-Dev on was a known issue, and
turned out to be more me asking for help with development with Python,
rather than development of Python. (My original use case was a
scripting language that could contain embedded Python code). Thanks to
Nick for his original answer.
Rob Cliffe
On 11/06/2018 23:31, Eric Fahlgren wrote:
On Mon, Jun 11, 2018 at 3:10 PM Rob Cliffe via Python-Dev
mailto:python-dev@python.org>> wrote:
Skip, I think you have misunderstood the point I was making. It was
not whether the loop variable should leak out of a list
comprehension.
Rather, it was whether a local variable should, so to speak, "leak
into"
a list comprehension. And the answer is: it depends on whether
the code
is executed normally, or via exec/eval. Example:
def Test():
x = 1
print([x+i for i in range(1,3)]) # Prints [2,3]
exec('print([x+i for i in range(1,3)])') # Raises NameError (x)
Test()
I (at least at first) found the difference in behaviour surprising.
Change 'def' to 'class' and run it again. You'll be even more
surprised.
Virus-free. www.avg.com
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
Skip Montanaro wrote: Yes, you'll have to pass in locals to exec. Exec changed between python 2 and 3. It used to be treated specially by the compiler so that it could see and modify the locals where it was used. But now it's just an ordinary function, so you can't expect it to magically know about anything that's not passed into it. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
On Mon, Jun 11, 2018 at 3:10 PM Rob Cliffe via Python-Dev <
python-dev@python.org> wrote:
> Skip, I think you have misunderstood the point I was making. It was
> not whether the loop variable should leak out of a list comprehension.
> Rather, it was whether a local variable should, so to speak, "leak into"
> a list comprehension. And the answer is: it depends on whether the code
> is executed normally, or via exec/eval. Example:
>
> def Test():
>x = 1
>print([x+i for i in range(1,3)]) # Prints [2,3]
>exec('print([x+i for i in range(1,3)])') # Raises NameError (x)
> Test()
>
> I (at least at first) found the difference in behaviour surprising.
>
Change 'def' to 'class' and run it again. You'll be even more surprised.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
> Skip, I think you have misunderstood the point I was making. It was > not whether the loop variable should leak out of a list comprehension. > Rather, it was whether a local variable should, so to speak, "leak into" > a list comprehension. And the answer is: it depends on whether the code > is executed normally, or via exec/eval. > Got it. Yes, you'll have to pass in locals to exec. (Can't verify, as I'm on the train, on my phone.) Builtins like range are global to everything, so no problem there. Your clarification also make it more of a Python programming question , I think. Skip > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
Skip, I think you have misunderstood the point I was making. It was
not whether the loop variable should leak out of a list comprehension.
Rather, it was whether a local variable should, so to speak, "leak into"
a list comprehension. And the answer is: it depends on whether the code
is executed normally, or via exec/eval. Example:
def Test():
x = 1
print([x+i for i in range(1,3)]) # Prints [2,3]
exec('print([x+i for i in range(1,3)])') # Raises NameError (x)
Test()
I (at least at first) found the difference in behaviour surprising.
Regards
Rob Cliffe
On 08/06/2018 19:27, Skip Montanaro wrote:
Is this a bug or a feature?
The bug was me being so excited about the new construct (I pushed in
someone else's work, can't recall who now, maybe Fredrik Lundh?) that
I didn't consider that leaking the loop variable out of the list
comprehension was a bad idea. Think of the Py3 behavior as one of
those "corrections" to things which were "got wrong" in Python 1 or 2.
:-)
Skip
---
This email has been checked for viruses by AVG.
http://www.avg.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
> Is this a bug or a feature? The bug was me being so excited about the new construct (I pushed in someone else's work, can't recall who now, maybe Fredrik Lundh?) that I didn't consider that leaking the loop variable out of the list comprehension was a bad idea. Think of the Py3 behavior as one of those "corrections" to things which were "got wrong" in Python 1 or 2. :-) Skip ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
On 06/06/2018 03:51 PM, Nick Coghlan wrote: > On 6 June 2018 at 15:31, Rob Cliffe via Python-Dev mailto:python-dev@python.org>> wrote: > ... > *In other words, it looks as if in Python 3.6.5, the compiled list comprehension** > **can "see" a pre-existing global variable but not a local one.* > > Yes, this is expected behaviour - the two-namespace form of exec (which is what you get implicitly when you use it inside a function body) is similar to a class body, and hence nested functions > (including the implicit ones created for comprehensions) can't see the top level local variables. In issue 13557 [1] Amaury gives the following explanation by quoting the documentation [2] "Free variables are not resolved in the nearest enclosing namespace, but in the global namespace" and hints at the same solution that is proposed by Nick. FWIW in issue 21161 [3] folks have been bitten by this when trying to run a list comprehension in pdb. [1] https://bugs.python.org/issue13557 [2] http://docs.python.org/py3k/reference/executionmodel.html#interaction-with-dynamic-features [3] https://bugs.python.org/issue21161 Xavier ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
On 6 June 2018 at 15:31, Rob Cliffe via Python-Dev
wrote:
> Is this a bug or a feature?
> Consider the following program:
>
> # TestProgram.py
> def Test():
> # global x
> x = 1
> exec(compile('print([x+1,x+2])', 'MyTest', 'exec'))
> exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'))
> Test()
>
> In Python 2.7.15 the output is
>
> [2, 3]
> [2, 3]
>
> In Python 3.6.5 the output is
> [2, 3]
> Traceback (most recent call last):
> File "TestProgram.py", line 7, in
> Test()
> File "TestProgram.py", line 6, in Test
> exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'))
> File "MyTest", line 1, in
> File "MyTest", line 1, in
> NameError: name 'x' is not defined
>
> If the "global x" declaration is uncommented, this "fixes" the Python
> 3.6.5 behaviour,
> i.e. no error occurs and the output is the same as for Python 2.7.15.
>
> *In other words, it looks as if in Python 3.6.5, the compiled list
> comprehension*
> *can "see" a pre-existing global variable but not a local one.*
>
Yes, this is expected behaviour - the two-namespace form of exec (which is
what you get implicitly when you use it inside a function body) is similar
to a class body, and hence nested functions (including the implicit ones
created for comprehensions) can't see the top level local variables.
You can override that and force the use of the single-namespace form by
passing the locals() namespace into exec() explicitly:
def explicit_local_namespace():
x = 1
exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'),
locals())
explicit_local_namespace()
(Note: you'll then need to use collections.ChainMap instead of separate
locals and globals namespaces if you want the exec'ed code to still be able
to see the module globals in addition to the function locals)
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python3 compiled listcomp can't see local var - bug or feature?
Is this a bug or a feature?
Consider the following program:
# TestProgram.py
def Test():
# global x
x = 1
exec(compile('print([x+1,x+2])', 'MyTest', 'exec'))
exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'))
Test()
In Python 2.7.15 the output is
[2, 3]
[2, 3]
In Python 3.6.5 the output is
[2, 3]
Traceback (most recent call last):
File "TestProgram.py", line 7, in
Test()
File "TestProgram.py", line 6, in Test
exec(compile('print([x+i for i in range(1,3)])', 'MyTest', 'exec'))
File "MyTest", line 1, in
File "MyTest", line 1, in
NameError: name 'x' is not defined
If the "global x" declaration is uncommented, this "fixes" the Python
3.6.5 behaviour,
i.e. no error occurs and the output is the same as for Python 2.7.15.
*In other words, it looks as if in Python 3.6.5, the compiled list
comprehension**
**can "see" a pre-existing global variable but not a local one.*
I have used dis to examine the code objects returned by compile()
(they are the same with or without the "global x"):
Python 2.7.15 first code object from 'print([x+1,x+2])':
1 0 LOAD_NAME 0 (x)
3 LOAD_CONST 0 (1)
6 BINARY_ADD
7 LOAD_NAME 0 (x)
10 LOAD_CONST 1 (2)
13 BINARY_ADD
14 BUILD_LIST 2
17 PRINT_ITEM
18 PRINT_NEWLINE
19 LOAD_CONST 2 (None)
22 RETURN_VALUE
Python 2.7.15 second code object from 'print([x+i for i in range(1,3)])':
1 0 BUILD_LIST 0
3 LOAD_NAME 0 (range)
6 LOAD_CONST 0 (1)
9 LOAD_CONST 1 (3)
12 CALL_FUNCTION 2
15 GET_ITER
>> 16 FOR_ITER 16 (to 35)
19 STORE_NAME 1 (i)
22 LOAD_NAME 2 (x)
25 LOAD_NAME 1 (i)
28 BINARY_ADD
29 LIST_APPEND 2
32 JUMP_ABSOLUTE 16
>> 35 PRINT_ITEM
36 PRINT_NEWLINE
37 LOAD_CONST 2 (None)
40 RETURN_VALUE
Python 3.6.5 first code object from 'print([x+1,x+2])':
1 0 LOAD_NAME 0 (print)
2 LOAD_NAME 1 (x)
4 LOAD_CONST 0 (1)
6 BINARY_ADD
8 LOAD_NAME 1 (x)
10 LOAD_CONST 1 (2)
12 BINARY_ADD
14 BUILD_LIST 2
16 CALL_FUNCTION 1
18 POP_TOP
20 LOAD_CONST 2 (None)
22 RETURN_VALUE
Python 3.6.5 second code object from 'print([x+i for i in range(1,3)])':
1 0 LOAD_NAME 0 (print)
2 LOAD_CONST 0 ( at
0x029F79C0, file "MyTest", line 1>)
4 LOAD_CONST 1 ('')
6 MAKE_FUNCTION 0
8 LOAD_NAME 1 (range)
10 LOAD_CONST 2 (1)
12 LOAD_CONST 3 (3)
14 CALL_FUNCTION 2
16 GET_ITER
18 CALL_FUNCTION 1
20 CALL_FUNCTION 1
22 POP_TOP
24 LOAD_CONST 4 (None)
26 RETURN_VALUE
You will see that in Python 3.6.5 the dis output for the second code object
does not show the internals of the listcomp, and in particular whether,
and how, it refers to the variable 'x'. I don't know how to investigate
further.
Best wishes
Rob Cliffe
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
