[issue44028] Request for locals().update() to work, it is

[wang xuancong]

wang xuancong  added the comment:

Thanks @terry.reedy for your expert-level good comments!

1.
"In Python 3, the one *implementation*, and its lookup mode, are fixed.  The 
slower implementation was dropped because it was not thought worth the bother."
If I remember correctly, the performance penalty due to the slower lookup mode 
is not quite significant, in most Python benchmarks, Python2 still performs 
much faster than Python3 because most codes that need speed does not contain 
exec/eval, so the slow mode won't affect in practice.

2.
"When you invoke the save function while playing a game, I am imagine that the 
save function does not have access to and does not same the locals of whatever 
function was executing at the time you hit the save key.  Rather a game and 
player states are serialized, and likely not in one line of code."
I have personally tried this on one implementation of deep neural network using 
Tensorflow, it works pretty well, especially on saving the network parameters 
at every Nth epoch, or resuming training from a particular epoch. The biggest 
advantage is that it does not scale with network size or complexity, so the 
Python code size has a O(1) complexity with network size/complexity and that is 
a small constant O(1) as it does not involve any Python loop. In practice, you 
can select what to save/load, such as those not starting with '_'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44028] Request for locals().update() to work, it is

[Terry J. Reedy]

Terry J. Reedy  added the comment:

When quoting, please include the source, preferably a direct link.

In Python 1 and 2, there were two function local namespace *implementations* -- 
array index by number or dictionary indexed by name.  The implementation, and 
hence the lookup mode, was fixed at compile time.  In Python 3, the one 
*implementation*, and its lookup mode, are fixed.  The slower implementation 
was dropped because it was not thought worth the bother.  

When you invoke the save function while playing a game, I am imagine that the 
save function does not have access to and does not same the locals of whatever 
function was executing at the time you hit the save key.  Rather a game and 
player states are serialized, and likely not in one line of code.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44028] Request for locals().update() to work, it is

[wang xuancong]

wang xuancong  added the comment:

Of course, I am aware of that. As elite-level Python programmers, we should all 
be aware of security issues whenever we deal with exec() and eval().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44028] Request for locals().update() to work, it is

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> loading the entire game or DNN (from STDIN) can be simply put into one line 
> as `locals().update(eval(sys.stdin.read()))`

This is how you get command injection attacks.

https://owasp.org/www-community/attacks/Command_Injection

https://cwe.mitre.org/data/definitions/77.html

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44028] Request for locals().update() to work, it is

[wang xuancong]

New submission from wang xuancong :

In general, the ability to update local variables is very important and it 
simplifies life tremendously. For example, in machine learning, it allows 
saving/loading arbitrary-format datasets and arbitrary-structure neural 
networks (NN) using a single line of code. In computer games, no matter how 
many complex data structures are there, saving and loading can be done in a 
single line.

Imagine you are game developer or a deep neural network (DNN) researcher, if 
all local variables are serializable, then no matter how complicated your game 
or your DNN structure is, saving the entire game or DNN (to STDOUT) can be 
simply put into one line as `print(locals())`, and loading the entire game or 
DNN (from STDIN) can be simply put into one line as 
`locals().update(eval(sys.stdin.read()))`.

Currently, `globals().update(...)` takes immediate effect but 
`locals().update(...)` does not work because Python documentation says:

> The default locals act as described for function locals() below:
> modifications to the default locals dictionary should not be
> attempted. Pass an explicit locals dictionary if you need to see
> effects of the code on locals after function exec() returns.

Why they design Python in such way is because of optimization and conforming 
the `exec` statement into a function:

> To modify the locals of a function on the fly is not possible without
> several consequences: normally, function locals are not stored in a
> dictionary, but an array, whose indices are determined at compile time
> from the known locales. This collides at least with new locals added
> by exec. The old exec statement circumvented this, because the
> compiler knew that if an exec without globals/locals args occurred in
> a function, that namespace would be "unoptimized", i.e. not using the
> locals array. Since exec() is now a normal function, the compiler does
> not know what "exec" may be bound to, and therefore can not treat is
> specially.

Since `global().update(...)` works, the following piece of code will work in 
root namespace (i.e., outside any function) because locals() is the same as 
globals() in root namespace:
```
locals().update({'a':3, 'b':4})
print(a, b)
```
But this will not work inside a function.

I have explored a few ways of hacking updating locals() on Python 3, it seems 
there is no way so far. The following piece of code seems to works:
```
def f1():
  sys._getframe(1).f_locals.update({'a':3, 'b':4})
  print(a, b)

f1()
```
However, that is because `sys._getframe(1)` is the root namespace, so 
`sys._getframe(1).f_locals.update()` is essentially `globals().update()`.

>From the above Python developer documentation, I understand that in Python 2, 
>local namespace lookup has 2 modes: optimized mode if there is no `exec` 
>statement, un-optimized mode if there exists an `exec` statement. But in 
>Python 3, `exec` becomes a function, so the compiler cannot determine which 
>namespace optimization mode at compile time (because `exec` can be overridden 
>or aliased into a different name). Therefore, Python 3 has only optimized 
>namespace lookup. My suggestion is that whenever this optimized local 
>namespace lookup fails, perform an un-optimized lookup (which will include 
>locals()). This should solve the problem.

Do you have any other ideas or suggestions for doing this? Thanks!

--
components: Interpreter Core
messages: 392852
nosy: xuancong84
priority: normal
severity: normal
status: open
title: Request for locals().update() to work, it is
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com