STINNER Victor <vstin...@python.org> added the comment:
In short, cloudpickle recreates a function in two steps:
* Create a function object: globals doesn't contain "__builtins__" key
* Update the function attributes (especially __globals__) with
cloudpickle.cloudpicke_fast._function_setstate() which always sets
"__builtins__" in the function __globals__ dictionary.
def _function_setstate(obj, state):
...
state, slotstate = state
...
obj_globals = slotstate.pop("__globals__")
...
obj.__globals__.update(obj_globals)
obj.__globals__["__builtins__"] = __builtins__
...
The internal PyFunctionObject.func_builtins can only by set when a function is
created, it cannot be updated later.
Why not exposing it in funcobject.c (func_memberlist) as we do for closure,
doc, globals and module? So cloudpickle can hack it for its usage.
Documentation of these function attributes:
https://docs.python.org/dev/library/inspect.html#types-and-members
In Python 3.9, _PyFrame_New_NoTrack() starts by looking at builtins in the
parent frame ("back"): use builtins of the parent frame if the parent frame
uses exactly the same dictionary of globals. Then it looks for "__builtins__"
keys in the globals dictionary.
In the current Python 3.10, the builtins are retrieved from the internal
PyFunctionObject.func_builtins member.
Here is the raw pickle bytecode created by cloudpickle:
$ ./python -m pickletools bug.pickle
0: \x80 PROTO 5
2: \x95 FRAME 479
11: \x8c SHORT_BINUNICODE 'cloudpickle.cloudpickle'
36: \x94 MEMOIZE (as 0)
37: \x8c SHORT_BINUNICODE '_builtin_type'
52: \x94 MEMOIZE (as 1)
53: \x93 STACK_GLOBAL
54: \x94 MEMOIZE (as 2)
55: \x8c SHORT_BINUNICODE 'LambdaType'
67: \x94 MEMOIZE (as 3)
68: \x85 TUPLE1
69: \x94 MEMOIZE (as 4)
70: R REDUCE
71: \x94 MEMOIZE (as 5)
72: ( MARK
73: h BINGET 2
75: \x8c SHORT_BINUNICODE 'CodeType'
85: \x94 MEMOIZE (as 6)
86: \x85 TUPLE1
87: \x94 MEMOIZE (as 7)
88: R REDUCE
89: \x94 MEMOIZE (as 8)
90: ( MARK
91: K BININT1 1
93: K BININT1 0
95: K BININT1 0
97: K BININT1 1
99: K BININT1 2
101: K BININT1 67
103: C SHORT_BINBYTES b't\x00|\x00\x83\x01S\x00'
113: \x94 MEMOIZE (as 9)
114: N NONE
115: \x85 TUPLE1
116: \x94 MEMOIZE (as 10)
117: \x8c SHORT_BINUNICODE 'len'
122: \x94 MEMOIZE (as 11)
123: \x85 TUPLE1
124: \x94 MEMOIZE (as 12)
125: \x8c SHORT_BINUNICODE 's'
128: \x94 MEMOIZE (as 13)
129: \x85 TUPLE1
130: \x94 MEMOIZE (as 14)
131: \x8c SHORT_BINUNICODE
'/home/vstinner/python/master/cloudpickle_bug.py'
180: \x94 MEMOIZE (as 15)
181: \x8c SHORT_BINUNICODE 'func'
187: \x94 MEMOIZE (as 16)
188: K BININT1 4
190: C SHORT_BINBYTES b'\x00\x01'
194: \x94 MEMOIZE (as 17)
195: ) EMPTY_TUPLE
196: ) EMPTY_TUPLE
197: t TUPLE (MARK at 90)
198: \x94 MEMOIZE (as 18)
199: R REDUCE
200: \x94 MEMOIZE (as 19)
201: } EMPTY_DICT
202: \x94 MEMOIZE (as 20)
203: ( MARK
204: \x8c SHORT_BINUNICODE '__package__'
217: \x94 MEMOIZE (as 21)
218: N NONE
219: \x8c SHORT_BINUNICODE '__name__'
229: \x94 MEMOIZE (as 22)
230: \x8c SHORT_BINUNICODE '__main__'
240: \x94 MEMOIZE (as 23)
241: \x8c SHORT_BINUNICODE '__file__'
251: \x94 MEMOIZE (as 24)
252: h BINGET 15
254: u SETITEMS (MARK at 203)
255: N NONE
256: N NONE
257: N NONE
258: t TUPLE (MARK at 72)
259: \x94 MEMOIZE (as 25)
260: R REDUCE
261: \x94 MEMOIZE (as 26)
262: \x8c SHORT_BINUNICODE 'cloudpickle.cloudpickle_fast'
292: \x94 MEMOIZE (as 27)
293: \x8c SHORT_BINUNICODE '_function_setstate'
313: \x94 MEMOIZE (as 28)
314: \x93 STACK_GLOBAL
315: \x94 MEMOIZE (as 29)
316: h BINGET 26
318: } EMPTY_DICT
319: \x94 MEMOIZE (as 30)
320: } EMPTY_DICT
321: \x94 MEMOIZE (as 31)
322: ( MARK
323: h BINGET 22
325: h BINGET 16
327: \x8c SHORT_BINUNICODE '__qualname__'
341: \x94 MEMOIZE (as 32)
342: h BINGET 16
344: \x8c SHORT_BINUNICODE '__annotations__'
361: \x94 MEMOIZE (as 33)
362: } EMPTY_DICT
363: \x94 MEMOIZE (as 34)
364: \x8c SHORT_BINUNICODE '__kwdefaults__'
380: \x94 MEMOIZE (as 35)
381: N NONE
382: \x8c SHORT_BINUNICODE '__defaults__'
396: \x94 MEMOIZE (as 36)
397: N NONE
398: \x8c SHORT_BINUNICODE '__module__'
410: \x94 MEMOIZE (as 37)
411: h BINGET 23
413: \x8c SHORT_BINUNICODE '__doc__'
422: \x94 MEMOIZE (as 38)
423: N NONE
424: \x8c SHORT_BINUNICODE '__closure__'
437: \x94 MEMOIZE (as 39)
438: N NONE
439: \x8c SHORT_BINUNICODE '_cloudpickle_submodules'
464: \x94 MEMOIZE (as 40)
465: ] EMPTY_LIST
466: \x94 MEMOIZE (as 41)
467: \x8c SHORT_BINUNICODE '__globals__'
480: \x94 MEMOIZE (as 42)
481: } EMPTY_DICT
482: \x94 MEMOIZE (as 43)
483: u SETITEMS (MARK at 322)
484: \x86 TUPLE2
485: \x94 MEMOIZE (as 44)
486: \x86 TUPLE2
487: R REDUCE
488: 0 POP
489: . STOP
highest protocol among opcodes = 4
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue43228>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
