New submission from Mark Dickinson <[email protected]>:
tl;dr - contrived (but relatively short) code involving nested try/finally
blocks can produce disproportionately large bytecode. I'm not expecting or
suggesting any action here, but the situation seemed at least worth noting.
Feel free to close this issue as a "well don't do that, then" (a.k.a. "wont
fix")
Longer: Python 3.9 changed the way that bytecode was generated for try/finally
(see #33387). For a "try" block body that can do any of raise, return or
fall-off-the-end-of-the-block, the corresponding finally block gets three
separate paths in the bytecode. If such trys are nested <n> times, we get 3^n
separate paths in the bytecode.
Example code:
----------------
def f():
try:
if something(): return
finally:
try:
if something(): return
finally:
try:
if something(): return
finally:
try:
if something(): return
finally:
try:
if something(): return
finally:
do_cleanup()
import dis
dis.dis(f)
----------------
On my machine, running this and counting the do_cleanup invocations gives, as
expected, a result of 243 = 3**5
% python3.9 nested_finally.py | grep do_cleanup | wc -l
243
That's fairly benign, but if I scale up to 10 nested blocks, the dis.dis call
takes over 10 minutes to complete (the compilation itself still only takes a
fraction of a second). The bytecode object is correspondingly large:
>>> len(f.__code__.co_code)
1741356
With 15 levels of nesting, compilation takes several seconds, and
the generated code is (again as expected) a couple of orders of magnitude
larger:
>>> len(f.__code__.co_code)
533859040
I didn't try pushing this further than 15 levels of nesting.
As I said above, it's not clear to me whether this is actually an issue that
needs to be addressed in practice. It seems unlikely that "real code" :TM:
would run into this, but the effect seemed worth noting.
----------
components: Interpreter Core
messages: 384718
nosy: Mark.Shannon, mark.dickinson
priority: normal
severity: normal
status: open
title: Exponential time and space requirements for compilation of nested
try/finally blocks
type: performance
versions: Python 3.10, Python 3.9
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue42873>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
