New issue 2779: Performance loss with deeply nested generators https://bitbucket.org/pypy/pypy/issues/2779/performance-loss-with-deeply-nested
Hrvoje Nikšić: PyPy (5.10.1) is significantly slower than CPython (3.6.3) on the attached benchmark. The script profiles traversing generators nested with `yield from` at various depths, which also comes up when suspending and resuming recursive or nested `async` functions. I was curious whether PyPy optimizes `yield from` delegation as mentioned [in the original PEP](https://www.python.org/dev/peps/pep-0380/#optimisations) and later in a [message by Guido](https://groups.google.com/forum/#!msg/python-tulip/bmphRrryuFk/BNEOOOuR1j4J). It seems that CPython [dropped](https://bugs.python.org/issue14230) the optimization due to an issue with tracebacks. It looks like PyPy doesn't implement the optimization either - but that is not the topic of this issue. I noticed a sharp slowdown in PyPy at the nesting depth of around 80. At that point the performance suddenly drops to levels significantly slower than CPython. For example, on my machine, with depth == 100: ``` $ python3 benchnestedyield.py 100 0.061139583587646484 0.06133460998535156 0.06076836585998535 0.06116318702697754 0.061196327209472656 $ pypy3 benchnestedyield.py 100 0.30632662773132324 0.31490111351013184 0.29983067512512207 0.31607627868652344 0.31775617599487305 $ pypy3 --jit off benchnestedyield.py 100 0.16022372245788574 0.1604137420654297 0.16206026077270508 0.161407470703125 0.16135311126708984 ``` With depth == 50, the numbers look like something I'd expect when comparing PyPy and CPython: ``` $ python3 benchnestedyield.py 50 0.029144287109375 0.029152870178222656 0.02940988540649414 0.02889084815979004 0.028936386108398438 $ pypy3 benchnestedyield.py 50 0.12029409408569336 0.005949974060058594 0.005345821380615234 0.005307197570800781 0.005133867263793945 $ pypy3 --jit off benchnestedyield.py 50 0.08134770393371582 0.08192563056945801 0.08233189582824707 0.08091974258422852 0.08127617835998535 ``` This is probably not a big issue in practice, but I wanted to report it in case it's an anomaly someone cares to investigate. _______________________________________________ pypy-issue mailing list pypy-issue@python.org https://mail.python.org/mailman/listinfo/pypy-issue
