Does CPython already has Peephole optimizations?
Hello All, I read about this article: http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/montanaro.html Just wanted to clarify whether CPython already includes these kind of byte code optimizations? Are all the temporary variables removed when byte code is generated? Regards, Laxmikant -- https://mail.python.org/mailman/listinfo/python-list
Re: Does CPython already has Peephole optimizations?
Laxmikant Chitare wrote: Hello All, I read about this article: http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/montanaro.html Just wanted to clarify whether CPython already includes these kind of byte code optimizations? Are all the temporary variables removed when byte code is generated? You can find out for yourself: Python 3.4.0rc1+ (default:2ba583191550+, Feb 12 2014, 00:08:44) [GCC 4.6.1] on linux Type help, copyright, credits or license for more information. import dis def f(): ... a, b, c = 1, 2, 3 ... dis.dis(f) 2 0 LOAD_CONST 4 ((1, 2, 3)) 3 UNPACK_SEQUENCE 3 6 STORE_FAST 0 (a) 9 STORE_FAST 1 (b) 12 STORE_FAST 2 (c) 15 LOAD_CONST 0 (None) 18 RETURN_VALUE def g(): ... q = 2 + 3j ... dis.dis(g) 2 0 LOAD_CONST 3 ((2+3j)) 3 STORE_FAST 0 (q) 6 LOAD_CONST 0 (None) 9 RETURN_VALUE If you can read C there is also http://hg.python.org/cpython/file/180e4b678003/Python/peephole.c -- https://mail.python.org/mailman/listinfo/python-list
Re: Does CPython already has Peephole optimizations?
On Mon, 17 Feb 2014 13:54:25 +0530, Laxmikant Chitare wrote: I read about this article: http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/ montanaro.html Just wanted to clarify whether CPython already includes these kind of byte code optimizations? Are all the temporary variables removed when byte code is generated? You can check these things for yourself: import dis dis.dis(function) will show you the byte code. But in general, I would expect not. CPython (that's the Python you probably use) doesn't do a lot of optimization apart from some simple constant folding. If you're interested in optimizing Python, you should look at the JIT optimizing Python compiler, PyPy. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: Does CPython already has Peephole optimizations?
On 2/17/14 3:59 AM, Steven D'Aprano wrote: On Mon, 17 Feb 2014 13:54:25 +0530, Laxmikant Chitare wrote: I read about this article: http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/ montanaro.html Just wanted to clarify whether CPython already includes these kind of byte code optimizations? Are all the temporary variables removed when byte code is generated? You can check these things for yourself: import dis dis.dis(function) will show you the byte code. But in general, I would expect not. CPython (that's the Python you probably use) doesn't do a lot of optimization apart from some simple constant folding. If you're interested in optimizing Python, you should look at the JIT optimizing Python compiler, PyPy. CPython does some constant folding, and also jump optimizations. In my role as coverage.py maintainer, I would love to see a way to disable all those optimizations. I tried filing a bug about it (http://bugs.python.org/issue2506), but it did not win the popular support I had hoped for. -- Ned Batchelder, http://nedbatchelder.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Does CPython already has Peephole optimizations?
On 2/17/2014 3:59 AM, Steven D'Aprano wrote: On Mon, 17 Feb 2014 13:54:25 +0530, Laxmikant Chitare wrote: I read about this article: http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/ montanaro.html Just wanted to clarify whether CPython already includes these kind of byte code optimizations? Most of the easily seen and obviously safe low-hanging fruits for the compile step have been plucked. Note that the effect of the peephole process would only save a few percent, if any, for real apps*. Improving the C code invoked by bytecode has resulted in much larger gains. * We now have a much better benchmark suite with some real apps. This is thanks in part to the pypy project. Are all the temporary variables removed when byte code is generated? You can check these things for yourself: import dis dis.dis(function) will show you the byte code. But in general, I would expect not. CPython (that's the Python you probably use) doesn't do a lot of optimization apart from some simple constant folding. If you're interested in optimizing Python, you should look at the JIT optimizing Python compiler, PyPy. For CPython, new optimization has mostly moved to AST tranformations prior to compilation. (Python ASTs are new since Skip started the peephole work.) I believe there are some open issues on the tracker. Once optimization constraint Skip did not mention is the correspondence between source lines and blocks of bytecode, which is used by profiling, tracing, and tracebacks. Effectively transforming if type(a) == types.ComplexType: x = cmath.sin(a) foo(x) else: x = math.sin(a) foo(x) into if type(a) == types.ComplexType: x = cmath.sin(a) else: x = math.sin(a) foo(x) breaks the correspondence. If foo(x) raises, which original line should be reported as the source of the exception? -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Does CPython already has Peephole optimizations?
Thank you all for the enlightening inputs. I have learnt a lot just with this one question. Great to know about dis library. Ned, from explanation I now realize how important it is to do impact analysis. Things are not always rosy :). I have always appreciated everyone over this list. This is just another opportunity. Best regards, Laxmikant On Mon, Feb 17, 2014 at 7:21 PM, Terry Reedy tjre...@udel.edu wrote: On 2/17/2014 3:59 AM, Steven D'Aprano wrote: On Mon, 17 Feb 2014 13:54:25 +0530, Laxmikant Chitare wrote: I read about this article: http://www.python.org/workshops/1998-11/proceedings/papers/montanaro/ montanaro.html Just wanted to clarify whether CPython already includes these kind of byte code optimizations? Most of the easily seen and obviously safe low-hanging fruits for the compile step have been plucked. Note that the effect of the peephole process would only save a few percent, if any, for real apps*. Improving the C code invoked by bytecode has resulted in much larger gains. * We now have a much better benchmark suite with some real apps. This is thanks in part to the pypy project. Are all the temporary variables removed when byte code is generated? You can check these things for yourself: import dis dis.dis(function) will show you the byte code. But in general, I would expect not. CPython (that's the Python you probably use) doesn't do a lot of optimization apart from some simple constant folding. If you're interested in optimizing Python, you should look at the JIT optimizing Python compiler, PyPy. For CPython, new optimization has mostly moved to AST tranformations prior to compilation. (Python ASTs are new since Skip started the peephole work.) I believe there are some open issues on the tracker. Once optimization constraint Skip did not mention is the correspondence between source lines and blocks of bytecode, which is used by profiling, tracing, and tracebacks. Effectively transforming if type(a) == types.ComplexType: x = cmath.sin(a) foo(x) else: x = math.sin(a) foo(x) into if type(a) == types.ComplexType: x = cmath.sin(a) else: x = math.sin(a) foo(x) breaks the correspondence. If foo(x) raises, which original line should be reported as the source of the exception? -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
