Re: [pypy-dev] Updated 'High Performance Python' tutorial (the one from EuroPython 2011)
Hi Antonio! Apologies for the slow reply, this got filed into a subfolder. The numbers are interesting, I'm also interested in the C version. I'm hoping that my tutorial will be accepted for PyCon next March (the talks are announced in two weeks), assuming I get to talk again I'll update my tutorial. Adding more for PyPy and having a C equivalent will be very useful. Given that the C version should be very similar to the ShedSkin version, maybe it just comes down to compiler differences? On my Macbook (where I originally wrote the talk) I think the differences in speed came from two versions of gcc (Cython seemed to prefer one, ShedSkin the other, I ran out of time trying to unify that test). Do you definitely use the same optimisation flags? ShedSkin (from memory) requests fast math and a few other things in the generated Makefile. Ian. On 7 November 2011 18:04, Antonio Cuni anto.c...@gmail.com wrote: Hello Ian, On 25/07/11 11:00, Ian Ozsvald wrote: Dear all, I've published v0.2 of my High Performance Python tutorial write-up from the session I ran at EuroPython: http://ianozsvald.com/2011/07/25/high-performance-python-tutorial-v0-2-from-europython-2011/ today I and Armin investigated a bit more about the performances of the mandelbrot algorithm that you wrote for your tutorial. What we found is very interesting :-). We compared three versions of the code: - a (slightly modified) pure python one on PyPy - the Cython one using calculate_z.pyx_2_bettermath - the shedskin one, using shedskin2.py The PyPy version looks like this: def calculate_z_serial_purepython(q, maxiter, z): Pure python with complex datatype, iterating over list of q and z output = [0] * len(q) for i in range(len(q)): zi = z[i] qi = q[i] for iteration in range(maxiter): zi = zi * zi + qi if (zi.real*zi.real + zi.imag*zi.imag) 4.0: output[i] = iteration break return output i.e., it is exactly the same as pure_python_2.py, but we avoid to use abs(zi), so it is comparable with the cython and shedskin version. First, we ran the programs to calculate passing 1000 1000 as arguments, and these are the results: PyPy: 1.95 secs Cython: 0.58 secs Shedskin: 0.42 secs so, PyPy is ~4.5x slower than Shedskin. However, we realized that using the default values for x1,x2,y1,y2, the innermost loop runs very few iterations most of the time, and this is one case in which PyPy suffer most, because it needs to go through a bridge to continue the execution, and at the moment bridges are slower than loops. So, we changed the values of x1,x2,y1,y2 to compute a different region, in which the innermost loop runs more frequently. We used these values: x1, x2, y1, y2 = 0.37865401-0.02, 0.37865401+0.02, 0.669227668-0.02, 0.669227668+0.02 and since all programs are faster to compute the image, we used 3000 3000 as arguments from the command line. These are the results: PyPy: 0.89 Cython: 1.76 Shedskin: 0.26 So, in this case, PyPy is ~2x faster than Cython and ~3.5x slower than Shedskin. In the meantime, Armin wrote a C version of it: http://paste.pocoo.org/raw/504216/ which tooks 0.946 seconds to complete. This is in line with the PyPy's result, but we are still investigating why the shedskin's version is so much faster. ciao, Anto -- Ian Ozsvald (A.I. researcher) i...@ianozsvald.com http://IanOzsvald.com http://MorConsulting.com/ http://StrongSteam.com/ http://SocialTiesApp.com/ http://TheScreencastingHandbook.com http://FivePoundApp.com/ http://twitter.com/IanOzsvald ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Updated 'High Performance Python' tutorial (the one from EuroPython 2011)
Hi, On Tue, Nov 15, 2011 at 15:54, Ian Ozsvald i...@ianozsvald.com wrote: ShedSkin (from memory) requests fast math and a few other things in the generated Makefile. Ah, it is cheating that way. Indeed, I didn't try to play with gcc options; I just used -O2 (or -O3, which made no difference). The C source code is completely obvious and surprise-less. You can see it here (it outputs the raw data to stdout, so you have to pipe it to a converter program to display the result): http://paste.pocoo.org/show/508215/ A bientôt, Armin. ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Updated 'High Performance Python' tutorial (the one from EuroPython 2011)
Hi, 2011/11/15 Armin Rigo ar...@tunes.org: On Tue, Nov 15, 2011 at 15:54, Ian Ozsvald i...@ianozsvald.com wrote: ShedSkin (from memory) requests fast math and a few other things in the generated Makefile. Ah, it is cheating that way. Indeed, I didn't try to play with gcc options; I just used -O2 (or -O3, which made no difference). FYI, here is the default FLAGS file for shedskin: CC=g++ CCFLAGS=-O2 -march=native -Wno-deprecated $(CPPFLAGS) LFLAGS=-lgc -lpcre $(LDFLAGS) But of course you can change the compiler and play with its flags to improve performance. Best regards, -- Jérémie ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
[pypy-dev] Detecting pypy vs cpython runtime
Hi everyone. I have a cellular automata framework in C++ and I use cython and cpython to run it. I found out that if I port it to pure python and run it with pypy, it's close to the same performance as the C++ version. (about 2x as slow, compared to 20x as slow when using pure python + cpython). When I throw in other overheads with pure python libraries, using the pure python and pypy is much faster than cpython with the C++ library, all things equal. What I'd like to do is detect if pypy or cpython is doing the importing of my module, and switch over to the pure python interface if pypy is found. As it stands I have to do it manually in my module's __init__.py. *Is there any way to detect if my module is being imported by pypy vs cpython?* Either via sys, or maybe some latent variable that is present, or something else. sys.argv[0] only has the script name (obviously), not the interpreter call. Keep up the outstanding work. Pypy is great! Thanks! Blaine ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Detecting pypy vs cpython runtime
On Tue, Nov 15, 2011 at 12:56 PM, Blaine frik...@gmail.com wrote: Hi everyone. I have a cellular automata framework in C++ and I use cython and cpython to run it. I found out that if I port it to pure python and run it with pypy, it's close to the same performance as the C++ version. (about 2x as slow, compared to 20x as slow when using pure python + cpython). When I throw in other overheads with pure python libraries, using the pure python and pypy is much faster than cpython with the C++ library, all things equal. What I'd like to do is detect if pypy or cpython is doing the importing of my module, and switch over to the pure python interface if pypy is found. As it stands I have to do it manually in my module's __init__.py. *Is there any way to detect if my module is being imported by pypy vs cpython?* Either via sys, or maybe some latent variable that is present, or something else. sys.argv[0] only has the script name (obviously), not the interpreter call. Keep up the outstanding work. Pypy is great! Thanks! Blaine ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev The canonical way is probably `import platform; platform.python_implementation()` which will return either PyPy or CPython. Alex -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Detecting pypy vs cpython runtime
On Tue, Nov 15, 2011 at 8:03 PM, Alex Gaynor alex.gay...@gmail.com wrote: On Tue, Nov 15, 2011 at 12:56 PM, Blaine frik...@gmail.com wrote: Hi everyone. I have a cellular automata framework in C++ and I use cython and cpython to run it. I found out that if I port it to pure python and run it with pypy, it's close to the same performance as the C++ version. (about 2x as slow, compared to 20x as slow when using pure python + cpython). When I throw in other overheads with pure python libraries, using the pure python and pypy is much faster than cpython with the C++ library, all things equal. What I'd like to do is detect if pypy or cpython is doing the importing of my module, and switch over to the pure python interface if pypy is found. As it stands I have to do it manually in my module's __init__.py. Is there any way to detect if my module is being imported by pypy vs cpython? Either via sys, or maybe some latent variable that is present, or something else. sys.argv[0] only has the script name (obviously), not the interpreter call. Keep up the outstanding work. Pypy is great! Thanks! Blaine ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev The canonical way is probably `import platform; platform.python_implementation()` which will return either PyPy or CPython. Alex If you need to support older pythons (which don't have platform.python_implementation), we use import sys is_pypy = '__pypy__' in sys.builtin_module_names or alternatively: try: import __pypy__ except ImportError: __pypy__ = None -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Detecting pypy vs cpython runtime
On Tue, Nov 15, 2011 at 1:08 PM, Maciej Fijalkowski fij...@gmail.comwrote: On Tue, Nov 15, 2011 at 8:03 PM, Alex Gaynor alex.gay...@gmail.com wrote: On Tue, Nov 15, 2011 at 12:56 PM, Blaine frik...@gmail.com wrote: Hi everyone. I have a cellular automata framework in C++ and I use cython and cpython to run it. I found out that if I port it to pure python and run it with pypy, it's close to the same performance as the C++ version. (about 2x as slow, compared to 20x as slow when using pure python + cpython). When I throw in other overheads with pure python libraries, using the pure python and pypy is much faster than cpython with the C++ library, all things equal. What I'd like to do is detect if pypy or cpython is doing the importing of my module, and switch over to the pure python interface if pypy is found. As it stands I have to do it manually in my module's __init__.py. Is there any way to detect if my module is being imported by pypy vs cpython? Either via sys, or maybe some latent variable that is present, or something else. sys.argv[0] only has the script name (obviously), not the interpreter call. Keep up the outstanding work. Pypy is great! Thanks! Blaine ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev The canonical way is probably `import platform; platform.python_implementation()` which will return either PyPy or CPython. Alex If you need to support older pythons (which don't have platform.python_implementation), we use import sys is_pypy = '__pypy__' in sys.builtin_module_names or alternatively: try: import __pypy__ except ImportError: __pypy__ = None -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev And don't forget `hasattr(sys, pypy_translation_info)` just for completeness, platform is the cleanest IMO though :) Alex -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] Detecting pypy vs cpython runtime
Awesome. Thanks everyone, exactly what I was looking for. Thank you! Blaine On Tue, Nov 15, 2011 at 1:10 PM, Alex Gaynor alex.gay...@gmail.com wrote: On Tue, Nov 15, 2011 at 1:08 PM, Maciej Fijalkowski fij...@gmail.comwrote: On Tue, Nov 15, 2011 at 8:03 PM, Alex Gaynor alex.gay...@gmail.com wrote: On Tue, Nov 15, 2011 at 12:56 PM, Blaine frik...@gmail.com wrote: Hi everyone. I have a cellular automata framework in C++ and I use cython and cpython to run it. I found out that if I port it to pure python and run it with pypy, it's close to the same performance as the C++ version. (about 2x as slow, compared to 20x as slow when using pure python + cpython). When I throw in other overheads with pure python libraries, using the pure python and pypy is much faster than cpython with the C++ library, all things equal. What I'd like to do is detect if pypy or cpython is doing the importing of my module, and switch over to the pure python interface if pypy is found. As it stands I have to do it manually in my module's __init__.py. Is there any way to detect if my module is being imported by pypy vs cpython? Either via sys, or maybe some latent variable that is present, or something else. sys.argv[0] only has the script name (obviously), not the interpreter call. Keep up the outstanding work. Pypy is great! Thanks! Blaine ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev The canonical way is probably `import platform; platform.python_implementation()` which will return either PyPy or CPython. Alex If you need to support older pythons (which don't have platform.python_implementation), we use import sys is_pypy = '__pypy__' in sys.builtin_module_names or alternatively: try: import __pypy__ except ImportError: __pypy__ = None -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev And don't forget `hasattr(sys, pypy_translation_info)` just for completeness, platform is the cleanest IMO though :) Alex -- I disapprove of what you say, but I will defend to the death your right to say it. -- Evelyn Beatrice Hall (summarizing Voltaire) The people's good is the highest law. -- Cicero ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Re: [pypy-dev] [pypy-commit] pypy ppc-jit-backend: setarrayitem and getarrayitem offsets are immediate values.
Unless I'm missing something offset for get/setarrayitem are not necesarilly intermediate values On Mon, Nov 14, 2011 at 9:27 PM, edelsohn nore...@buildbot.pypy.org wrote: Author: edelsohn Branch: ppc-jit-backend Changeset: r49414:3a6600bf032a Date: 2011-11-14 14:27 -0500 http://bitbucket.org/pypy/pypy/changeset/3a6600bf032a/ Log: setarrayitem and getarrayitem offsets are immediate values. diff --git a/pypy/jit/backend/ppc/ppcgen/opassembler.py b/pypy/jit/backend/ppc/ppcgen/opassembler.py --- a/pypy/jit/backend/ppc/ppcgen/opassembler.py +++ b/pypy/jit/backend/ppc/ppcgen/opassembler.py @@ -367,11 +367,10 @@ value_loc, base_loc, ofs_loc, scale, ofs = arglocs if scale.value 0: scale_loc = r.r0 - self.mc.load_imm(r.r0, scale.value) if IS_PPC_32: - self.mc.slw(r.r0.value, ofs_loc.value, r.r0.value) + self.mc.slwi(r.r0.value, ofs_loc.value, scale.value) else: - self.mc.sld(r.r0.value, ofs_loc.value, r.r0.value) + self.mc.sldi(r.r0.value, ofs_loc.value, scale.value) else: scale_loc = ofs_loc @@ -396,11 +395,10 @@ res, base_loc, ofs_loc, scale, ofs = arglocs if scale.value 0: scale_loc = r.r0 - self.mc.load_imm(r.r0, scale.value) if IS_PPC_32: - self.mc.slw(r.r0.value, ofs_loc.value, scale.value) + self.mc.slwi(r.r0.value, ofs_loc.value, scale.value) else: - self.mc.sld(r.r0.value, ofs_loc.value, scale.value) + self.mc.sldi(r.r0.value, ofs_loc.value, scale.value) else: scale_loc = ofs_loc if ofs.value 0: ___ pypy-commit mailing list pypy-com...@python.org http://mail.python.org/mailman/listinfo/pypy-commit ___ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
