Thanks a lot for the benchmarks. So we still have to wait , i guess. Would be nice a blogpost with PyPy 2 vs PyPy3 vs CPython 2 and 3.
On Wed, Oct 29, 2014 at 12:53 AM, Philip Jenvey <pjen...@underboss.org> wrote: > > On Oct 24, 2014, at 3:02 PM, Phyo Arkar <phyo.arkarl...@gmail.com> wrote: > >> Congrats! >> I am using Pypy 2.4 for tornado based web development , performance >> improvement is the real deal there. >> >> i am waiting to switch to python3 and its becoming near! Its when >> pypy3 3.4 is ready (ie , Python3 is only starting to interesting after >> 3.4.x due to asyncio module) . >> >> I have a question: >> In CPython , python 2 vs python 3 is around 10-30% performance >> difference. In PyPy world is there such difference? Same performance >> between pypy2 and pypy3 ? > > Hi there, > > PyPy3 performance isn’t quite as good as PyPy2’s yet. Our main focus has been > on compatibility, as you said yourself, you probably won’t switch until a > later version of Python 3’s supported. > > The benchmark infrastructure for PyPy3 is in need of help. Pypy's own > benchmark suite used by http://speed.pypy.org isn't Python 3 compatible yet, > nor is the site really setup to run PyPy3 benchmark runs. > > A few years ago there were plans for the creation of a shared between VMs/the > PSF version of speed.pypy.org (to be http://speed.python.org), unfortunately > these efforts seemed to have failed or at the very least stalled indefinitely. > > However the benchmark suite intended for this new site supports Python 3. > It’s similar to pypy’s own suite, albeit with a few different benchmarks and > unfortunately separately maintained. Here’s output from a run of it with > CPython 3.2.5 vs PyPy3 2.4.0: though this was ran on a laptop which isn’t > going to produce the most stable numbers but it'll give you some idea: > > > ### 2to3 ### > 8.652382 -> 9.627286: 1.11x slower > > ### call_method ### > Min: 0.385220 -> 0.011149: 34.55x faster > Avg: 0.412039 -> 0.012474: 33.03x faster > Significant (t=293.52) > Stddev: 0.01644 -> 0.00279: 5.8934x smaller > > ### call_method_slots ### > Min: 0.394147 -> 0.011669: 33.78x faster > Avg: 0.419456 -> 0.013503: 31.06x faster > Significant (t=185.89) > Stddev: 0.02655 -> 0.00326: 8.1550x smaller > > ### call_method_unknown ### > Min: 0.416863 -> 0.016139: 25.83x faster > Avg: 0.444017 -> 0.019560: 22.70x faster > Significant (t=187.44) > Stddev: 0.02645 -> 0.00835: 3.1669x smaller > > ### call_simple ### > Min: 0.324141 -> 0.011182: 28.99x faster > Avg: 0.343808 -> 0.013248: 25.95x faster > Significant (t=235.96) > Stddev: 0.01667 -> 0.00406: 4.1072x smaller > > ### chameleon ### > Min: 0.061195 -> 0.010301: 5.94x faster > Avg: 0.066450 -> 0.015595: 4.26x faster > Significant (t=52.46) > Stddev: 0.00668 -> 0.00982: 1.4710x larger > > ### chaos ### > Min: 0.340240 -> 0.011240: 30.27x faster > Avg: 0.366752 -> 0.018671: 19.64x faster > Significant (t=60.31) > Stddev: 0.01713 -> 0.03704: 2.1620x larger > > ### django_v2 ### > Min: 0.692480 -> 0.029994: 23.09x faster > Avg: 0.731315 -> 0.037337: 19.59x faster > Significant (t=173.98) > Stddev: 0.02693 -> 0.00837: 3.2160x smaller > > ### etree_generate ### > Min: 0.599923 -> 1.087517: 1.81x slower > Avg: 0.636505 -> 1.149231: 1.81x slower > Significant (t=-77.16) > Stddev: 0.02478 -> 0.03993: 1.6114x larger > > ### etree_parse ### > Min: 0.296519 -> 0.931630: 3.14x slower > Avg: 0.331876 -> 1.010314: 3.04x slower > Significant (t=-78.30) > Stddev: 0.03799 -> 0.04807: 1.2652x larger > > ### etree_process ### > Min: 0.421074 -> 0.739647: 1.76x slower > Avg: 0.454918 -> 0.798387: 1.76x slower > Significant (t=-55.61) > Stddev: 0.03127 -> 0.03048: 1.0260x smaller > > ### fannkuch ### > Min: 1.744042 -> 0.179821: 9.70x faster > Avg: 1.815167 -> 0.193547: 9.38x faster > Significant (t=293.69) > Stddev: 0.03619 -> 0.01465: 2.4707x smaller > > ### fastpickle ### > Min: 0.671650 -> 109.014578: 162.31x slower > Avg: 0.708033 -> 115.379135: 162.96x slower > Significant (t=-299.42) > Stddev: 0.02026 -> 2.70799: 133.6909x larger > > ### fastunpickle ### > Min: 0.586665 -> 55.736337: 95.01x slower > Avg: 0.625024 -> 56.833369: 90.93x slower > Significant (t=-357.75) > Stddev: 0.02488 -> 1.11069: 44.6438x larger > > ### float ### > Min: 0.358756 -> 0.026437: 13.57x faster > Avg: 0.387649 -> 0.041561: 9.33x faster > Significant (t=106.89) > Stddev: 0.02159 -> 0.00762: 2.8351x smaller > > ### formatted_logging ### > Min: 0.306111 -> 0.086534: 3.54x faster > Avg: 0.331045 -> 0.116844: 2.83x faster > Significant (t=32.06) > Stddev: 0.02016 -> 0.04273: 2.1198x larger > > ### go ### > Min: 0.640692 -> 0.110813: 5.78x faster > Avg: 0.686035 -> 0.153473: 4.47x faster > Significant (t=81.23) > Stddev: 0.02366 -> 0.03987: 1.6853x larger > > ### hexiom2 ### > Min: 168.499830 -> 13.845624: 12.17x faster > Avg: 170.731699 -> 14.320609: 11.92x faster > Significant (t=68.55) > Stddev: 3.15634 -> 0.67173: 4.6988x smaller > > ### iterative_count ### > Min: 0.155776 -> 0.001828: 85.22x faster > Avg: 0.169719 -> 0.001959: 86.63x faster > Significant (t=66.83) > Stddev: 0.01775 -> 0.00017: 104.6217x smaller > > ### json_dump ### > Min: 0.563337 -> 2.088108: 3.71x slower > Avg: 0.600209 -> 2.212642: 3.69x slower > Significant (t=-92.38) > Stddev: 0.01963 -> 0.12185: 6.2066x larger > > ### json_dump_v2 ### > Min: 3.703154 -> 12.685391: 3.43x slower > Avg: 3.906239 -> 13.462007: 3.45x slower > Significant (t=-73.61) > Stddev: 0.16381 -> 0.90320: 5.5139x larger > > ### json_load ### > Min: 0.637112 -> 0.284530: 2.24x faster > Avg: 0.691114 -> 0.340403: 2.03x faster > Significant (t=48.13) > Stddev: 0.03443 -> 0.03834: 1.1134x larger > > ### mako ### > Min: 0.068892 -> 0.023433: 2.94x faster > Avg: 0.073817 -> 0.034535: 2.14x faster > Significant (t=45.74) > Stddev: 0.01044 -> 0.00868: 1.2032x smaller > > ### mako_v2 ### > Min: 0.055735 -> 0.023094: 2.41x faster > Avg: 0.060464 -> 0.034531: 1.75x faster > Significant (t=39.36) > Stddev: 0.01003 -> 0.01079: 1.0764x larger > > ### meteor_contest ### > Min: 0.234554 -> 0.101157: 2.32x faster > Avg: 0.253522 -> 0.112062: 2.26x faster > Significant (t=45.33) > Stddev: 0.01735 -> 0.01364: 1.2717x smaller > > ### nbody ### > Min: 0.332543 -> 0.024746: 13.44x faster > Avg: 0.352898 -> 0.026261: 13.44x faster > Significant (t=111.01) > Stddev: 0.02059 -> 0.00298: 6.9071x smaller > > ### normal_startup ### > Min: 0.974583 -> 1.578062: 1.62x slower > Avg: 1.011902 -> 1.640190: 1.62x slower > Significant (t=-144.15) > Stddev: 0.01753 -> 0.02535: 1.4458x larger > > ### nqueens ### > Min: 0.357568 -> 0.052418: 6.82x faster > Avg: 0.386644 -> 0.055879: 6.92x faster > Significant (t=98.78) > Stddev: 0.02223 -> 0.00816: 2.7244x smaller > > ### pathlib ### > Min: 0.119940 -> 0.384461: 3.21x slower > Avg: 0.130969 -> 0.504688: 3.85x slower > Significant (t=-142.33) > Stddev: 0.01389 -> 0.03912: 2.8161x larger > > ### pickle_dict ### > Min: 0.496007 -> 8.778371: 17.70x slower > Avg: 0.522927 -> 9.051905: 17.31x slower > Significant (t=-306.08) > Stddev: 0.01866 -> 0.19615: 10.5126x larger > > ### pickle_list ### > Min: 0.295601 -> 6.132281: 20.75x slower > Avg: 0.315390 -> 6.327789: 20.06x slower > Significant (t=-215.11) > Stddev: 0.01909 -> 0.19672: 10.3048x larger > > ### pidigits ### > Min: 0.345745 -> 0.297060: 1.16x faster > Avg: 0.362924 -> 0.316708: 1.15x faster > Significant (t=13.14) > Stddev: 0.01719 -> 0.01798: 1.0456x larger > > ### raytrace ### > Min: 1.527975 -> 0.046997: 32.51x faster > Avg: 1.575889 -> 0.052239: 30.17x faster > Significant (t=344.42) > Stddev: 0.02992 -> 0.00911: 3.2830x smaller > > ### regex_compile ### > Min: 0.462712 -> 0.076845: 6.02x faster > Avg: 0.489184 -> 0.159849: 3.06x faster > Significant (t=11.67) > Stddev: 0.01954 -> 0.19853: 10.1609x larger > > ### regex_effbot ### > Min: 0.061423 -> 0.034389: 1.79x faster > Avg: 0.065657 -> 0.041867: 1.57x faster > Significant (t=16.14) > Stddev: 0.00658 -> 0.00808: 1.2285x larger > > ### richards ### > Min: 0.201109 -> 0.004319: 46.56x faster > Avg: 0.215034 -> 0.004968: 43.28x faster > Significant (t=91.42) > Stddev: 0.01619 -> 0.00140: 11.5878x smaller > > ### silent_logging ### > Min: 0.080098 -> 0.007613: 10.52x faster > Avg: 0.085028 -> 0.009930: 8.56x faster > Significant (t=43.10) > Stddev: 0.00980 -> 0.00747: 1.3113x smaller > > ### simple_logging ### > Min: 0.296057 -> 0.081953: 3.61x faster > Avg: 0.315012 -> 0.110826: 2.84x faster > Significant (t=33.22) > Stddev: 0.01866 -> 0.03925: 2.1031x larger > > ### spectral_norm ### > Min: 0.365987 -> 0.016084: 22.75x faster > Avg: 0.393062 -> 0.018651: 21.07x faster > Significant (t=124.58) > Stddev: 0.02014 -> 0.00679: 2.9647x smaller > > ### startup_nosite ### > Min: 0.485740 -> 0.613260: 1.26x slower > Avg: 0.518384 -> 0.654354: 1.26x slower > Significant (t=-49.49) > Stddev: 0.01842 -> 0.02038: 1.1066x larger > > ### telco ### > Min: 0.773569 -> 0.043440: 17.81x faster > Avg: 0.800002 -> 0.049813: 16.06x faster > Significant (t=276.68) > Stddev: 0.01300 -> 0.01409: 1.0844x larger > > ### threaded_count ### > Min: 0.167072 -> 0.005613: 29.76x faster > Avg: 0.193020 -> 0.005944: 32.47x faster > Significant (t=88.19) > Stddev: 0.01499 -> 0.00036: 41.4676x smaller > > ### tornado_http ### > Min: 0.307076 -> 0.390125: 1.27x slower > Avg: 0.378676 -> 0.459946: 1.21x slower > Significant (t=-8.21) > Stddev: 0.04109 -> 0.09005: 2.1916x larger > > ### unpack_sequence ### > Min: 0.000052 -> 0.000057: 1.10x slower > Avg: 0.000077 -> 0.000404: 5.21x slower > Significant (t=-16.30) > Stddev: 0.00011 -> 0.00447: 41.4670x larger > > ### unpickle_list ### > Min: 0.455635 -> 0.963103: 2.11x slower > Avg: 0.514364 -> 1.017570: 1.98x slower > Significant (t=-79.98) > Stddev: 0.02640 -> 0.03581: 1.3564x larger > > > So PyPy3’s generally faster, except on startup and tornado_http, where it’s > only slightly slower. > > Then it’s quite a bit slower on “mini interpreters” such as > etree/json/Pickle. These modules also happen to be pure/mostly pure python > versions on PyPy vs C versions used by CPython. Pickle looks particularly > bad, I have a guess that it’s due to removal of an optimization in marshal > that happened during py3k compatibility work, see: > > https://bitbucket.org/pypy/pypy/src/a245775c1e3687662879fe57964a5ece689cd781/pypy/module/marshal/interp_marshal.py?at=py3k#cl-13 > > and it’s still slower than PyPy2 on many things. > > Anyway, to reiterate we’ve pretty busy with compatibility work. We could use > some help with benchmarking infrastructure =] > > -- > Philip Jenvey > > _______________________________________________ > pypy-dev mailing list > pypy-dev@python.org > https://mail.python.org/mailman/listinfo/pypy-dev _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
