Hi David Beazley noticed that PyPy's GIL isn't very good compared to CPython's :
https://twitter.com/#!/dabeaz/status/118889721358327808 https://twitter.com/#!/dabeaz/status/118888789136523264 https://twitter.com/#!/dabeaz/status/118864260175634433 IMO it's the same issue Cheers Romain On Thu, Sep 29, 2011 at 07:17:45PM +0200, Laura Creighton wrote: > > He says, currently, PyPy's threading does not scale properly. More below. > Maybe we > want to use his benchmark? > > Laura > > > ------- Forwarded Message > > Return-Path: rus...@russel.org.uk > Delivery-Date: Thu Sep 29 13:53:50 2011 > Subject: PyPy and multiprocessing > From: Russel Winder <rus...@russel.org.uk> > To: Laura Creighton <l...@openend.se> > > - --=-I2STZOatYEgK/vXAGHYd > Content-Type: text/plain; charset="UTF-8" > Content-Transfer-Encoding: quoted-printable > > Laura, > > I have a collection of various versions (using various features of > various languages) of the embarrassingly parallel problem of calculating > Pi using quadrature. It is a micro-benchmark and so suffers from all > the issues they suffer from (especially on the JVM). The code is a > Bazaar branch http://www.russel.org.uk/Bazaar/Pi_Quadrature. > > I am writing as there appears to be an interesting feature using PyPy > and the microprocessing package in pool mode. > > This is a twin-Xeon machine so has 8 cores =E2=80=94 a 32 thread run should= > only > go as fast as an 8 thread run. Scaling should be linear in the number > of cores. > > Using CPython 2.7, I get: > > |> python2.7 pi_python2_multiprocessing_pool.py > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 3.5378549099 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 1 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 1.97133994102 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 2 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 0.515691041946 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 8 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 0.521239995956 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 32 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > Using PyPy 1.6 I get: > > |> pypy pi_python2_multiprocessing_pool.py > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 0.249331951141 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 1 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 0.104065895081 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 2 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 0.0764398574829 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 8 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > =3D=3D=3D=3D Python Multiprocessing Pool pi =3D 3.14159265359 > =3D=3D=3D=3D Python Multiprocessing Pool iteration count =3D 10000000 > =3D=3D=3D=3D Python Multiprocessing Pool elapse =3D 0.124751091003 > =3D=3D=3D=3D Python Multiprocessing Pool process count =3D 32 > =3D=3D=3D=3D Python Multiprocessing Pool processor count =3D 8 > > There is no statistical significance to these one off numbers but I am > fairly confident that there are no large variations should a proper > collection of data be taken. > > The point here is that whereas CPython shows the expected scaling, PyPy > does not give the expected scaling for larger nubmers of cores. Indeed > having more threads than cores is detrimental to PyPy but not to > CPython. > > Hopefully we will soon be seeing PyPy be Python 3.2 compliant! > > - --=20 > Russel. > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= > =3D=3D > Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.n= > et > 41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@russel.org.uk > London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder > > - --=-I2STZOatYEgK/vXAGHYd > Content-Type: application/pgp-signature; name="signature.asc" > Content-Description: This is a digitally signed message part > Content-Transfer-Encoding: 7bit > > - -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iEYEABECAAYFAk6EWpIACgkQr2EGkixYSbrjHQCeJCcoOamtzwY3rPSFofuXACYK > 7RgAnjuWAuGHI0mSUzQ/BhS/clZbZaUy > =kVgo > - -----END PGP SIGNATURE----- > > - --=-I2STZOatYEgK/vXAGHYd-- > > ------- End of Forwarded Message > > _______________________________________________ > pypy-dev mailing list > pypy-dev@python.org > http://mail.python.org/mailman/listinfo/pypy-dev _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
