I have some Django benchmarks that I was too lazy to contribute yet. I will do so soon.
2015-02-09 8:41 GMT+02:00 Maciej Fijalkowski <fij...@gmail.com>: > The answer is - yes maybe, but there is no such thing as "django > site". Most of the time the bottlenecks are somewhere else (as you can > see, the admin does not help you) and is often something stupid (e.g. > like here people defining classes at runtime for no good reason) that > has to be looked on a case to case basis. Ideally we would not care > and just optimize everything away anyway, but it's difficult and the > sort of Python that PyPy improves is growing but still not covering > everything we would want. > > Cheers, > fijal > > > On Mon, Feb 9, 2015 at 1:02 AM, Tin Tvrtković <tinches...@gmail.com> > wrote: > > Wow, that's what I call customer support! :) Can confirm PyPy is better > than > > CPython on the admin now, post warmup. > > > > You're right, it doesn't help my site very much, but that's my fault for > > preparing the wrong benchmark fixture. :) > > > > On the other hand, it doesn't really make sense for you PyPy devs to help > > each and every user with their Django site. Would it be worth it for > PyPy to > > add a few benchmarks involving Django (and say, Flask) to the regular > > benchmark suite? I see there's a "django" benchmark but it only involves > the > > templating subsystem. > > > > For example, one benchmark might be serving the admin page; another might > > involve a dummy site with a custom app, 3-4 models from sqlite, and 1-2 > > templates? And the same for Flask with Flask-SQLAlchemy. Another option > > would be using a complex, existing free software Django app (Review > Board, > > Seafile, ...) > > > > This would probably need to be worked out further (i.e. which versions to > > use, when to bump them, etc) but there's probably a large number of users > > out there using Python for exactly these kinds of things. > > > > In any case, thanks for looking into this, it sure does feel nice when > > upstream devs engage with users. :) > > > > > > On 08.02.2015 23:09, Maciej Fijalkowski wrote: > >> > >> It got merged into django, PyPy (2, didn't test the 3) is now faster > >> than cpython on django admin. It likely does not help your cause > >> though so you need to come up with a better benchmark :-) > >> > >> On Sun, Feb 8, 2015 at 8:30 PM, Maciej Fijalkowski <fij...@gmail.com> > >> wrote: > >>> > >>> Hey Tin. > >>> > >>> We are in the process of two (trivial) pull requests to django that > >>> drop the rendering time from 25ms -> 8ms for this case. I'm not a > >>> farseer, however I suspect something like this can be done with your > >>> site too (depending what it really does). > >>> > >>> On Sun, Feb 8, 2015 at 8:23 PM, Tin Tvrtković <tinches...@gmail.com> > >>> wrote: > >>>> > >>>> Hello, > >>>> > >>>> thanks to everyone for their input (especially Maciej). > >>>> > >>>> Since I've ripped out all my code from the test project, it's not a > >>>> Python 3 > >>>> project anymore, so I did try PyPy 2 with it too. It's faster, yes; > the > >>>> last > >>>> test looked something like: > >>>> > >>>> PyPy 2 20.206 [ms] (mean) > >>>> PyPy 3 27.588 [ms] (mean) > >>>> CPython 3.4 10.865 [ms] (mean) > >>>> > >>>> (tried both PyPy 2.5, downloaded directly, and 2.4 from the Ubuntu > PPA - > >>>> about the same.) > >>>> > >>>> I don't know, maybe the admin page isn't a good test candidate since > it > >>>> might do more introspective/reflective/meta stuff. I've tried doing > this > >>>> same test (10k warmup requests, 100 benchmark requests) on the main > page > >>>> of > >>>> the site I'm working on; the results are: > >>>> > >>>> PyPy 3 15.863 [ms] (mean) > >>>> CPython 3.48.668 [ms] (mean) > >>>> > >>>> and I can be certain there's nothing much going on there. Just some > >>>> permission checks, grabbing some stuff from the database, checking > some > >>>> query parameters and a template render. > >>>> > >>>> I suppose if folks have CPU intensive stuff in their view functions > >>>> (nested > >>>> for-loops etc) the optimizations PyPy can do to that far outweigh the > >>>> non-optimized Django parts; my sites generally don't though. > >>>> > >>>> > >>>> On 08.02.2015 18:44, Maciej Fijalkowski wrote: > >>>>> > >>>>> then it's used in the wrong ways in say django admin, look at > >>>>> invocations to lazy there (this is how it showed up in my profile) > >>>>> > >>>>> On Sun, Feb 8, 2015 at 5:08 PM, Alex Gaynor <alex.gay...@gmail.com> > >>>>> wrote: > >>>>>> > >>>>>> FWIW, I've definitely seen and worked on Django sites that got major > >>>>>> improvements out of PyPy -- both the templates and ORM are both sped > >>>>>> up > >>>>>> substantially by it. I think we should try to fix issues as we see > >>>>>> them, > >>>>>> before writing it off. > >>>>>> > >>>>>> FWIW: lazy does not create a new class per call of a function, it's > >>>>>> only > >>>>>> one > >>>>>> class per decorated function. > >>>>>> > >>>>>> Alex > >>>>>> > >>>>>> On Sun, Feb 8, 2015 at 6:59 AM, Maciej Fijalkowski < > fij...@gmail.com> > >>>>>> wrote: > >>>>>>> > >>>>>>> I don't know :-( not sure if fixing those issues one by one is the > >>>>>>> way > >>>>>>> to go, it's not like *this* particular code is a problem, it's the > >>>>>>> culture that breeds those sort of things > >>>>>>> > >>>>>>> On Sun, Feb 8, 2015 at 2:09 PM, Omer Katz <omer.d...@gmail.com> > >>>>>>> wrote: > >>>>>>>> > >>>>>>>> Isn't there anything we can do about it? > >>>>>>>> We can open issues at the Django issue tracker if some code slows > >>>>>>>> down > >>>>>>>> execution in PyPy. > >>>>>>>> > >>>>>>>> 2015-02-08 12:17 GMT+02:00 Maciej Fijalkowski <fij...@gmail.com>: > >>>>>>>>> > >>>>>>>>> Hi Tin > >>>>>>>>> > >>>>>>>>> I have a bit sad news for you, but essentially from what I looked > >>>>>>>>> at > >>>>>>>>> profling, the answer seems to be "don't use django, it has not > been > >>>>>>>>> written with performance in mind". For example here (which > features > >>>>>>>>> in > >>>>>>>>> django admin quite prominently, some stuff calls lazy() at each > >>>>>>>>> call): > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > https://github.com/django/django/blob/master/django/utils/functional.py#L48 > >>>>>>>>> > >>>>>>>>> What does it, per call that pypy does not like: > >>>>>>>>> > >>>>>>>>> * redefines classes > >>>>>>>>> > >>>>>>>>> * walks the class __mro__ and defines tons of new methods > >>>>>>>>> > >>>>>>>>> * proxies everything through a not so efficietn mechanisms > >>>>>>>>> > >>>>>>>>> Those things quite defy the purpose of the JIT - you're making > tons > >>>>>>>>> of > >>>>>>>>> very dynamic things that pypy generally assumes is "static > enough", > >>>>>>>>> if > >>>>>>>>> you wanted to do the same thing in C++, you would need to call > gcc > >>>>>>>>> with each request, that would not be so much fun. > >>>>>>>>> > >>>>>>>>> If you really care about performance, consider swapping your web > >>>>>>>>> framework to something more performance-oriented where JIT > >>>>>>>>> compilation > >>>>>>>>> can help. > >>>>>>>>> > >>>>>>>>> As a side note, we (me and my company, not to be confused with > PyPy > >>>>>>>>> open source project) do offer looking at proprietary code for > >>>>>>>>> benchmarking purposes as a commercial service, look at > >>>>>>>>> baroquesoftware.com > >>>>>>>>> > >>>>>>>>> Cheers, > >>>>>>>>> fijal > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Sat, Feb 7, 2015 at 1:12 AM, Tin Tvrtković > >>>>>>>>> <tinches...@gmail.com> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> Hello, PyPy folks! > >>>>>>>>>> > >>>>>>>>>> While trying to speed up one of my Django sites, I noticed a new > >>>>>>>>>> version > >>>>>>>>>> of > >>>>>>>>>> PyPy > >>>>>>>>>> had just been released. So I grabbed a fresh download of PyPy 3 > >>>>>>>>>> (since > >>>>>>>>>> this > >>>>>>>>>> is > >>>>>>>>>> a Python 3 codebase) and tried taking it out for a spin. > >>>>>>>>>> > >>>>>>>>>> However, as far as I can see, whatever I try PyPy is > consistently > >>>>>>>>>> slower > >>>>>>>>>> than > >>>>>>>>>> CPython for this. > >>>>>>>>>> > >>>>>>>>>> Since this is a proprietary site, I've basically ripped out all > >>>>>>>>>> the > >>>>>>>>>> code > >>>>>>>>>> except > >>>>>>>>>> my settings.py and my requirements; and am benchmarking the > Django > >>>>>>>>>> admin > >>>>>>>>>> index. > >>>>>>>>>> The results are about the same. > >>>>>>>>>> > >>>>>>>>>> I've set up a small repo that can be used to reproduce the > >>>>>>>>>> environment: > >>>>>>>>>> https://github.com/Tinche/PyPy-Django-Playground. There's > >>>>>>>>>> additional > >>>>>>>>>> info in > >>>>>>>>>> the README there. > >>>>>>>>>> > >>>>>>>>>> These tests have been carried out on Ubuntu Trusty, 64-bit. > >>>>>>>>>> CPython 3 > >>>>>>>>>> is > >>>>>>>>>> the > >>>>>>>>>> system Python, 3.4. PyPy has been downloaded from the official > >>>>>>>>>> site > >>>>>>>>>> and > >>>>>>>>>> unzipped. > >>>>>>>>>> > >>>>>>>>>> So what I basically do is set up an admin session, and use the > >>>>>>>>>> Django > >>>>>>>>>> main > >>>>>>>>>> admin > >>>>>>>>>> page. 200 warmup requests, then 100 benchmarked requests, look > at > >>>>>>>>>> the > >>>>>>>>>> mean > >>>>>>>>>> request time. > >>>>>>>>>> > >>>>>>>>>> Some results: > >>>>>>>>>> > >>>>>>>>>> Django's runserver, DEBUG mode: > >>>>>>>>>> > >>>>>>>>>> PyPy3 485.389 [ms] > >>>>>>>>>> CPython 3.4 105.777 [ms] > >>>>>>>>>> > >>>>>>>>>> Django's runserver, no debug: > >>>>>>>>>> > >>>>>>>>>> PyPy3 44.661 [ms] > >>>>>>>>>> CPython 3.4 18.697 [ms] > >>>>>>>>>> > >>>>>>>>>> Gunicorn, 1 worker, no debug: > >>>>>>>>>> > >>>>>>>>>> PyPy3 28.615 [ms] > >>>>>>>>>> CPython 3.4 13.532 [ms] > >>>>>>>>>> > >>>>>>>>>> I don't exactly claim to be an expert on benchmarking, but > >>>>>>>>>> assuming > >>>>>>>>>> my > >>>>>>>>>> site > >>>>>>>>>> is similar to the Django admin, CPython's gonna be giving me > >>>>>>>>>> better > >>>>>>>>>> performance. > >>>>>>>>>> Also the debug runserver performance is kinda worrying. Nobody's > >>>>>>>>>> going > >>>>>>>>>> to be > >>>>>>>>>> running this in production, but it makes development a little > >>>>>>>>>> slower > >>>>>>>>>> and > >>>>>>>>>> more > >>>>>>>>>> annoying than it should be. > >>>>>>>>>> > >>>>>>>>>> Is there anything to make PyPy more competitive in these kinds > of > >>>>>>>>>> scenarios? > >>>>>>>>>> > >>>>>>>>>> Kind regards, > >>>>>>>>>> Tin > >>>>>>>>>> _______________________________________________ > >>>>>>>>>> pypy-dev mailing list > >>>>>>>>>> pypy-dev@python.org > >>>>>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev > >>>>>>>>> > >>>>>>>>> _______________________________________________ > >>>>>>>>> pypy-dev mailing list > >>>>>>>>> pypy-dev@python.org > >>>>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev > >>>>>>>> > >>>>>>>> > >>>>>>> _______________________________________________ > >>>>>>> pypy-dev mailing list > >>>>>>> pypy-dev@python.org > >>>>>>> https://mail.python.org/mailman/listinfo/pypy-dev > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> "I disapprove of what you say, but I will defend to the death your > >>>>>> right > >>>>>> to > >>>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > >>>>>> "The people's good is the highest law." -- Cicero > >>>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084 > >>>> > >>>> > > >
_______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
