Hi Brett, Thank you for your post. I'm strongly agree with you, it's my feeling.
FYI, I've remade my HTTP application benchmarks with the changes suggested by Mike for the sync part with Flask+threads: Yes, the performances are better, but it's still slower than AsyncIO version. I'll publish this week all data+a setup to reproduce that by everybody. Be very prudent with benchmarks, especially with micro-benchmarks to conclude on architecture for your application. To illustrate that, I give you a concrete example: I like a lot PyPy, everybody knows this graph: http://speed.pypy.org/ Two years ago, when we started to have performances issues, we migrated a Flask project to PyPy. After that, I've benchmarked the application. It was slower than CPython version. Finally, we migrated to AsyncIO. Maybe that now it shouldn't be true anymore because PyPy seems to have better I/O performances. Certainly if I worked for a company with a lot of CPU-bound problem, as a scientific company, my speech will be the contrary. But, my day-to-day problem is more I/O than CPU, certainly as most Webapp developers. At least for me, Mike's post it's a good impulse to dig more about theses problematics. To reminder, I don't want to start a war, only retrieve good ideas from everybody to have the fastest Python architecture as possible for production applications. Another pattern it could help to improve AsyncIO performances: have multiplexing support from frontend to backend protocols. It only means that when you send a request to the server, you give an id to identify response. With that, you can stack a lot of requests. I already use that a lot with Asterisk. When I'll the time, I'll publish a benchmark on that. Apparently, HTTP/2 has also this feature. If somebody is working on HTTP/2 support aiohttp, I'm really interested in to test. For databases, apprently somebody written that for MySQL: https://github.com/tailhook/zorro/blob/master/zorro/mysql.py If somebody already dig in this direction, I'm also interested in. -- Ludovic Gasc On Tue, Feb 17, 2015 at 8:53 AM, Brett Slatkin <[email protected]> wrote: > Mike's post is good and I enjoyed it. The benchmarks are interesting. I > started writing a reply to this email thread, but it kept getting bigger > and bigger! I decided to post my reply here instead: > > http://www.onebigfluke.com/2015/02/asyncio-is-for-composition.html > > I'd love to hear if people on this list agree or disagree with my > reasoning. Perhaps the real problem here is the goals of asyncio aren't > understood well enough by most Python programmers out there? Or maybe I'm > wrong. :) > > Otherwise, I'm curious what low hanging fruit in the library could be > identified with some more profiling (and what fruit has already been > plucked?). > > -Brett > > > > On Mon, Feb 16, 2015 at 3:05 PM, Victor Stinner <[email protected]> > wrote: > >> Hi, >> >> Mike Bayer, the author of SQLAchemy, wrote a long article about >> asyncio and databases: >> http://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ >> >> IMO the most interesting part if the benchmark written for this article: >> https://bitbucket.org/zzzeek/bigdata/ >> >> The benchmark inserts a lot of rows (more than 9 millions) in a >> PosgreSQL server using psycopg2 and aiopg. It compares performances of >> threads, gevent and asyncio. Bad news: asyncio is much slower on this >> benchmark (between 1.3x and 3.3x slower). >> >> It's not easy to create the setup to run the benchmark (ex: you have >> to install a PostgreSQL server and configure it to run the benchmark), >> you have to find the best pool size for your setup and then you have >> to analyze bencmark results (there is no unique number at the end, >> just a long list of numbers). On my first setup (desktop: benchmark, >> laptop: server, slow LAN), I had to stop the benchmark after 2 hours. >> >> Mike see between 6,000 and 26,000 SQL INSERT queries per second >> depending on his setup and on the benchmark parameter. Ah yes, there >> are also options to tune the benchmark, but I don't think that you are >> supposed to use them. >> >> I'm trying to reproduce the benchmark to check if I get similar >> results and then to try to run asyncio in a profiler. I never used >> aiopg, nor psycopg2, and I don't remember when I installed a >> PostgreSQL server for the last time :-) >> >> Victor >> > >
