Yes This really deserve a blog post . I haven't finished writing my own blog yet tho :D.
On Fri, Sep 5, 2014 at 11:31 PM, Maciej Fijalkowski <fij...@gmail.com> wrote: > Hi > > Can you please put it all in a blog post (ideally with examples), it's > a lot of useful info. > > I'm willing to help > > On Fri, Sep 5, 2014 at 10:47 AM, Jolitz Ben - bjolit > <ben.jol...@acxiom.com> wrote: >> I don’t have specific suggestions on Mongo, but I can share what I’ve >> learned in a few months of using PyPy and Tornado. >> >> You want to make use of CFFI in PyPy to accelerate operations that would >> usually be slow in Python, namely encryption and database drivers. But >> always test first to see if you really need to go to C. >> >> /Any/ CPython C-Extensions will torpedo performance. Anything blocking (like >> the majority of DB drivers) will similarly destroy Tornado performance. >> >> Code that is overly dynamic also does lousy on PyPy. If your driver has a >> ton of paths or makes idiot use of threading.Lock, expect to have an uphill >> struggle in optimization. >> >> When in doubt, ask yourself if the algorithm is appropriate. >> >> If you can’t make the Python driver performant and there exists a C-API for >> it, then it is trivial to wrap it with CFFI. >> >> If it doesn’t support nonblocking operations, you can find alternatives. For >> example in MySQLdb, others have found you can add the Connection._fd to the >> IOLoop and use it to do a send_query, read_query. If you still can’t find an >> alternative to a blocking call, you can still mimic nonblocking IO by using >> pthreads, a work queue and a callback pthread. It’s not perfect and theres a >> lot you can optimize, but it can easily allow you to delegate long running >> operations to C. >> >> Another thing I learned was to avoid generating Python-side c callback >> pointers frequently. If C is going to callback into Python with your result >> to a unique Request, you’re going to need to tag it appropriately. I prefer >> to use dictionaries and the callback attribute handed to me by gen.Task and >> pass a unique-enough key to C to callback in with. >> >> As with everything in optimization, profile your code first. If you’re >> losing speed heavily somewhere else, then the above will only serve to >> distract you. >> >> Cheers, >> >> Ben >> >> >> From: Phyo Arkar <phyo.arkarl...@gmail.com> >> Date: Thursday, September 4, 2014 at 1:55 PM >> To: Ben Jolitz <ben.jol...@acxiom.com> >> Cc: pypy-dev <pypy-dev@python.org> >> >> Subject: Re: [pypy-dev] Pypy Benchmark of Tornado. >> >> Thanks alot Ben, >> Ok , as PyPy is choice of quora and they also use tornado , i might keep >> testing on larget projects. >> How about mongodb performance on PyPy , i heard its slower due to no C >> Extension (no CFFI) for pypy. >> Your suggestion will be very appreciated. >> >> >> On Thu, Sep 4, 2014 at 4:56 AM, Jolitz Ben - bjolit <ben.jol...@acxiom.com> >> wrote: >>> >>> I use Tornado and have found PyPy can yield a 30-50% performance increase >>> for a moderately complex project. >>> >>> Ben >>> >>> From: Phyo Arkar <phyo.arkarl...@gmail.com> >>> Date: Wednesday, September 3, 2014 at 8:21 AM >>> To: pypy-dev <pypy-dev@python.org> >>> Subject: Re: [pypy-dev] Pypy Benchmark of Tornado. >>> >>> I expect pypy to be faster in those cases but select io is not cpu >>> intensive thing to do so no real benefit using pypy here i guess. >>> On Wed, Sep 3, 2014 at 4:43 PM, Phyo Arkar <phyo.arkarl...@gmail.com> >>> wrote: >>> > >>> > It just return json document with a few thousand characters (1053 bytes) >>> > $siege -c 400 -t 20s -r 2000 http://localhost:9999/js >>> > >>> > Python 2.7.7: >>> > >>> > Lifting the server siege... done. >>> > >>> > Transactions: 14478 hits >>> > Availability: 100.00 % >>> > Elapsed time: 19.10 secs >>> > Data transferred: 14.54 MB >>> > Response time: 0.01 secs >>> > Transaction rate: 758.01 trans/sec >>> > Throughput: 0.76 MB/sec >>> > Concurrency: 8.91 >>> > Successful transactions: 14478 >>> > Failed transactions: 0 >>> > Longest transaction: 1.08 seconds >>> > Shortest transaction: 0.00 >>> > >>> > pypy-2.3.1 stable: >>> > >>> > Transactions: 15149 hits >>> > Availability: 100.00 % >>> > Elapsed time: 19.63 secs >>> > Data transferred: 15.21 MB >>> > Response time: 0.02 secs >>> > Transaction rate: 771.73 trans/sec >>> > Throughput: 0.77 MB/sec >>> > Concurrency: 11.92 >>> > Successful transactions: 15149 >>> > Failed transactions: 0 >>> > Longest transaction: 1.09 seconds >>> > Shortest transaction: 0.00 >>> > >>> > >>> > >>> > pypy--c-jit-73283-912dd9df99a8-linux64 (latest nightly build) >>> > >>> > Lifting the server siege... done. >>> > >>> > Transactions: 14361 hits >>> > Availability: 100.00 % >>> > Elapsed time: 19.13 secs >>> > Data transferred: 14.42 MB >>> > Response time: 0.03 secs >>> > Transaction rate: 750.71 trans/sec >>> > Throughput: 0.75 MB/sec >>> > Concurrency: 21.53 >>> > Successful transactions: 14361 >>> > Failed transactions: 0 >>> > Longest transaction: 3.03 seconds >>> > Shortest transaction: 0.00 >>> > >>> > >>> > >>> > >>> > Pypy Nightly have some request Randomly get to 3.0 Seconds , normally >>> > those requests (in Cpython) are only ~0.001 to 0.002 sec. >>> > >>> >>> It just return json document with a few thousand characters (1053 bytes) >>> $siege -c 400 -t 20s -r 2000 http://localhost:9999/js >>> >>> Python 2.7.7: >>> >>> Lifting the server siege... done. >>> >>> Transactions: 14478 hits >>> Availability: 100.00 % >>> Elapsed time: 19.10 secs >>> Data transferred: 14.54 MB >>> Response time: 0.01 secs >>> Transaction rate: 758.01 trans/sec >>> Throughput: 0.76 MB/sec >>> Concurrency: 8.91 >>> Successful transactions: 14478 >>> Failed transactions: 0 >>> Longest transaction: 1.08 seconds >>> Shortest transaction: 0.00 >>> >>> pypy-2.3.1 stable: >>> >>> Transactions: 15149 hits >>> Availability: 100.00 % >>> Elapsed time: 19.63 secs >>> Data transferred: 15.21 MB >>> Response time: 0.02 secs >>> Transaction rate: 771.73 trans/sec >>> Throughput: 0.77 MB/sec >>> Concurrency: 11.92 >>> Successful transactions: 15149 >>> Failed transactions: 0 >>> Longest transaction: 1.09 seconds >>> Shortest transaction: 0.00 >>> >>> >>> >>> pypy--c-jit-73283-912dd9df99a8-linux64 (latest nightly build) >>> >>> Lifting the server siege... done. >>> >>> Transactions: 14361 hits >>> Availability: 100.00 % >>> Elapsed time: 19.13 secs >>> Data transferred: 14.42 MB >>> Response time: 0.03 secs >>> Transaction rate: 750.71 trans/sec >>> Throughput: 0.75 MB/sec >>> Concurrency: 21.53 >>> Successful transactions: 14361 >>> Failed transactions: 0 >>> Longest transaction: 3.03 seconds >>> Shortest transaction: 0.00 >>> >>> >>> >>> >>> Pypy Nightly have some request Randomly get to 3.0 Seconds , normally >>> those requests (in Cpython) are only ~0.001 to 0.002 sec. >>> >>> >>> *************************************************************************** >>> The information contained in this communication is confidential, is >>> intended only for the use of the recipient named above, and may be legally >>> privileged. >>> >>> If the reader of this message is not the intended recipient, you are >>> hereby notified that any dissemination, distribution or copying of this >>> communication is strictly prohibited. >>> >>> If you have received this communication in error, please resend this >>> communication to the sender and delete the original message or any copy >>> of it from your computer system. >>> >>> Thank You. >>> >>> **************************************************************************** >>> >>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev@python.org >>> https://mail.python.org/mailman/listinfo/pypy-dev >>> >> >> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev@python.org >> https://mail.python.org/mailman/listinfo/pypy-dev >> _______________________________________________ pypy-dev mailing list pypy-dev@python.org https://mail.python.org/mailman/listinfo/pypy-dev
