I believe those results are due to the current limitation of 10 concurrent requests per instance.
There is a feature request to make that configurable: http://code.google.com/p/googleappengine/issues/detail?id=7927 I made a quick benchmark with urlfetch to a go backend that store k,v in memory. While I get managed to get 350+qps out of it, I don't get a performance boost by using a B8 instead of B1, see below: $ ab -c 300 -n 3000 -r http://proppy-loadtest-backend.appspot.com/?backend=gob8 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking proppy-loadtest-backend.appspot.com (be patient) Completed 300 requests Completed 600 requests Completed 900 requests Completed 1200 requests Completed 1500 requests Completed 1800 requests Completed 2100 requests Completed 2400 requests Completed 2700 requests Completed 3000 requests Finished 3000 requests Server Software: Google Server Hostname: proppy-loadtest-backend.appspot.com Server Port: 80 Document Path: /?backend=gob8 Document Length: 31 bytes Concurrency Level: 300 Time taken for tests: 8.480 seconds Complete requests: 3000 Failed requests: 0 Write errors: 0 Total transferred: 2777169 bytes HTML transferred: 93000 bytes Requests per second: 353.77 [#/sec] (mean) Time per request: 848.006 [ms] (mean) Time per request: 2.827 [ms] (mean, across all concurrent requests) Transfer rate: 319.82 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 17 18 0.7 18 31 Processing: 133 769 293.1 763 1904 Waiting: 133 769 293.1 763 1904 Total: 151 787 293.1 780 1921 Percentage of the requests served within a certain time (ms) 50% 780 66% 907 75% 996 80% 1040 90% 1162 95% 1272 98% 1394 99% 1475 100% 1921 (longest request) $ ab -c 300 -n 3000 -r http://proppy-loadtest-backend.appspot.com/?backend=gob1 This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking proppy-loadtest-backend.appspot.com (be patient) Completed 300 requests Completed 600 requests Completed 900 requests Completed 1200 requests Completed 1500 requests Completed 1800 requests Completed 2100 requests Completed 2400 requests Completed 2700 requests Completed 3000 requests Finished 3000 requests Server Software: Google Server Hostname: proppy-loadtest-backend.appspot.com Server Port: 80 Document Path: /?backend=gob1 Document Length: 31 bytes Concurrency Level: 300 Time taken for tests: 8.262 seconds Complete requests: 3000 Failed requests: 0 Write errors: 0 Total transferred: 2773964 bytes HTML transferred: 93000 bytes Requests per second: 363.11 [#/sec] (mean) Time per request: 826.197 [ms] (mean) Time per request: 2.754 [ms] (mean, across all concurrent requests) Transfer rate: 327.88 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 17 18 0.6 18 28 Processing: 136 759 331.8 722 2176 Waiting: 136 758 331.8 722 2176 Total: 154 776 331.8 740 2193 Percentage of the requests served within a certain time (ms) 50% 740 66% 881 75% 978 80% 1045 90% 1219 95% 1373 98% 1520 99% 1659 100% 2193 (longest request) On Wed, Aug 8, 2012 at 7:30 AM, [email protected] <[email protected]>wrote: > Very interesting result. > > Sent from my HTC > > > ----- Reply message ----- > From: "Jeff Schnitzer" <[email protected]> > To: "Google App Engine" <[email protected]> > Subject: [google-appengine] Backend performance, compared > Date: Wed, Aug 8, 2012 09:10 > > > If you've been reading various threads on this list you know that > Richard has been having trouble getting his mobile game to run > smoothly on GAE. It's a little unusual because timing is coordinated > precisely: > > * At T+0, all clients submit scores > * At T+5s, a reaper process aggregates the scores and builds a result set > * At T+10s, all clients fetch scores > > The question is: Where to submit the score data so that the reaper > can fetch and aggregate it? > > Here's some answers that didn't work: > > * The datastore. Eventual consistency is too eventual to query for > all the scores and get them. > * Pull queues. There's too much of a delay between task insertion > and when it appears for leasing. > * A single backend. One backend cannot handle more than ~80qps. > > He eventually got a system working reliably, sharded across ten B1 > instances, at a cost (beyond other charges) of ~$600/mo. It can > collect a couple thousand scores within the 5s deadline (barely). > > I thought this was insane, so I built a few experiments to see what > other technologies can do, using the exact program logic of Richard's > collector. Here are the results: > > The environment: 256MB Rackspacecloud VPS running Ubuntu 10.04.4 LTS > The cost: $11/mo > The command: ab -c 10000 -n 10000 -r http://theurl (that's 10k > requests, all concurrent). > > Node.js: ~2500 qps. Rock solid through multiple test runs, all > complete before the deadline. > Java SimpleHTTP: ~2100 qps. Had to bump heap up to 128MB. > Python Twisted: ~1600 qps. Failed a lot of requests on most test runs. > Python Tornado: ~1500 qps, but rock solid through multiple test runs. > > So basically, an $11/mo VPS server running Javascript vastly exceeds > the capabilities of 10 backends at $60/mo each. > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
