Terrifying. So, I assume Richard's medium-term plan is to set up an "outboard" collector endpoint, and have a single GAE backend reap from that?
- Kris On Tuesday, August 7, 2012 7:10:43 PM UTC-7, Jeff Schnitzer wrote: > > If you've been reading various threads on this list you know that > Richard has been having trouble getting his mobile game to run > smoothly on GAE. It's a little unusual because timing is coordinated > precisely: > > * At T+0, all clients submit scores > * At T+5s, a reaper process aggregates the scores and builds a result set > * At T+10s, all clients fetch scores > > The question is: Where to submit the score data so that the reaper > can fetch and aggregate it? > > Here's some answers that didn't work: > > * The datastore. Eventual consistency is too eventual to query for > all the scores and get them. > * Pull queues. There's too much of a delay between task insertion > and when it appears for leasing. > * A single backend. One backend cannot handle more than ~80qps. > > He eventually got a system working reliably, sharded across ten B1 > instances, at a cost (beyond other charges) of ~$600/mo. It can > collect a couple thousand scores within the 5s deadline (barely). > > I thought this was insane, so I built a few experiments to see what > other technologies can do, using the exact program logic of Richard's > collector. Here are the results: > > The environment: 256MB Rackspacecloud VPS running Ubuntu 10.04.4 LTS > The cost: $11/mo > The command: ab -c 10000 -n 10000 -r http://theurl (that's 10k > requests, all concurrent). > > Node.js: ~2500 qps. Rock solid through multiple test runs, all > complete before the deadline. > Java SimpleHTTP: ~2100 qps. Had to bump heap up to 128MB. > Python Twisted: ~1600 qps. Failed a lot of requests on most test runs. > Python Tornado: ~1500 qps, but rock solid through multiple test runs. > > So basically, an $11/mo VPS server running Javascript vastly exceeds > the capabilities of 10 backends at $60/mo each. > > Jeff > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/LArEHI-YEucJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
