As others have said, why not Cloud SQL? Mysql can handle tens of thousands of inserts per second into an average-width table: http://www.mysqlperformanceblog.com/2012/05/16/benchmarking-single-row-insert-performance-on-amazon-ec2/
I don't know what kind of performance you can expect from Google's setup but considering your table would be very narrow ( a couple number columns) and with few to no indexes, you can probably get at least a couple thousand per second. You may also consider it acceptable to use MEMORY tables. I just tested creating one so they are supported. And if you do need to keep this data for a while, doing a 'insert into innodbtable select * from memtable' is much faster than individual inserts, so you can perhaps 'archive' your raw entries after every leaderboard calculation and truncate. http://dev.mysql.com/doc/refman/5.0/en/memory-storage-engine.html Lastly, you can then do your ranking in sql which may run a lot fater than loading tens of thousands of records into the backend to do the calculation in the app. Of course, this will not scale indefinitely, but in this particular use case seems like sharding wouldn't be much of an issue: front end writes to any of 4 sqldbs, reaper backend queries all 4 (use threads to do asynchronously), collates results. On Sunday, July 29, 2012 7:19:25 AM UTC-7, Richard wrote: > > This is for all you guys who know app engine really well: > > I want to be able to move data RELIABLY and with low latency from many > front end instances to a backend within 5 seconds. > > I am currently getting the F1's to do a db.put() on a lightweight object. > The data comes from web clients. Around 10 seconds later, the backend > reaps them. This should work nicely in theory. > > However, it is just unreliable. Sometimes it will handle a load of 600 db > put()'s in that 10 second window.... and other times (in the same day!), it > won't even have completed 100 put()'s in 10 seconds.... so when I do the > reap... I get nothing! It seems put() has some problems... 'eventually > consistent' is useless if it is several minutes later! > > How can I do this reliably in a 10 second window ? I have had using a > PULL queue suggested, but I don't want to do all the work of converting the > app over if it will be just as unreliable (I remember posts about > tasksqueue's getting "stuck").... > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/4skTYPy52hIJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
