This definitely sounds like a good strategy... especially with memory tables and no indexes.
Back when cloud sql was in beta, there was a restriction of 5 queries per second, so I "wrote it off" for doing any kind of heavy lifting. Presumably that restriction has been lifted. Jeff On Wed, Aug 1, 2012 at 12:56 PM, Mauricio Aristizabal <[email protected]> wrote: > As others have said, why not Cloud SQL? > > Mysql can handle tens of thousands of inserts per second into an > average-width table: > http://www.mysqlperformanceblog.com/2012/05/16/benchmarking-single-row-insert-performance-on-amazon-ec2/ > > I don't know what kind of performance you can expect from Google's setup but > considering your table would be very narrow ( a couple number columns) and > with few to no indexes, you can probably get at least a couple thousand per > second. > > You may also consider it acceptable to use MEMORY tables. I just tested > creating one so they are supported. And if you do need to keep this data > for a while, doing a 'insert into innodbtable select * from memtable' is > much faster than individual inserts, so you can perhaps 'archive' your raw > entries after every leaderboard calculation and truncate. > http://dev.mysql.com/doc/refman/5.0/en/memory-storage-engine.html > > Lastly, you can then do your ranking in sql which may run a lot fater than > loading tens of thousands of records into the backend to do the calculation > in the app. > > Of course, this will not scale indefinitely, but in this particular use case > seems like sharding wouldn't be much of an issue: front end writes to any of > 4 sqldbs, reaper backend queries all 4 (use threads to do asynchronously), > collates results. > > > > On Sunday, July 29, 2012 7:19:25 AM UTC-7, Richard wrote: >> >> This is for all you guys who know app engine really well: >> >> I want to be able to move data RELIABLY and with low latency from many >> front end instances to a backend within 5 seconds. >> >> I am currently getting the F1's to do a db.put() on a lightweight object. >> The data comes from web clients. Around 10 seconds later, the backend reaps >> them. This should work nicely in theory. >> >> However, it is just unreliable. Sometimes it will handle a load of 600 db >> put()'s in that 10 second window.... and other times (in the same day!), it >> won't even have completed 100 put()'s in 10 seconds.... so when I do the >> reap... I get nothing! It seems put() has some problems... 'eventually >> consistent' is useless if it is several minutes later! >> >> How can I do this reliably in a 10 second window ? I have had using a >> PULL queue suggested, but I don't want to do all the work of converting the >> app over if it will be just as unreliable (I remember posts about >> tasksqueue's getting "stuck").... >> >> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/4skTYPy52hIJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
