I think the 5 QPS restriction is still there... but it's only for connections from outside GAE, AFAIK to discourage using CloudSQL from an outside app. For GAE there are only limits on concurrent connections (1000) and requests (100), and request/response size (16MB)
http://stackoverflow.com/questions/10424442/what-are-the-connection-limits-for-google-cloud-sql-from-app-engine-and-how-to https://developers.google.com/cloud-sql/faq#sizeqps On Wednesday, August 1, 2012 2:15:13 PM UTC-7, Jeff Schnitzer wrote: > > This definitely sounds like a good strategy... especially with memory > tables and no indexes. > > Back when cloud sql was in beta, there was a restriction of 5 queries > per second, so I "wrote it off" for doing any kind of heavy lifting. > Presumably that restriction has been lifted. > > Jeff > > On Wed, Aug 1, 2012 at 12:56 PM, Mauricio Aristizabal wrote: > > As others have said, why not Cloud SQL? > > > > Mysql can handle tens of thousands of inserts per second into an > > average-width table: > > > http://www.mysqlperformanceblog.com/2012/05/16/benchmarking-single-row-insert-performance-on-amazon-ec2/ > > > > > I don't know what kind of performance you can expect from Google's setup > but > > considering your table would be very narrow ( a couple number columns) > and > > with few to no indexes, you can probably get at least a couple thousand > per > > second. > > > > You may also consider it acceptable to use MEMORY tables. I just tested > > creating one so they are supported. And if you do need to keep this > data > > for a while, doing a 'insert into innodbtable select * from memtable' is > > much faster than individual inserts, so you can perhaps 'archive' your > raw > > entries after every leaderboard calculation and truncate. > > http://dev.mysql.com/doc/refman/5.0/en/memory-storage-engine.html > > > > Lastly, you can then do your ranking in sql which may run a lot fater > than > > loading tens of thousands of records into the backend to do the > calculation > > in the app. > > > > Of course, this will not scale indefinitely, but in this particular use > case > > seems like sharding wouldn't be much of an issue: front end writes to > any of > > 4 sqldbs, reaper backend queries all 4 (use threads to do > asynchronously), > > collates results. > > > > > > > > On Sunday, July 29, 2012 7:19:25 AM UTC-7, Richard wrote: > >> > >> This is for all you guys who know app engine really well: > >> > >> I want to be able to move data RELIABLY and with low latency from many > >> front end instances to a backend within 5 seconds. > >> > >> I am currently getting the F1's to do a db.put() on a lightweight > object. > >> The data comes from web clients. Around 10 seconds later, the backend > reaps > >> them. This should work nicely in theory. > >> > >> However, it is just unreliable. Sometimes it will handle a load of 600 > db > >> put()'s in that 10 second window.... and other times (in the same > day!), it > >> won't even have completed 100 put()'s in 10 seconds.... so when I do > the > >> reap... I get nothing! It seems put() has some problems... > 'eventually > >> consistent' is useless if it is several minutes later! > >> > >> How can I do this reliably in a 10 second window ? I have had using a > >> PULL queue suggested, but I don't want to do all the work of converting > the > >> app over if it will be just as unreliable (I remember posts about > >> tasksqueue's getting "stuck").... > >> > >> > > -- > > You received this message because you are subscribed to the Google > Groups > > "Google App Engine" group. > > To view this discussion on the web visit > > https://groups.google.com/d/msg/google-appengine/-/4skTYPy52hIJ. > > > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/xaQA1_USc5cJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
