Ok, so I started looking at Cloud SQL. To add an entry, appstats says the following: rdbms.OpenConnection 182 msec rdbms.Exec 144 msec rdbms.ExecOp 82 msec rdbms.CloseConnection 45msec
It seems to me that I should be able to cache the connection and keep it open for the life of the instance .... or at least while the instance is receiving a lot of scores. This should remove almost half the DB overhead. However, I don't know enough to understand under what conditions the DB connection would be dropped, how to detect the connection is stale and most importantly, how to close the connection when the instance is torn down. Does anyone have a code pattern for this sort of thing ? -R On Thursday, August 2, 2012 2:10:59 PM UTC-4, Takashi Matsuo (Google) wrote: > > Richard, > > Yes. The result is almost the same with 20 B1. Apparently the bottleneck > is within somewhere other than backends themselves. > > > On Fri, Aug 3, 2012 at 2:19 AM, Richard <[email protected]> wrote: > >> Thanks for the testing and benchmarking Takashi! >> >> Considering B4 = 4x B1 (we not really, but this is to illustrate a >> point!) ... and you had 20 of them, you effectively managed 200 QPS with 80 >> x B1's..... that's umm... 2.5 QPS each ?!? That's crazy! I think I see a >> performance development project in someone's future :) >> >> >> On Thursday, August 2, 2012 12:50:53 PM UTC-4, Takashi Matsuo (Google) >> wrote: >>> >>> On Fri, Aug 3, 2012 at 12:46 AM, Richard <[email protected]> wrote: >>> >>>> Well, I tried to save Score() objects with a key name equal to the >>>> username (guaranteed unique) last night. GAE had the same problem in the >>>> wee hours of the morning when (most) of my users are not playing (only >>>> around 100-150 people playing). So, apparently it is not a tablet issue. >>>> >>>> Time to play with Cloud SQL maybe ? >>> >>> >>> That's a good idea. Certainly 1k QPS is something Cloud SQL can handle >>> theoretically, and Cloud SQL has a strong consistency which is important >>> for your use case. I should have came to this conclusion earlier. Sorry >>> about that. I'll try to experiment with Cloud SQL tomorrow and get back to >>> you. >>> >>> BTW, I have dome some experiment withs in-memory score server using >>> backend instances, and it constantly marked around 200QPS with 20 B4 >>> backends. So it might not be a good fit for your need this time. >>> >>> -- Takashi >>> >>> >>>> >>>> On Thursday, August 2, 2012 12:36:04 AM UTC-4, Richard Watson wrote: >>>>> >>>>> If you reverse the user id on saving the score, it'll be random and >>>>> you'll avoid the tablet issue. >>>>> >>>>> On Wednesday, August 1, 2012 9:52:24 PM UTC+2, Richard wrote: >>>>>> >>>>>> Hi Takashi, >>>>>> Yeah, the first few characters would probably be very similar. I am >>>>>> not sure if that would be a problem ? >>>>>> >>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To view this discussion on the web visit https://groups.google.com/d/** >>>> msg/google-appengine/-/**BBYrAdRDF0EJ<https://groups.google.com/d/msg/google-appengine/-/BBYrAdRDF0EJ> >>>> . >>>> >>>> To post to this group, send email to google-appengine@googlegroups.** >>>> com <[email protected]>. >>>> To unsubscribe from this group, send email to >>>> google-appengine+unsubscribe@**googlegroups.com<google-appengine%[email protected]> >>>> . >>>> For more options, visit this group at http://groups.google.com/** >>>> group/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>>> . >>>> >>> >>> >>> >>> -- >>> Takashi Matsuo >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/google-appengine/-/eIvPq4dCXqAJ. >> >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > Takashi Matsuo > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/B_GguOuQtVgJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
