On Tue, Jul 31, 2012 at 11:08 PM, Richard <[email protected]> wrote: > Thank you for that interesting post. Some points though: > > - behavior is observed at the same time each day > - behavior is observed when QPS is lowest > - behavior is not observed at high QPS
The traffic patterns seems identical, so isn't it possible that the issue always happens after deleting bunch of entities? What happens if you just leave those entities instead of deleting them? > > Object datamodel: > > class Score(db.Model): > a = db.StringProperty(indexed=False) > b = db.ReferenceProperty(indexed=False) > c = db.StringProperty(indexed=False) > d = db.StringProperty(indexed=False) > e = db.StringProperty(indexed=False) > f = db.IntegerProperty(indexed=False) > g = db.StringProperty(indexed=False) > h = db.IntegerProperty(indexed=False) > i = db.IntegerProperty(indexed=False) > j = db.TextProperty(indexed=False) > > I really don't think this is the problem. Unless it is happening to another > app on the same hardware, and I am seeing issues too ? Not sure if that is > possible ? > > > On Tuesday, July 31, 2012 12:30:17 AM UTC-4, Takashi Matsuo (Google) wrote: >> >> Hi Richard, >> >> I'm still not 100% sure, but my first guess is that it might be >> because of datastore tablets splitting/merging behavior. When you >> create or delete lots of entities, the datastore tablets might be >> splited/merged into larger/fewer number of tablets. If that happens, >> you may experience such high latencies. >> >> This blog post by Ikai explains this issue very well: >> >> http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/ >> >> Questions are: >> How often those entities are created? How much QPS? >> What kind of keys do you use for those small entities? >> >> If you write entities at a somewhat high rate like >100QPS, and you're >> using keys which are not distributed well, the culprit might be >> splitting/merging behavior mentioned above. >> >> Also, maybe you can consider using pull queues for such tasks. >> >> -- Takashi >> >> >> On Tue, Jul 31, 2012 at 4:04 AM, Richard <[email protected]> wrote: >> > And, of course.... including an image from Google docs does not actually >> > work. Grrr! >> > >> > >> > https://docs.google.com/drawings/pub?id=1nbrpRb7bSYIvS7329fY2jmynKCJOtUtlCMImo0wXnpE&w=1440&h=1080 >> > >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msg/google-appengine/-/tGsoCfpFC_kJ. >> > >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> >> >> >> -- >> Takashi Matsuo > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/Sgdd6z4nzPsJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- Takashi Matsuo -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
