There was some extended latency on Tuesday after the maintenance period which likely explained the errors you were seeing earlier. Are you back up to 200 keys? Given Don's calculations, if each key required 9 ms of time, then you should be able to scale up to 500 keys, so there is no misrepresentation in our documentation. - Jason
On Wed, Aug 19, 2009 at 11:20 AM, repairman <[email protected]> wrote: > > thanks for your input. you said in your new app you think you can > delete 200 keys per request given the 30-sec timeout, however, your > quota page > http://code.google.com/appengine/docs/java/datastore/overview.html#Quotas_and_Limits > says the max batch delete is 500 which means something is still wrong > that bottlenecks the delete performance that it never hit the 500 mark > based on your testing. In other words, ppl who set the batch delete > size to 499 thinking it falls under the 500 limit will still get error > (just a different error). Can you either file a bug about slow delete > that will never hit 500 batch size as stated in the doc or change the > documentation on that page to say 200 instead of 500. It looks like a > pit fall for everyone to fall into. this kinda things is something > Micro$oft does and google should do better. thanks. > > > On Aug 19, 8:46 am, Don Schwarz <[email protected]> wrote: > > On Tue, Aug 18, 2009 at 11:02 PM, repairman <[email protected]> wrote: > > > > > don, > > > > > my appid is bubblesixapp, it used to work before the 8/18 maintenance > > > with 200 keys in the datastore.delete(listKey). Then i lowered it to > > > 100 today and still got the exception, then I lowered it to 5 keys and > > > it worked. this mean the latency on datastore.delete is much much > > > longer now. I think there is a performance degradation on delete that > > > no one is detecting. below is my code. can you try to insert 1000 > > > objects then try to delete 200 keys at a time. Do you guys have a > > > guide line on how 'slow' is normal? can you please check? let's not > > > have any system performance degradation go unnoticed. thanks. > > > > Deletion of a single key is fairly stable at 50ms right now, as reported > > both by the Status Site link I posted early and our internal cluster-wide > > monitoring. This is, if anything, slightly lower than it was several > days > > ago before the maintenance occurred. > > > > I did some quick testing and at least in a brand new application, bulk > > deletes take approximately 9-12ms per key. Since the deadline for > > individual API calls is currently 5 seconds, I would say that 200 should > > generally be a fairly safe batch size. > > > > It is possible that a small number of individual applications were > adversely > > affected by the maintenance and will take a few days for their > performance > > to normalize. It is also possible that the rate at which you are > creating > > and deleting entities is affecting your performance (i.e. the more often > you > > run this code in a 10-15 minute period of time the slower it will be). > > > > I would recommend choosing a relatively low batch size (did you try 50?) > and > > monitoring the performance over a few days. Bigtable (upon which the App > > Engine datastore is built) is optimized for steady-state performance, and > > your application isn't getting much traffic at the moment. If you want > to > > know more about how bigtable works internally, you can refer to the > research > > paper: > > > > http://labs.google.com/papers/bigtable.html > > > > Thanks, > > Don > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~----------~----~----~----~------~----~------~--~---
