Hi Jon I appreciate the look and the feedback, other than the index the timeouts really only started occuring recently so I am presuming they were caused by long URL fetches (which also only seemed to have occurred recently - the twitter api hasn't been slow from another service I run)
For sure, the datastore timeouts may be at the end, but now I am only really seeing errors because I put a lot of retry logic inplace. The inserts are the end of the script because I was trying to be performant by using a batch put. I think On 28 May 2009, at 00:16, Jon McAlister <[email protected]> wrote: > > Hi Paul, > > The reason the index building has taken this long for you is just > because of the amount of data your app has in the datastore. However, > our index building workflow is not very loaded right now so I reset > your "Workflow Backend Index Task Count" quota level and it should go > a bit faster now. Afaict, it's been building now for 12 hours, but it > is now about 80% done. A big feature I'd like to see done is making > this data available in the admin console so you can track the progress > more than just the "Building|Serving|Error" we show now. Hopefully we > should have that out soon. > > Also, about the datastore timeouts, I took a brief glance at your > error logs. It appears as if you are only starting the datastore > request near the end of the 30-second request deadline. It looks like > you do some urlfetch calls and those sometimes take up to 30 seconds, > and then you kick off one datastore call and it deadlines. Have a look > at the timestamps in the logs and you'll see what I mean. I'm sure > this doesn't account for all of your datastore timeouts, but I did see > a fair amount of that going on. > > Jon > > On Wed, May 27, 2009 at 11:33 AM, Paul Kinlan > <[email protected]> wrote: >> Hi Guys. >> >> AppId: Twitterautofollow. >> >> I have three main issues at the moment. >> >> I am seeing an insane number of Datastore timeouts at the moment >> reads and >> writes - this has been occurring for a couple of days now? >> I understand there have been issues with UrlFetch yesterday. But I >> am still >> seeing a massive number of issues with requests timing out. >> I created one new index today and now I am seeing "Your application >> is >> exceeding a quota: Workflow Backend Index Task Count" It is a >> relativly >> simple index that includes a __key__ property. The object in >> question has >> no List Properties. >> >> I am getting increasingly frustrated at the moment, especially >> since I am >> paying a relativly significant amount each month now. >> >> Paul. >> >>> >> > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
