Tim, your app is probably not doing much urlfetching? We have an app (h-script) that performs several urlfetch operations per task. We are talking to other Google services only. On a good day, each task takes between 3 and 10 seconds. This goes up to 20 seconds sometimes. The CPU time for the tasks is minimal - 60-100 ms. It's all spent in urlfetch.
This should present GAE with no problem - we time out gracefully, etc. Trying to run this full bore (several queues, 50/s) results in 98+% task failure rate. Running across 5 queues and at 6/m drops that to about 30% failure (and MUCH longer processing time obviously). With the back-off kicking in, some tasks end up running all day. I understand the 1sec rule, but it supposedly does NOT apply to tasks. And with a dependence on urlfetch, there is no way to optimize to that level. As it stands, there is either a bug in there somewhere triggering a clampdown erroneously, or a limiter Google has not documented... In either case, it makes it very difficult to do certain things that GAE should be awesome at. I would guess your usecase is different..? J On Aug 26, 8:24 pm, Tim Hoffman <[email protected]> wrote: > Unfortunately you haven't provided a great deal of information about > what you are doing in your tasks. > > My guess is whatever you are running in the task is just taking too > long. > > Queued will retry unless you exit cleanly. And each failed tasks time > between retries will > will increase. > > You should put some logging in your task to see where it is getting > too. Then you will have some idea > where the bottleneck is. > > The task queue subsystem is in my experience quite robust and reliable > if you > code you tasks taking into account its limitations. > > Regards > > Tim > > On Aug 26, 2:06 pm, "Jan Z/ Hapara" <[email protected]> wrote: > > > Not relevant unfortunately. > > > There is not try / catch here - when this happens, your code doesn't > > even start executing - GAE just fails it outright, and worse yet, the > > back-off algorithm kicks in so after a few generations the whole thing > > degenerates into uselessness (I've seen tasks that fail with this > > error 23 times in a row) > > > Jan > > > On Aug 17, 6:58 am, Alon Carmel <[email protected]> wrote: > > > > When you fetch data from another server you use the urlfetch service. url > > > fetch service holds another limit for requests. when you relay on external > > > servers they hold another load which you cannot expect on your end. so if > > > an > > > external service has some latency suddenly and plus fall under a database > > > high load your tasks will fail enventually. > > > > try > > > > catch > > > > :) > > > > - > > > Cheers, > > > > def AlonCarmel(request) > > > import simplejson as json > > > contact = {} > > > contant['email'] = '[email protected]' > > > contact['twitter'] = '@aloncarmel' > > > contact['web'] = 'http://aloncarmel.me' > > > contact['phone'] = '+972-54-4860380' > > > return HttpResponse(json.dumps(contact)) > > > > * If you received an unsolicited email from by mistake that wasn't of your > > > matter please delete immediately. All E-mail sent from Alon Carmel is > > > copyrighted to Alon Carmel 2008. Any details revealed in e-mails sent by > > > Alon Carmel are owned by the Author only. Any attempt to duplicate or > > > imitate any of the Content is prohibited under copyright law 2008. > > > > On Sat, Aug 14, 2010 at 8:28 PM, Dmitry <[email protected]> wrote: > > > > Hi app team! > > > > > "Request was aborted after waiting too long to attempt to service your > > > > request. This may happen sporadically when the App Engine serving > > > > cluster is under unexpectedly high or uneven load. If you see this > > > > message frequently, please contact the App Engine team." > > > > > What may cause this error? I'm getting this quite often (may be every > > > > 1 minute) when using task queue. Queue rate isn't so big (5-7 > > > > simultaneous requests, ~10 sec in average). Task just fetchs data from > > > > another server and processes it. > > > > > thx > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to [email protected]. > > > > To unsubscribe from this group, send email to > > > > [email protected]<google-appengine%2Bunsubscrib > > > > [email protected]> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
