We are also seeing fairly normal operation today.
On Thursday, December 3, 2015 at 9:21:08 AM UTC-8, Jim wrote: > > We have been seeing latency of 5 hours or more at times, although it seems > to be functioning normally today. > > > > On Thursday, December 3, 2015 at 11:07:22 AM UTC-6, Rene Marty wrote: >> >> Dear Google Team, >> >> I send this issue three weeks ago to one of your developers advocates. >> I'm very experienced in GAE (three years developing mission critical apps) >> and if you need my support or testing, it will be a pleasure. >> >> I think there are two issues with the taskqueue: >> 1) The start of execution is delayed, between 15 and 120 seconds from the >> estimated time of execution. Until three weeks ago, it was inmediate, only >> 1 or 3 seconds of delay at rush hour. >> 2) When the task is being filled with processes not starting, the process >> that called the taskqueue.add(...) freezes waiting for the response, and >> subsecuents request to the same instance fails and causes a Deadline >> Exceeded 123 error. I discover that this is the cause beacuse when I >> replaced all the taskqueue.add(...) to async urlfetch.make_fetch_call(...) >> without waiting for the rpc result, I never get again that Deadline >> Exceeded 123 error. >> >> Please try to solve this issue ASAP. The taskqueue functionality is >> critical for many developers trying to offer a good sutomer experience. If >> this becomes unstable, we will need to look another platforms more robust, >> I'm a big fan of Google, but I don't want loose customers due to platform >> issues. At the end, my clients blame my company not to Google for these >> problems... >> >> Thanks, >> >> Rene Marty >> >> El jueves, 3 de diciembre de 2015, 13:09:28 (UTC-3), Jim escribió: >>> >>> David, thanks I think you're right that looks like the issue. >>> >>> Google, it sure would be nice if the status page was more granular and >>> showed the status of the task queue separately like it does the datastore >>> and cloud storage. Task Queues are a critically important part of our >>> application architecture and having zero visibility into issues related to >>> them is really a pain when things like this happen. It's understandable >>> that issues occur, but it sure would be nice to be able to know that Google >>> engineers are aware of them and are working on them without poking around >>> blind. >>> >>> >>> >>> On Wednesday, December 2, 2015 at 6:05:45 PM UTC-6, David Fischer wrote: >>>> >>>> I think this is the status for this event: >>>> https://status.cloud.google.com/incident/appengine/15024 >>>> >>>> On Wednesday, December 2, 2015 at 11:55:04 AM UTC-8, Jim wrote: >>>>> >>>>> Can anyone from Google comment on this? >>>>> >>>>> >>>>> On Wednesday, December 2, 2015 at 8:40:24 AM UTC-6, John Wheeler wrote: >>>>>> >>>>>> Whew! So it's not just me then. I've got exactly the problem. >>>>>> >>>>>> I've attached a picture of my dashboard graph for the last 30 days >>>>>> which you can see very regular peaks at the beginning turn into >>>>>> irregularity since I've been experiencing issues with task queues. >>>>>> >>>>>> John >>>>>> >>>>>> On Tuesday, December 1, 2015 at 8:03:25 AM UTC-8, Jim wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> Has anyone else noticed a lot of latency in Task Queues within the >>>>>>> past few days? We've noticed that our push queues are backing up with >>>>>>> thousands, sometimes tens of thousands of tasks. Nothing has changed >>>>>>> in >>>>>>> our app or traffic profile. Over the past few days we've seen several >>>>>>> instances where the queues just seem to stop "pushing" tasks and they >>>>>>> fill >>>>>>> up. We can go in and manually trigger individual tasks to run; tasks >>>>>>> are >>>>>>> not generating errors, they're just not running automatically like they >>>>>>> should based on our configuration. Eventually things seem to catch up >>>>>>> again. Anyone else seeing similar behavior? >>>>>>> >>>>>>> Jim >>>>>>> >>>>>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/5ce48f93-c4e0-431b-96c2-f880c5a8ee06%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
