Thank you for posting your error logs. According to 'Google Cloud Status <https://status.cloud.google.com/>', there are no know outages for Task Queues or App Engine modules at the moment.
As you've mentioned, the default deadline for a automatic scaled modules is 10 minutes. This is documented in 'Task deadlines <https://cloud.google.com/appengine/docs/python/taskqueue/overview-push#task_deadlines>'. Please note that the deadline can be as high as 24 hours for manual or basic scaling modules. Also note that the DeadlineExceededException is raised when the task's execution time nears the deadline, thus allowing you to save your work or log something before the deadline is actually exceeded. That being said, more information is required to get a better idea of what might really be the culprit here: 1. What is the .yaml configuration for the related task-handling modules? (feel free to censor any project-sensitive data) 2. What is the time frame? (When did this start; is it ongoing; if not, when did it end) 3. Have you encountered any other timeout or deadline exceeded errors from other modules, including those unrelated to task queues? 4. How is your queue.yaml configured <https://cloud.google.com/appengine/docs/python/config/queue> for the queues experiencing these issues? 5. Do retries <https://cloud.google.com/appengine/docs/python/taskqueue/overview-push#task_retries> or increased module scalability help reduce the frequency of these errors? Though the thread 'Suddenly App Engine not Serving Anymore: Error Code 123 <https://groups.google.com/forum/#!msg/google-appengine/ujF7fzOmEew/0HbZFIehAAAJ>' seems to describe similar errors, the scope of these errors and what systems are causing them is unclear in said thread so I would advise to continue here going forward. On Tuesday, January 19, 2016 at 9:01:51 PM UTC-5, Michael Sander wrote: > > Hi All, > > Over the past few days I've been getting a lot of Error Code 123 in my > Taksqueue. I understand that this error is a deadline, but Taskqueues are > supposed to live for 10 minutes, and I get this error far before that. It > is difficult to debug because these requests have practically no logging > information in them. My app normally spits out a lot of logs, so either GAE > is dropping my logs or this crash occurs outside of my code. > > > 1. 2016-01-19 20:36:45.991 /_ah/queue/deferred 500 190547ms 0kb > AppEngine-Google; > (+http://code.google.com/appengine) module=default version=18 > > 0.1.0.2 - - [19/Jan/2016:17:36:45 -0800] "POST /_ah/queue/deferred > HTTP/1.1" 500 0 "http://docketupdate.appspot.com/_ah/queue/deferred"; > "AppEngine-Google; (+http://code.google.com/appengine)" > "docketupdate.appspot.com" ms=190547 cpu_ms=0 cpm_usd=0.000065 > queue_name=default task_name=37491651647795200701 exit_code=123 > app_engine_release=1.9.31 instance=00c61b117c4864f5a842a8ae7ca496d20dbaa9f8 > <https://appengine.google.com/instances?app_id=s~docketupdate&version_id=18.390103371601440022&key=00c61b117c4864f5a842a8ae7ca496d20dbaa9f8#00c61b117c4864f5a842a8ae7ca496d20dbaa9f8> > > 2. E2016-01-19 20:36:45.991 > > Process terminated because the request deadline was exceeded. (Error code > 123) > > > > Here's another: > > > > 1. 2016-01-19 19:48:58.581 /_ah/queue/deferred 500 84488ms 0kb > AppEngine-Google; > (+http://code.google.com/appengine) module=default version=18 > > 0.1.0.2 - - [19/Jan/2016:16:48:58 -0800] "POST /_ah/queue/deferred > HTTP/1.1" 500 0 "http://docketupdate.appspot.com/_ah/queue/deferred"; > "AppEngine-Google; (+http://code.google.com/appengine)" > "docketupdate.appspot.com" ms=84489 cpu_ms=0 queue_name=indexsearch > task_name=5718499252586827456 exit_code=123 app_engine_release=1.9.31 > instance=00c61b117cbebad1360673a661a18cf0e69c7cb0 > <https://appengine.google.com/instances?app_id=s~docketupdate&version_id=18.390103371601440022&key=00c61b117cbebad1360673a661a18cf0e69c7cb0#00c61b117cbebad1360673a661a18cf0e69c7cb0> > > 2. E2016-01-19 19:48:58.580 > > Process terminated because the request deadline was exceeded. (Error code > 123) > > > > > And another: > > > 1. 2016-01-19 19:48:58.583 /_ah/queue/deferred 500 105886ms 0kb > AppEngine-Google; > (+http://code.google.com/appengine) module=default version=18 > > 0.1.0.2 - - [19/Jan/2016:16:48:58 -0800] "POST /_ah/queue/deferred > HTTP/1.1" 500 0 "http://docketupdate.appspot.com/_ah/queue/deferred"; > "AppEngine-Google; (+http://code.google.com/appengine)" > "docketupdate.appspot.com" ms=105887 cpu_ms=0 cpm_usd=0.009467 > queue_name=officialdocumentqueue task_name=8648504684653226301 exit_code=123 > app_engine_release=1.9.31 instance=00c61b117cbebad1360673a661a18cf0e69c7cb0 > <https://appengine.google.com/instances?app_id=s~docketupdate&version_id=18.390103371601440022&key=00c61b117cbebad1360673a661a18cf0e69c7cb0#00c61b117cbebad1360673a661a18cf0e69c7cb0> > > 2. E2016-01-19 19:48:58.583 > > Process terminated because the request deadline was exceeded. (Error code > 123) > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/11324aa8-729a-40bb-9657-d061835813c5%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
