Hi, We have a cron job that runs every hour on a backend module and creates tasks. The cron job runs queries on the Cloud SQL database, and the tasks make HTTP calls to other servers and also update the database. Normally they run great, even when thousands of tasks as created, but sometimes it gets "stuck" and there is nothing in the logs that can shed some light on the situation. For example, yesterday we monitored the cron job while it created a few tens of tasks and then it stopped, along with 8 of the tasks that also got stuck in the queue. When it was obvious that nothing was happening we ran the process a few more times and each time completed successfully.
After a day the original task was killed with a DeadlineExceededException (as expected), and the 8 tasks with an error 104. Until then we saw absolutely no record of them in the logs, and now that we see them there are no log records before the time of the DeadlineExceededException, so we have no idea at what point they got stuck. We checked the processes running on the Cloud SQL instance during that time and found only a dozen idle connections. What could be the cause of this and how can we investigate it further? Trace data for the cron job: http://imgur.com/a/H5wGG. Our module's class and scaling configuration is: <instance-class>B4</instance-class> <basic-scaling> <max-instances>11</max-instances> <idle-timeout>10m</idle-timeout> </basic-scaling> The configuration of the queue is: <rate>5/s</rate> <max-concurrent-requests>100</max-concurrent-requests> <mode>push</mode> <retry-parameters> <task-retry-limit>5</task-retry-limit> <min-backoff-seconds>10</min-backoff-seconds> <max-backoff-seconds>200</max-backoff-seconds> </retry-parameters> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/2ab9279c-fa1a-4a3c-bb97-3a4ec885e51b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
