Hi ckhan, I have debugged a similar problem for an internal app once. The root cause was that they were mixing synchronous and async API calls. In particular, inside a tasklet they were making a synchronous call. The synchronous call is run by invoking the event loop recursively; what may happen in this case is that another tasklet is spawned that does the same thing, and so on, until you have many recursive event loop invocations stacked on top of each other.
The solution is to find the synchronous call(s) in your tasklet(s) and replace them with "yield <async call>". If this is indirect, e.g. your tasklet calls one of your own non-tasklet helper functions which makes the synchronous call, you will have to convert the helper into a tasklet too. Good luck! It shouldn't be too hard to track this down. --Guido On Monday, October 29, 2012 12:31:55 PM UTC-7, ckhan wrote: > > Hello. I'm having trouble reading an exception log on my app. > > Quick background: > - request was made by taskqueue > - request creates several dozen tasklets that have this code path: > (module/function names simplified): > > mycode3.py:234 myFunc3() > --> mycode2.py:64 myFunc2() > --> mycode1.py:37 myFunc1 > > myFunc1 is yielding on async urlfetch; line 37 is the > @ndb.tasklet decorator > > - taskqueue was configured to re-try 2 more times, which it > did, each failing in exactly the same way > > Actual log: > > 2012-10-29 08:36:59.649 initial generator myFunc1(mycode1.py:37) raised > RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.654 initial generator myFunc1(mycode1.py:37) raised > RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.655 suspended generator myFunc2(mycode2.py:64) > raised RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.665 initial generator myFunc1(mycode1.py:37) raised > RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.669 initial generator myFunc1(mycode1.py:37) raised > RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.670 suspended generator myFunc2(mycode2.py:64) > raised RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.674 initial generator myFunc1(mycode1.py:37) raised > RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:36:59.674 suspended generator myFunc2(mycode2.py:64) > raised RuntimeError(maximum recursion depth exceeded) > W 2012-10-29 08:37:00.658 suspended generator myFunc3(mycode3:234) raised > RuntimeError(maximum recursion depth exceeded) > E 2012-10-29 08:37:04.055 > Exception: Deadlock waiting for <Future b36ce4ec0b0cf118 created by > __call__(_webapp25.py:712) for tasklet flush(context.py:239); pending> > Traceback (most recent call last): > File > "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/_webapp25.py", > > line 712, in __call__ > handler.post(*groups) > File > "/base/python_runtime/python_lib/versions/1/google/appengine/ext/ndb/tasklets.py", > > line 1050, in add_context_wrapper > ctx.flush().check_success() > File > "/base/python_runtime/python_lib/versions/1/google/appengine/ext/ndb/tasklets.py", > > line 322, in check_success > raise self._exception.__class__, self._exception, self._traceback > RuntimeError: Deadlock waiting for <Future b36ce4ec0b0cf118 created by > __call__(_webapp25.py:712) for tasklet flush(context.py:239); pending> > E 2012-10-29 08:37:04.086 Traceback (most recent call last): > E 2012-10-29 08:37:04.086 File > "/base/python_runtime/python_lib/versions/1/google/appengine/api/app_logging.py", > > line 98, in emit > E 2012-10-29 08:37:04.086 Traceback (most recent call last): > E 2012-10-29 08:37:04.086 File > "/base/python_runtime/python_lib/versions/1/google/appengine/api/app_logging.py", > > line 91, in emit > E 2012-10-29 08:37:04.086 Traceback (most recent call last): > E 2012-10-29 08:37:04.086 File > "/base/python_runtime/python_lib/versions/1/google/appengine/api/app_logging.py", > > line 98, in emit > E 2012-10-29 08:37:04.086 Traceback (most recent call last): > E 2012-10-29 08:37:04.086 File > "/base/python_runtime/python_lib/versions/1/google/appengine/api/app_logging.py", > > line 91, in emit > E 2012-10-29 08:37:04.086 Traceback (most recent call last): > E 2012-10-29 08:37:04.086 File > "/base/python_runtime/python_lib/versions/1/google/appengine/api/app_logging.py", > > line 98, in emit > > The warnings for the generators would appear to be from 9 separate, > otherwise unrelated tasklets that are all executing concurrently and > all happen to be at different points in the code path. Is that > correct? > > The recursion depth error seems to come from the app_logging. But is > that the underlying cause, or the symptom of the failure? If the > latter, how do I narrow what got me into this situation? > > Any help much appreciated. > -ckhan > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/kOophWUiNpsJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
