Task Queues and Cron Tasks are not running while we are waiting for the datastore to come back completely. Sorry, I've updated the downtime notify thread to point this out.
On May 25, 2:38 pm, Harlan Crystal <[email protected]> wrote: > Logs are showing "InternalError: internal error." being thrown in all my apps. > > On Tue, May 25, 2010 at 5:37 PM, Harlan Crystal > > > > <[email protected]> wrote: > > +1. > > > It is happening to all of my apps. Task queues are not being processed. > > > On Tue, May 25, 2010 at 5:31 PM, Juan Pablo Guereca <[email protected]> > > wrote: > >> Hi, taskqueues don't work in my app, the tasks are added to the queue but > >> they don't get executed ( I see no entry in the log and no retries ). > > >> Cheers, > >> JP > > >> On 25/05/10 22:50, App Engine Team wrote: > > >>> App Engine is currently experiencing a datastore outage: > > >>>http://groups.google.com/group/google-appengine-downtime-notify/brows... > > >>> Things are improving now, but we're still seeing some high latency. > >>> Please let us know if you are still seeing issues. > > >>> On May 25, 1:20 pm, Jeff Schnitzer<[email protected]> wrote: > > >>>> I'm also seeing 500 errors. My logs are full of this: > > >>>> com.google.apphosting.api.ApiProxy$CapabilityDisabledException: The > >>>> API call datastore_v3.Put() is temporarily unavailable. > >>>> at > >>>> com.google.net.rpc.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.jav > >>>> a:1025) > >>>> at > >>>> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.jav > >>>> a:444) > >>>> at > >>>> com.google.tracing.TraceContext.runInContext(TraceContext.java:684) > >>>> at > >>>> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInherited > >>>> ContextNoUnref(TraceContext.java:322) > >>>> at > >>>> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInherited > >>>> Context(TraceContext.java:314) > >>>> at > >>>> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java: > >>>> 442) > >>>> at > >>>> com.google.net.rpc.RpcStub$RpcCallbackDispatcher.rpcFinished(RpcStub.java:1 > >>>> 046) > >>>> at com.google.net.rpc.RPC.internalFinish(RPC.java:2047) > >>>> at > >>>> com.google.net.rpc.impl.RpcNetChannel.finishRpc(RpcNetChannel.java:2338) > >>>> at > >>>> com.google.net.rpc.impl.RpcNetChannel.messageReceived(RpcNetChannel.java:12 > >>>> 65) > >>>> at > >>>> com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319) > >>>> at > >>>> com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290) > >>>> at > >>>> com.google.net.async.Connection.handleReadEvent(Connection.java:474) > >>>> at > >>>> com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.j > >>>> ava:831) > >>>> at > >>>> com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207) > >>>> at > >>>> com.google.net.async.EventDispatcher.loop(EventDispatcher.java:103) > >>>> at > >>>> com.google.net.async.GlobalEventRegistry$2.runLoop(GlobalEventRegistry.java > >>>> :95) > >>>> at > >>>> com.google.net.async.LoopingEventDispatcher$EventDispatcherThread.run(Loopi > >>>> ngEventDispatcher.java:378) > > >>>> Also, about 30 minutes ago I deployed but ended up with this error msg: > > >>>> ... > >>>> Will check again in 32 seconds > >>>> Will check again in 64 seconds > >>>> Closing update: new version is ready to start serving. > >>>> Rolling back the update. > >>>> java.io.IOException: Error posting to > >>>> URL:https://appengine.google.com/api/appversion/startserving?app_id=mobca... > >>>> 409 Conflict > >>>> No transaction is in progress. > > >>>> Debugging information may be found in > > >>>> /private/var/folders/pU/pUOPLRDNG9KrxpuBZ7mFp++++TI/-Tmp-/appengine-deploy5 > >>>> 053223885430039791.log > > >>>> (app is mobca-st) > >>>> Jeff > > >>>> On Tue, May 25, 2010 at 1:15 PM, Jerome<[email protected]> wrote: > > >>>>> Yes, same issue as yesterday... customers are screaming... Help! We > >>>>> are strong advocate of AppEngine, but these repeated outages are > >>>>> hurting very bad. > > >>>>>http://code.google.com/status/appenginesaysthere is high latency... > >>>>> But I am seeing the Data Store throwing read only mode exceptions > >>>>> right now, when it is not just a 500. > >>>>> If not that, this is this one: > >>>>> A serious problem was encountered with the process that handled this > >>>>> request, causing it to exit. This is likely to cause a new process to > >>>>> be used for the next request to your application. If you see this > >>>>> message frequently, you may be throwing exceptions during the > >>>>> initialization of your application. (Error code 104) > > >>>>> Please let us know what we should expect such as we can communicate > >>>>> with our customers which are dead in the water with our service right > >>>>> now! > > >>>>> Jerome > > >>>>> On May 25, 9:46 pm, Jesse K<[email protected]> wrote: > > >>>>>> App Engine just went down again -- seems the same as what was reported > >>>>>> yesterday in this thread: can't access apps, can't access admin > >>>>>> dashboard. > > >>>>>> App Engine Team, please get this under control! > > >>>>>> On May 24, 6:54 pm, Sanjay<[email protected]> wrote: > > >>>>>>> Seems like a more general issue, my tasks are stuck as well, and this > >>>>>>> is reflected in the task queue dashboard as well: > > >>>>>>>http://code.google.com/status/appengine/detail/taskqueue/2010/05/24#a... > > >>>>>>> On May 24, 6:52 pm, Harlan Crystal<[email protected]> wrote: > > >>>>>>>> Things look *very wrong* on my apps. For 2/3 of my most important > >>>>>>>> applications, 0 tasks are being run in the last minute. > > >>>>>>>> Help! I have no idea how to resolve this problem on my own. > > >>>>>>>> -harlan > > >>>>>>>> On Mon, May 24, 2010 at 9:35 PM, Wesley C > >>>>>>>> (Google)<[email protected]> wrote: > > >>>>>>>>> we've had high datastore latencies the past hour or so, but things > >>>>>>>>> seem to be getting back to normal now as i was able to apps and > >>>>>>>>> dashboards with no 500s. let us know if this isn't the case. > > >>>>>>>>> -- wesley > >>>>>>>>> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - > >>>>>>>>> "Core Python Programming", Prentice Hall, (c)2007,2001 > >>>>>>>>> "Python Fundamentals", Prentice Hall, (c)2009 > >>>>>>>>> http://corepythonprogramming.com > > >>>>>>>>> wesley.j.chun :: [email protected] > >>>>>>>>> developer relations :: google app engine > >>>>>>>>> @app_engine :: googleappengine.blogspot.com > > >>>>>>>>> -- > >>>>>>>>> You received this message because you are subscribed to the Google > >>>>>>>>> Groups "Google App Engine" group. > >>>>>>>>> To post to this group, send email to > >>>>>>>>> [email protected]. > >>>>>>>>> To unsubscribe from this group, send email to > >>>>>>>>> [email protected]. > >>>>>>>>> For more options, visit this group > >>>>>>>>> athttp://groups.google.com/group/google-appengine?hl=en. > > >>>>>>>> -- > >>>>>>>> You received this message because you are subscribed to the Google > >>>>>>>> Groups "Google App Engine" group. > >>>>>>>> To post to this group, send email to > >>>>>>>> [email protected]. > >>>>>>>> To unsubscribe from this group, send email to > >>>>>>>> [email protected]. > >>>>>>>> For more options, visit this group > >>>>>>>> athttp://groups.google.com/group/google-appengine?hl=en. > > >>>>>>> -- > >>>>>>> You received this message because you are subscribed to the Google > >>>>>>> Groups "Google App Engine" group. > >>>>>>> To post to this group, send email to > >>>>>>> [email protected]. > >>>>>>> To unsubscribe from this group, send email to > >>>>>>> [email protected]. > >>>>>>> For more options, visit this group > >>>>>>> athttp://groups.google.com/group/google-appengine?hl=en. > > >>>>> -- > >>>>> You received this message because you are subscribed to the Google > >>>>> Groups "Google App Engine" group. > >>>>> To post to this group, send email to [email protected]. > >>>>> To unsubscribe from this group, send email to > >>>>> [email protected]. > >>>>> For more options, visit this group > >>>>> athttp://groups.google.com/group/google-appengine?hl=en. > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]. > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
