I think it's not related to a particular User as we see really a lot
of HardDeadlineExceededError errors (followed by the error code 104 message
at the end of log) for the last few days, which is very disturbing.
>From what I've seen, it happens when request is routed to a new instance
that fails to start in 60 seconds (although our application usually start
in less than 20). In our case HardDeadlineExceededError was thrown mostly
on jar file read (which is beyond our control):
Failed startup of context
com.google.apphosting.utils.jetty.RuntimeAppEngineWebAppContext@1d6e3d2{/,/base/data/home/apps/s~bugdigger-hr/241r2.361896469816654578}
com.google.apphosting.api.DeadlineExceededException: This request
(878f5068315d018e) started at 2012/09/24 21:26:31.318 UTC and was still
executing at 2012/09/24 21:27:30.767 UTC.
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:231)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.io.PushbackInputStream.read(PushbackInputStream.java:186)
at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238)
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158)
at java.util.zip.ZipInputStream.read(ZipInputStream.java:163)
at java.util.jar.JarInputStream.read(JarInputStream.java:194)
at java.util.zip.ZipInputStream.closeEntry(ZipInputStream.java:109)
at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:87)
at java.util.jar.JarInputStream.getNextEntry(JarInputStream.java:135)
at java.util.jar.JarInputStream.getNextJarEntry(JarInputStream.java:166)
at
net.sourceforge.stripes.vfs.DefaultVFS.listResources(DefaultVFS.java:169)
at net.sourceforge.stripes.vfs.DefaultVFS.list(DefaultVFS.java:62)
at net.sourceforge.stripes.vfs.VFS.list(VFS.java:200)
at net.sourceforge.stripes.util.ResolverUtil.find(ResolverUtil.java:202)
at
net.sourceforge.stripes.util.ResolverUtil.findImplementations(ResolverUtil.java:164)
at
net.sourceforge.stripes.config.BootstrapPropertyResolver.getClassPropertyList(BootstrapPropertyResolver.java:222)
at
net.sourceforge.stripes.config.RuntimeConfiguration.init(RuntimeConfiguration.java:298)
at
net.sourceforge.stripes.controller.StripesFilter.init(StripesFilter.java:125)
at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662)
at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
at
org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
at
org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
at
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
at
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:219)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
at
com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
at
com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
at com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
at
com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
at
com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
at java.lang.Thread.run(Thread.java:679)
Doesn't look good.
On Monday, September 24, 2012 11:39:12 PM UTC+2, Rodrigo Kossmann wrote:
>
> Hi guys.
> Happens the following error when entering my application with a specific User:
>
>
> 1.
>
> 201.86.195.206 - - [24/Sep/2012:14:24:48 -0700] "GET / HTTP/1.1" 500 0 -
> "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko)
> Chrome/21.0.1180.89 Safari/537.1" "tci.chainit.com.br" ms=60020 cpu_ms=22858
> exit_code=104 instance=00c61b117c6e51b4aec6d2e4089516b479c2a0
> <https://appengine.google.com/instances?app_id=s~proeloc&version_id=10-21.361990932472860292&key=00c61b117c6e51b4aec6d2e4089516b479c2a0#00c61b117c6e51b4aec6d2e4089516b479c2a0>
>
> 2. E2012-09-24 18:24:47.974
>
> org.datanucleus.transaction.Transaction rollback: Operation rollback
> failed on resource:
> org.datanucleus.store.appengine.DatastoreXAResource@2d659, error code UNKNOWN
> and transaction: [DataNucleus Transaction, ID=Xid=
>
> 3. W2012-09-24 18:24:48.375
>
> Error for /
> com.google.apphosting.runtime.HardDeadlineExceededError: This request
> (b08d8999601781ee) started at 2012/09/24 21:23:48.426 UTC and was still
> executing at 2012/09/24 21:24:48.373 UTC.
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:263)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
> at
> com.google.appengine.tools.development.TimedFuture.get(TimedFuture.java:41)
> at
> com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.datastore.FutureHelper$CumulativeAggregateFuture.get(FutureHelper.java:145)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
> at
> com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:113)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:110)
> at
> com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java:31)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:110)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:94)
> at
> com.google.apphosting.runtime.jetty.DatastoreSessionStore.saveSession(DatastoreSessionStore.java:94)
> at
> com.google.apphosting.runtime.jetty.SessionManager$AppEngineSession.save(SessionManager.java:167)
> at
> com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:41)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
> at
> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
> at
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
> at
> com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:146)
> at
> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
> at
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
> at
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
> at com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
> at
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
> at
> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
> at java.lang.Thread.run(Thread.java:679)
>
> 4. C2012-09-24 18:24:48.376
>
> Uncaught exception from servlet
> com.google.apphosting.runtime.HardDeadlineExceededError: This request
> (b08d8999601781ee) started at 2012/09/24 21:23:48.426 UTC and was still
> executing at 2012/09/24 21:24:48.373 UTC.
> at sun.misc.Unsafe.park(Native Method)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
> at
> com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:263)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96)
> at
> com.google.appengine.tools.development.TimedFuture.get(TimedFuture.java:41)
> at
> com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:63)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.datastore.FutureHelper$CumulativeAggregateFuture.get(FutureHelper.java:145)
> at
> com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
> at
> com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
> at
> com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:113)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl$2.runInternal(DatastoreServiceImpl.java:110)
> at
> com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java:31)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:110)
> at
> com.google.appengine.api.datastore.DatastoreServiceImpl.put(DatastoreServiceImpl.java:94)
> at
> com.google.apphosting.runtime.jetty.DatastoreSessionStore.saveSession(DatastoreSessionStore.java:94)
> at
> com.google.apphosting.runtime.jetty.SessionManager$AppEngineSession.save(SessionManager.java:167)
> at
> com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:41)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
> at
> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
> at
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
> at
> com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:146)
> at
> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
> at
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
> at
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
> at com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
> at
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
> at
> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
> at java.lang.Thread.run(Thread.java:679)
>
> 5. W2012-09-24 18:24:48.427
>
> A problem was encountered with the process that handled this request,
> causing it to exit. This is likely to cause a new process to be used for the
> next request to your application. If you see this message frequently, you may
> be throwing exceptions during the initialization of your application. (Error
> code 104)
>
>
> We believe that this User object is in some state which impedes saving.
> Someone has gone through this error before?
> Thanks.
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/1iO5eBhsJQ0J.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
