More information: * The failure began at 10:54am (pacific). * Same app on different appid has the same problem.
(as listed in the stacktraces, the appid is voost0) Jeff On Wed, Sep 12, 2012 at 11:53 AM, Jeff Schnitzer <[email protected]> wrote: > HEEEEEEEEEEEEELP! > > We have tried everything at this point. Shut down instances, tried to > deploy a new version, even tried old versions. We've reported a > production issue. Something is broken inside of GAE. The Guice error > must be a symptom; the smoking gun seems to be: > > java.lang.ClassNotFoundException: java.io.FileOutputStream > > We've been down for 30 mins and getting complaints from our clients. > This looks really, really bad. It's my nightmare scenario - an outage > in GAE that is small enough not to raise major alarm bells, but > nevertheless cripples my business. It is not the first time this has > happened. It is shaking my faith in GAE. > > Jeff > > On Wed, Sep 12, 2012 at 11:11 AM, Jeff Schnitzer <[email protected]> wrote: >> Our app has been running fine on the same version, no code deploys >> (our versions are timestamped so this is 100% certain), for two days. >> All of a sudden (10 mins ago) our app stopped running. Every attempt >> to run a request produces this cryptic message: >> >> -------- >> >> 2012-09-12 10:56:21.791 >> com.google.inject.servlet.GuiceFilter setPipeline: Multiple Servlet >> injectors detected. This is a warning indicating that you have more >> than one GuiceFilter running in your web application. If this is >> deliberate, you may safely ignore this message. If this is NOT >> deliberate however, your application may not work as expected. >> D 2012-09-12 10:56:21.792 >> st.voo.tick.util.cambridge.CambridgeSetup <init>: Establishing >> cambridge view resolver >> I 2012-09-12 10:56:21.792 >> st.voo.tick.GuiceConfig contextInitialized: Guice initialization took 514 >> millis >> W 2012-09-12 10:56:21.888 >> Failed startup of context >> com.google.apphosting.utils.jetty.RuntimeAppEngineWebAppContext@1479784{/,/base/data/home/apps/s~voost0/2012-09-10-1715.361669184733923098} >> java.lang.RuntimeException: java.lang.RuntimeException: Unable to >> instantiate MessageBodyReader >> at >> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:35) >> at >> org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:211) >> at >> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:67) >> at >> org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39) >> at >> st.voo.tick.util.GuiceResteasyFilterDispatcher.init(GuiceResteasyFilterDispatcher.java:48) >> at >> com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114) >> at >> com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98) >> at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172) >> at >> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662) >> at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) >> at >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:219) >> at >> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194) >> at >> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134) >> at >> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447) >> at >> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452) >> at >> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459) >> at >> com.google.tracing.TraceContext.runInContext(TraceContext.java:701) >> at >> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336) >> at >> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328) >> at >> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456) >> at >> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251) >> at java.lang.Thread.run(Thread.java:679) >> Caused by: java.lang.RuntimeException: Unable to instantiate >> MessageBodyReader >> at >> org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:761) >> at >> org.jboss.resteasy.plugins.providers.RegisterBuiltin.registerProviders(RegisterBuiltin.java:70) >> at >> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:31) >> ... 27 more >> Caused by: java.lang.SecurityException: Unable to get members for >> class org.jboss.resteasy.plugins.providers.DataSourceProvider >> at >> com.google.appengine.runtime.Request.process-b6ca2b194d66ed23(Request.java) >> at java.lang.Class.getConstructors(Class.java:291) >> at >> org.jboss.resteasy.util.PickConstructor.pickSingletonConstructor(PickConstructor.java:27) >> at >> org.jboss.resteasy.spi.ResteasyProviderFactory.getProviderInstance(ResteasyProviderFactory.java:1032) >> at >> org.jboss.resteasy.spi.ResteasyProviderFactory.addMessageBodyReader(ResteasyProviderFactory.java:478) >> at >> org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:757) >> at >> org.jboss.resteasy.plugins.providers.RegisterBuiltin.registerProviders(RegisterBuiltin.java:70) >> at >> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:31) >> at >> org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:211) >> at >> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:67) >> at >> org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39) >> at >> st.voo.tick.util.GuiceResteasyFilterDispatcher.init(GuiceResteasyFilterDispatcher.java:48) >> at >> com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114) >> at >> com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98) >> at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172) >> at >> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662) >> at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) >> at >> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250) >> at >> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517) >> at >> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467) >> at >> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >> at >> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452) >> at >> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459) >> at >> com.google.tracing.TraceContext.runInContext(TraceContext.java:701) >> at >> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336) >> at >> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328) >> at >> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456) >> ... 1 more >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:616) >> ... 29 more >> Caused by: java.lang.NoClassDefFoundError: java/io/FileOutputStream >> at java.lang.Class.getDeclaredConstructors0(Native Method) >> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2406) >> ... 29 more >> Caused by: java.lang.ClassNotFoundException: java.io.FileOutputStream >> ... 29 more >> C 2012-09-12 10:56:21.889 >> Uncaught exception from servlet >> javax.servlet.UnavailableException: Initialization failed. >> at >> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:228) >> at >> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194) >> at >> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134) >> at >> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447) >> at >> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452) >> at >> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459) >> at >> com.google.tracing.TraceContext.runInContext(TraceContext.java:701) >> at >> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336) >> at >> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328) >> at >> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456) >> at >> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251) >> at java.lang.Thread.run(Thread.java:679) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
