HEEEEEEEEEEEEELP!

We have tried everything at this point.  Shut down instances, tried to
deploy a new version, even tried old versions.  We've reported a
production issue.  Something is broken inside of GAE.  The Guice error
must be a symptom; the smoking gun seems to be:

java.lang.ClassNotFoundException: java.io.FileOutputStream

We've been down for 30 mins and getting complaints from our clients.
This looks really, really bad.  It's my nightmare scenario - an outage
in GAE that is small enough not to raise major alarm bells, but
nevertheless cripples my business.  It is not the first time this has
happened.  It is shaking my faith in GAE.

Jeff

On Wed, Sep 12, 2012 at 11:11 AM, Jeff Schnitzer <[email protected]> wrote:
> Our app has been running fine on the same version, no code deploys
> (our versions are timestamped so this is 100% certain), for two days.
> All of a sudden (10 mins ago) our app stopped running.  Every attempt
> to run a request produces this cryptic message:
>
> --------
>
> 2012-09-12 10:56:21.791
> com.google.inject.servlet.GuiceFilter setPipeline: Multiple Servlet
> injectors detected. This is a warning indicating that you have more
> than one GuiceFilter running in your web application. If this is
> deliberate, you may safely ignore this message. If this is NOT
> deliberate however, your application may not work as expected.
> D 2012-09-12 10:56:21.792
> st.voo.tick.util.cambridge.CambridgeSetup <init>: Establishing
> cambridge view resolver
> I 2012-09-12 10:56:21.792
> st.voo.tick.GuiceConfig contextInitialized: Guice initialization took 514 
> millis
> W 2012-09-12 10:56:21.888
> Failed startup of context
> com.google.apphosting.utils.jetty.RuntimeAppEngineWebAppContext@1479784{/,/base/data/home/apps/s~voost0/2012-09-10-1715.361669184733923098}
> java.lang.RuntimeException: java.lang.RuntimeException: Unable to
> instantiate MessageBodyReader
>         at 
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:35)
>         at 
> org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:211)
>         at 
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:67)
>         at 
> org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39)
>         at 
> st.voo.tick.util.GuiceResteasyFilterDispatcher.init(GuiceResteasyFilterDispatcher.java:48)
>         at 
> com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114)
>         at 
> com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98)
>         at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172)
>         at 
> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
>         at 
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at 
> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662)
>         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
>         at 
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
>         at 
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
>         at 
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
>         at 
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at 
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:219)
>         at 
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194)
>         at 
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
>         at 
> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
>         at com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
>         at 
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
>         at 
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
>         at 
> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
>         at java.lang.Thread.run(Thread.java:679)
> Caused by: java.lang.RuntimeException: Unable to instantiate MessageBodyReader
>         at 
> org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:761)
>         at 
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.registerProviders(RegisterBuiltin.java:70)
>         at 
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:31)
>         ... 27 more
> Caused by: java.lang.SecurityException: Unable to get members for
> class org.jboss.resteasy.plugins.providers.DataSourceProvider
>         at 
> com.google.appengine.runtime.Request.process-b6ca2b194d66ed23(Request.java)
>         at java.lang.Class.getConstructors(Class.java:291)
>         at 
> org.jboss.resteasy.util.PickConstructor.pickSingletonConstructor(PickConstructor.java:27)
>         at 
> org.jboss.resteasy.spi.ResteasyProviderFactory.getProviderInstance(ResteasyProviderFactory.java:1032)
>         at 
> org.jboss.resteasy.spi.ResteasyProviderFactory.addMessageBodyReader(ResteasyProviderFactory.java:478)
>         at 
> org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:757)
>         at 
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.registerProviders(RegisterBuiltin.java:70)
>         at 
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:31)
>         at 
> org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:211)
>         at 
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:67)
>         at 
> org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39)
>         at 
> st.voo.tick.util.GuiceResteasyFilterDispatcher.init(GuiceResteasyFilterDispatcher.java:48)
>         at 
> com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114)
>         at 
> com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98)
>         at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172)
>         at 
> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
>         at 
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at 
> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662)
>         at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
>         at 
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
>         at 
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
>         at 
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
>         at 
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
>         at com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
>         at 
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
>         at 
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
>         ... 1 more
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         ... 29 more
> Caused by: java.lang.NoClassDefFoundError: java/io/FileOutputStream
>         at java.lang.Class.getDeclaredConstructors0(Native Method)
>         at java.lang.Class.privateGetDeclaredConstructors(Class.java:2406)
>         ... 29 more
> Caused by: java.lang.ClassNotFoundException: java.io.FileOutputStream
>         ... 29 more
> C 2012-09-12 10:56:21.889
> Uncaught exception from servlet
> javax.servlet.UnavailableException: Initialization failed.
>         at 
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:228)
>         at 
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194)
>         at 
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
>         at 
> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
>         at com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
>         at 
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
>         at 
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
>         at 
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
>         at 
> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
>         at java.lang.Thread.run(Thread.java:679)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to 
[email protected].
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Reply via email to