Sorry for all the updates, hopefully someone will find it useful though!! Latency is through the roof! - what is causing this?! can I fix it?...
Total number of instancesAverage QPS*Average Latency*Average Memory7 total (3 Resident)0.00233671.0 ms68.9 MBytes*Instances *[image: help]<https://developers.google.com/appengine/docs/adminconsole/instances> QPS*Latency*RequestsErrorsAgeMemoryLogsAvailabilityShutdown0.0000.0 ms30 0:04:2067.0 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117c8b437629ba0681cf452275258982&filter_type=regex&date_type=now&limit=20&view=Search>[image: Resident Icon]Resident0.0000.0 ms1200:04:2067.8 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117c3a9330bdcf5e1074ecba490b5935&filter_type=regex&date_type=now&limit=20&view=Search>[image: Resident Icon]Resident0.01733671.0 ms100:01:2852.0 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117c157a0c8723723a486140193eb8&filter_type=regex&date_type=now&limit=20&view=Search>[image: Resident Icon]Resident0.0000.0 ms200:02:2355.3 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117c9bb59efbad64506f2b0498baa5&filter_type=regex&date_type=now&limit=20&view=Search>[image: Dynamic Icon]Dynamic0.0000.0 ms1400:04:1068.5 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117cf91ad57f164b7c2e5d22806846b1&filter_type=regex&date_type=now&limit=20&view=Search>[image: Dynamic Icon]Dynamic0.0000.0 ms200:02:0167.4 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117cc3d4aff4f08dc57b18c11ecc0f34bf&filter_type=regex&date_type=now&limit=20&view=Search>[image: Dynamic Icon]Dynamic0.0000.0 ms2900:07:19104.1 MBytesView Logs<https://appengine.google.com/logs?app_id=idebanet&version_id=6-20.362344675424379244&severity_level_override=1&severity_level=3&filter=instance%3D00c61b117cb72247c66e582eded2d6293f502db2&filter_type=regex&date_type=now&limit=20&view=Search>[image: Dynamic Icon]Dynamic On Friday, 8 February 2013 13:27:21 UTC, John V Denley wrote: > > I just adjusted my instance setting so the number of idle instances was > 3-24 and the pending latency was 0-500ms > > and suddenly everything seemed to kick into action... > > I changed my settings back to auto-auto and auto-auto its not working > again.... > > more tests... > > On Friday, 8 February 2013 13:09:32 UTC, John V Denley wrote: >> >> Also Just noticed this which REALLY doesn't seem right!! >> >> Total number of instancesAverage QPS*Average Latency*Average Memory0 >> totalUnknownUnknown msUnknown MBytes >> On Friday, 8 February 2013 01:04:45 UTC, Marcin wrote: >>> >>> Hi, >>> >>> I am experiencing extremely long response times for a couple of hours >>> now. I am using GAE/J. >>> >>> I can provide app id via email. >>> >>> My app has billing enabled and is using SSL VIP. >>> >>> This affects both requests with API calls and static resources like >>> images: >>> >>> Static image: >>> >>> ms=2655 cpu_ms=0 >>> >>> Controller with API calls displaying dashboard: >>> >>> ms=8430 cpu_ms=131 >>> >>> >>> For a list of activities 131 cpu_ms seems like a good result, but why >>> does the whole request last for 8.5secs? >>> >>> Other odd things that I have noticed: >>> - My resident / reserved instance is missing - I have only dynamic ones, >>> there was always a permanent instance with green shield icon. >>> - After disabling and reenabling the application I always get >>> /_ah/warmup 500 87610ms 0kb - 90 seconds to start an instance? I'm not >>> using Spring, it usually took about 7 secs to spin a new one. >>> >>> com.google.apphosting.api.DeadlineExceededException: This request >>> (18f5b04d255c672c) started at 2013/02/08 00:28:56.487 UTC and was still >>> executing at 2013/02/08 00:29:56.229 UTC. >>> at java.util.zip.ZipFile.read(Native Method) >>> at java.util.zip.ZipFile.access$1200(ZipFile.java:57) >>> at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:476) >>> at java.util.zip.ZipFile$1.fill(ZipFile.java:259) >>> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) >>> at java.io.DataInputStream.readFully(DataInputStream.java:195) >>> at java.util.jar.JarFile.hasClassPathAttribute(JarFile.java:482) >>> at >>> java.util.jar.JavaUtilJarAccessImpl.jarFileHasClassPathAttribute(JavaUtilJarAccessImpl.java:33) >>> at sun.misc.URLClassPath$JarLoader.getClassPath(URLClassPath.java:1176) >>> at sun.misc.URLClassPath.getLoader(URLClassPath.java:374) >>> at sun.misc.URLClassPath.findResource(URLClassPath.java:201) >>> at java.net.URLClassLoader$2.run(URLClassLoader.java:379) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findResource(URLClassLoader.java:376) >>> at >>> com.google.apphosting.runtime.security.UserClassLoader.findResource(UserClassLoader.java:723) >>> at java.lang.ClassLoader.getResource(ClassLoader.java:977) >>> at org.mortbay.resource.Resource.newSystemResource(Resource.java:203) >>> at >>> org.mortbay.jetty.webapp.WebXmlConfiguration.configureDefaults(WebXmlConfiguration.java:159) >>> at >>> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1230) >>> at >>> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517) >>> at >>> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467) >>> at >>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) >>> at >>> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:219) >>> at >>> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194) >>> at >>> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134) >>> at >>> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447) >>> at >>> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:454) >>> at >>> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:461) >>> at com.google.tracing.TraceContext.runInContext(TraceContext.java:703) >>> at >>> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:338) >>> at >>> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:330) >>> at >>> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:458) >>> at >>> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251) >>> at java.lang.Thread.run(Thread.java:679) >>> >>> Uncaught exception from servlet >>> javax.servlet.UnavailableException: Initialization failed. >>> at >>> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:228) >>> at >>> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194) >>> at >>> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134) >>> at >>> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447) >>> at >>> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:454) >>> at >>> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:461) >>> at com.google.tracing.TraceContext.runInContext(TraceContext.java:703) >>> at >>> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:338) >>> at >>> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:330) >>> at >>> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:458) >>> at >>> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251) >>> at java.lang.Thread.run(Thread.java:679) >>> >>> >>> We are starting to consider moving out of App Engine. This issue and the >>> thing that our users tend to hit cold instances every n requests makes it >>> not suitable for our need >>> >>> >>> Marcin >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
