Thanks for the reply Peter. I think that this is definitely something that needs some attention if we are going to consider GAE as a truly viable solution for even modest apps. The app I am working with is pretty smallish Mach-II app with only 7 listeners, 3 filters, 3 plugins and about a dozen beans defined in CS.
Since my initial post, I unhooked autowiring in exchange for explicit getX() methods in all framework components that set local variables from the service factory beans as needed, and went through and set all CS beans to lazy-init="true". Even with that I am still getting the HDEE. I even stripped out the caching property and removed all framework caching trying to make it as simple as possible. Still no love... I will be interested to read Alan's thoughts on the precompiling notion that you threw out. That could potentially be a big boost if it was feasible. As to your notion of defer directives, I really like that idea. Let's take that offline since this list probably isn't the place for that. Holler at me when you get time. ~Dave Shuck /************************************** blog: www.daveshuck.com tweets: www.twitter.com/dshuck cf peeps: www.dfwcfug.org **************************************/ On Thu, Aug 19, 2010 at 9:29 PM, Peter J. Farrell <[email protected]> wrote: > Yeah, you are definitely hitting an HDEE (that's what a lot of Java guys > developing on GAE call them). The problem is that Mach-II and ColdSpring > are caching a lot of things on the initial application load for > performance. Now, this is totally acceptable in the normal CFML world and > actually has become the dominating architecture for most of the frameworks > these days. Sadly with GAE it becomes a problems you've seen first hand. > > There are a few suggestions: > > * Defer as much as you can from the initial load. I've gone as far as to > not use the beans attribute in listeners, writing getters that check to see > if a service is available and if not ask ColdSpring for it and then cache > the service. This defers a lot of ColdSpring stuff unless your model is > pretty interconnected. This way getBean() is not called until a listener > really needs it. Benefits vary with your application style / model. > > * Don't instantiate any objects in the onSessionStart() phase. Use a > session facade and have getters that auto create when needed before > returning a reference to that object. Your mileage will vary depending on > what you do. In a non-GAE application, we did this because Google was > hammering our site and we were creating a "cart" object and other stuff > during the onSessionStart(). Since GoogleBot is not stateful, this caused > onSessionStart() to run on each request and therefore we did see big spikes > in memory if Google hit us hard. One time we did start seeing out of memory > errors after Google can and indexed 10,000 pages in under 5 minutes. > Sessions were set to expire in 20 minutes -- you do the math. So we also > look at the cgi.user_agent and set low timeouts for bots (like 30 seconds > seems to work well). > > * Alan -- is there a way we can have OpenBD precompile the "stack" into a > file that can be read instead of JIT compiling it? It would be nice if you > could do this for GAE kind of like javac. I'm un-aware of the exact way > OpenBD does this, but I do know it's *not* like how Adobe CF does it to > .class files. > > * Lastly, Dave -- I'd be willing to work with you to work in some optional > "defer" directives for Mach-II so app start up times can be spread over a > few requests. For example, it's not necessary to load in *all* > event-handlers from the get go. So if you are willing to help with the code > -- I can help you with that. Plus, it's a great way to contribute back to > the project I just don't have the time to spend on "defer" options right > now. > > HTH, > .Peter > > Dave Shuck said the following on 08/19/2010 08:26 PM: > > I have an app that I have been working with for some time locally. It is a > Mach-II/ColdSpring application, so the initial bootstrap takes a few seconds > when working locally, but nothing terribly extreme. Recently I gave it a > deployment and all seemed to work well. However, after hitting a milestone > today I did another deployment and can no longer get a 200 response. I get > the big "Error: Server Error" on the page when I try to load it. In the log > files I see the stack trace below. Based on the fact there is exactly 30 > secs between the start and exception time, I am pretty certain that it is a > fixed timeout threshold that I am passing. > > Does anyone have any thoughts or suggestions on getting around it? > > Stack trace: > ____________________ > > Error for / > com.google.apphosting.runtime.HardDeadlineExceededError: This request > (82ac02bd351743d5) started at 2010/08/20 00:16:37.405 UTC and was still > executing at 2010/08/20 00:17:07.294 UTC. > > at java.lang.Object.wait(Native Method) > at java.lang.Object.wait(Object.java:443) > at java.util.concurrent.TimeUnit.timedWait(Unknown Source) > at com.google.apphosting.runtime.AsyncFuture.get(AsyncFuture.java:65) > > at > com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:356) > at > com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.get(ApiProxyImpl.java:225) > at > com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:137) > > at > com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:45) > at > com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:110) > at > com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:108) > > at java.security.AccessController.doPrivileged(Native Method) > at > com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:108) > at > com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:45) > > at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) > at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:48) > at > com.google.appengine.api.memcache.MemcacheServiceImpl.makeSyncCall(MemcacheServiceImpl.java:179) > > at > com.google.appengine.api.memcache.MemcacheServiceImpl.get(MemcacheServiceImpl.java:277) > at > com.newatlanta.appengine.datastore.CachingDatastoreService.get(CachingDatastoreService.java:181) > at > com.newatlanta.appengine.datastore.CachingDatastoreService.get(CachingDatastoreService.java:175) > > at > com.newatlanta.appengine.vfs.provider.GaeFileObject.getMetaData(GaeFileObject.java:160) > at > com.newatlanta.appengine.vfs.provider.GaeFileObject.doAttach(GaeFileObject.java:153) > at > org.apache.commons.vfs.provider.AbstractFileObject.attach(AbstractFileObject.java:1469) > > at > org.apache.commons.vfs.provider.AbstractFileObject.getType(AbstractFileObject.java:460) > at > org.apache.commons.vfs.provider.AbstractFileObject.exists(AbstractFileObject.java:448) > at > com.newatlanta.appengine.vfs.provider.GaeFileSystemManager.resolveFile(GaeFileSystemManager.java:173) > > at > org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:648) > at > org.apache.commons.vfs.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:604) > at > com.newatlanta.appengine.vfs.provider.GaeVFS.resolveFile(GaeVFS.java:171) > > at com.nary.io.FileUtils.getLastModified(Unknown Source) > at com.naryx.tagfusion.cfm.file.cachedFile.setRealPath(Unknown Source) > at com.naryx.tagfusion.cfm.file.cfmlFileCache._getCfmlFile2(Unknown > Source) > at com.naryx.tagfusion.cfm.file.cfmlFileCache._getCfmlFile(Unknown > Source) > > at com.naryx.tagfusion.cfm.file.cfmlFileCache.getCfmlFile(Unknown > Source) > at com.naryx.tagfusion.cfm.engine.cfSession.getFile(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfINCLUDE.loadTemplate(Unknown Source) > > at > com.naryx.tagfusion.cfm.engine.ComponentFactory.loadRawComponent(Unknown > Source) > at com.naryx.tagfusion.cfm.engine.cfComponentData.<init>(Unknown Source) > at com.naryx.tagfusion.expression.function.createObject.execute(Unknown > Source) > > at > com.naryx.tagfusion.expression.compile.CFFunctionExpression.Eval(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.CFAssignmentExpression.Eval(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.run(Unknown Source) > > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.cfm.tag.cfSET.render(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfIF.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTRY.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.coreRender(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.renderToString(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.realRun(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > at com.naryx.tagfusion.cfm.script.userDefinedFunction.execute(Unknown > Source) > > at > com.naryx.tagfusion.expression.compile.CFFunctionExpression.Eval(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.CFAssignmentExpression.Eval(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.run(Unknown Source) > > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.cfm.tag.cfSET.render(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfIF.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfIF.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.coreRender(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.renderToString(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.realRun(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > at com.naryx.tagfusion.cfm.script.userDefinedFunction.execute(Unknown > Source) > > at > com.naryx.tagfusion.cfm.engine.cfComponentData.invokeComponentFunction(Unknown > Source) > at > com.naryx.tagfusion.cfm.engine.cfComponentData.invokeComponentFunction(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.cfFullVarExpression.Eval(Unknown > Source) > > at > com.naryx.tagfusion.expression.compile.cfFullVarExpression.evalNatural(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.cfFullVarExpression.Eval(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.run(Unknown Source) > > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.cfm.tag.cfSET.render(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfTag.coreRender(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.renderToString(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.realRun(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > at com.naryx.tagfusion.cfm.script.userDefinedFunction.execute(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.CFFunctionExpression.Eval(Unknown > Source) > > at com.naryx.tagfusion.expression.compile.runTime.run(Unknown Source) > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.runExpression(Unknown > Source) > > at com.naryx.tagfusion.cfm.tag.cfSET.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfIF.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfIF.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.coreRender(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfTag.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfLOCK.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfIF.render(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfTag.coreRender(Unknown Source) > > at com.naryx.tagfusion.cfm.tag.cfTag.renderToString(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.realRun(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > at com.naryx.tagfusion.cfm.tag.cfFUNCTION.run(Unknown Source) > > at com.naryx.tagfusion.cfm.script.userDefinedFunction.execute(Unknown > Source) > at > com.naryx.tagfusion.cfm.engine.cfComponentData.invokeComponentFunction(Unknown > Source) > at > com.naryx.tagfusion.cfm.engine.cfComponentData.invokeComponentFunction(Unknown > Source) > > at > com.naryx.tagfusion.expression.compile.cfFullVarExpression.Eval(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.cfFullVarExpression.evalNatural(Unknown > Source) > at > com.naryx.tagfusion.expression.compile.cfFullVarExpression.Eval(Unknown > Source) > > at > com.naryx.tagfusion.expression.compile.CFAssignmentExpression.Eval(Unknown > Source) > at com.naryx.tagfusion.expression.compile.runTime.run(Unknown Source) > at com.naryx.tagfusion.expression.compile.runTime.runExpressio > > > > > ~Dave Shuck > /************************************** > blog: www.daveshuck.com > tweets: www.twitter.com/dshuck > cf peeps: www.dfwcfug.org > **************************************/ > -- > Open BlueDragon Public Mailing List > http://www.openbluedragon.org/ http://twitter.com/OpenBlueDragon > official manual: http://www.openbluedragon.org/manual/ > Ready2Run CFML http://www.openbluedragon.org/openbdjam/ > > mailing list - http://groups.google.com/group/openbd?hl=en > > > -- > Open BlueDragon Public Mailing List > http://www.openbluedragon.org/ http://twitter.com/OpenBlueDragon > official manual: http://www.openbluedragon.org/manual/ > Ready2Run CFML http://www.openbluedragon.org/openbdjam/ > > mailing list - http://groups.google.com/group/openbd?hl=en > -- Open BlueDragon Public Mailing List http://www.openbluedragon.org/ http://twitter.com/OpenBlueDragon official manual: http://www.openbluedragon.org/manual/ Ready2Run CFML http://www.openbluedragon.org/openbdjam/ mailing list - http://groups.google.com/group/openbd?hl=en
