I am going to start working on the answers to your questions. I've made some headway, but I do have one question.
In looking closer at the cache, it seems that both the global cache and the file cache both contain entries to the vm_global*_libraries. These files are very large cache entries for us. Can you tell me which cache they must be in? If they must only be in the global cache, how hard would it be to take it out of the normal file cache? Thanks Ryan -----Original Message----- From: Will Glass-Husain [mailto:[EMAIL PROTECTED] Sent: Monday, February 26, 2007 4:07 PM To: Velocity Developers List Subject: Re: Memory Footprint Help Ryan, To my knowledge we have not done significant work on memory profiling with Velocity, this effort could be a big help. Some possible questions to investigate: * What classes are the memory hogs? The VM related files? Some of the particular AST files? * Which factors seem to tie up the most memory? Includes? Macros? * Does the size of the context matter? * Does the memory usage go up over time with continued compiling (e.g. is there a growing leak) WILL On 2/26/07, Ryan Smith <[EMAIL PROTECTED]> wrote: > > I have posted the picture and attached it to bug > (http://issues.apache.org/jira/browse/VELOCITY-223). The picture was > there to simply illustrate the size of the velocity library file. > > I've done the same tests with the 1.5beta2 and the results are the same > although there is a noticeable difference in performance from a speed > perspective. From a memory footprint perspective 1.5 is about 1% larger > than 1.4. > > The pages were continuously being compiled because they were not in the > cache since the cache size needed to be small to keep the memory > footprint down. It wasn't the fault of any of the velocity code. > > At this time I am not that familiar with the Velocity code base but any > assistance in where to start looking would be greatly appreciated. > > Are there any places where velocity is hanging on to strings, tokens, or > processing instructions where we could potentially free them up? > > Are there other ways of factoring macros, files, or #parses that will > help in reducing the footprint? > > Is there potentially extra data in any of the AST classes that isn't > necessary after parsing or is potentially duplicate? > > I will do some more analysis of the memory, tokens, and directives and > post the results to the JIRA bug. > > Thanks for your help. > > Ryan Smith > > -----Original Message----- > From: Will Glass-Husain [mailto:[EMAIL PROTECTED] > Sent: Saturday, February 24, 2007 7:47 PM > To: Velocity Developers List > Subject: Re: Memory Footprint Help > > Ryan, > > Unfortunately your picture was removed by the mail list software-- > perhaps you can raise a JIRA issue on this and attach the image and > data? > > To answer your question, we would welcome assistance on this issue. > If you have time and motivation, please dig into this. We'd be happy > to help answer any questions on the code and/or offer ideas if not > more direct help. If performance increases and the regression tests > continue to pass we will almost certainly commit relevant changes. > > One note though -- there were several bug fixes related to caching, > synchronization, introspection, and other subtle issues for Velocity > 1.5. This version will be released in the next few days, but if you > work with Velocity 1.5beta2 it is almost the same thing. Have you > checked results from Velocity 1.5beta2? > > If the pages are continuously compiled that means the cache is not > working, correct? How are you determining this? > > WILL > > > > > On 2/24/07, Ryan Smith <[EMAIL PROTECTED]> wrote: > > > > > > > > > > > > We have a web site where we use velocity to generate our HTML pages. > > Recently I was asked to help troubleshoot some performance issues and > the > > root cause of our problem was that the velocity cache had grown to > well over > > 1GB in size causing the JVM to continuously GC to try to free up > memory. > > > > > > > > As a short term fix I have grabbed the 1.5 beta ResourceCacheImpl > class so > > that a maximum cache size can be set and enforced. Unfortunately when > this > > was done the performance of the site degraded significantly as the > pages > > were continuously compiled. > > > > > > > > I used the YourKit memory profiler and found the following information > about > > the individual velocity cache entries (see attached picture): > > > > Name Cache Size File Size > > > > --------------------------------------------------- > > > > VM_framework_library.vm 9,596,472 130,500 > > > > VM_buttons_library.vm 1,195,680 39,113 > > > > VM_layout_library.vm 1,683,256 54,371 > > > > admin/AdminHome.vm 32,505,168 979 > > > > poNewGrid.vm 14,399,648 753 > > > > poTemplateGrid.vm 14,369,000 774 > > > > po/details.vm 11,140,952 8,368 > > > > sub.vm 10,115,096 24,576 > > > > > > > > At this time we have made a very heavy investment in velocity as our > > presentation layer framework and love it. In order to meet our > performance > > goals, we need to keep as many of the velocity pages in cache as we > can but > > if we do that, we can only fit 2 web applications per Tomcat > deployment in a > > 32 bit environment. > > > > In searching through the JIRA issues, I found ( > > http://issues.apache.org/jira/browse/VELOCITY-450 ) and ( > > http://issues.apache.org/jira/browse/VELOCITY-223 ) that > > reference this exact issue as well as the wiki entry talking about how > to > > reduce the memory footprint. > > > > I am sending email to the developers list offering up my time to > assist with > > this issue. I was hoping that there would be someone who would be > able to > > point me in a direction to get me started and that there would > potentially > > be some "big wins" that we could take advantage of. > > > > Are there any places where velocity is hanging on to strings, tokens, > or > > processing instructions where we could potentially free them up? Are > there > > other ways of factoring macros, files, or #parses that will help in > reducing > > the footprint? Is there potentially extra data in any of the AST > classes > > that isn't necessary after parsing or is potentially duplicate? > > > > > > > > Thank you, > > > > > > > > Ryan Smith > > > > > > > > <<VelocityMemory.JPG>> > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > -- > Forio Business Simulations > > Will Glass-Husain > [EMAIL PROTECTED] > www.forio.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Forio Business Simulations Will Glass-Husain [EMAIL PROTECTED] www.forio.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
