There are a lot of issues that show up here. 1) Using the JSP means that the entire article text (among other things) is being cached into RAM. 2) Some of the pages are "popular" enough that they are getting moved into the permanent area of memory, and thus avoiding the routine, "cheap" GCs. 3) The question whether Tomcat is somehow "holding on" to the JSP references to prevent them from being GC'd at all (I don't think this is the case, but you never know).
As for solutions, there are several, none of which are particularly elegant I don't think. First, of course, "Buy more RAM", also pronounced "Let's make our Full GC's take even LONGER!". Another is to simply reboot Tomcat during a "quiet" time. At least this way, the down time is predictable rather than having it "Crash" at an unscheduled time. If the restart time is fast enough, and the site quiet enough, this may effectively solve the issue. You mentioned that the system is running several days reliably now, so, perhaps a nightly restart is not a horrible concession. Another is to mirror the site behind some kind of load balancer so that restarting the site does not actually affect users, and then go to solution one. Oh boy, more hardware to configure. You can "rengineer" the site. If your JSPs are reasonably regular to the point where you can write fairly simple filters to extract the actual content without resorting to crafting a JSP parser, then that is a good first step. So, going on the crass assumption that getting the article text out of your generated JSPs is not an onerous task, then the next step, to me, would be to stick a servlet Filter in front of every request. This Magic Filter (a notable anti-pattern, but...) is configured so that you will not have to actually update and rewrite all of the zillion URLs scattered throughout the site. So, www.yoursite.com/area/document.jsp in the "old" site is the same in the "new" site. The Magic Filter determines if the JSP file is a "real" jsp (no doubt you have some JSPs that do something other than dump static articles), versus one of your generated documents. If it is one of your generated documents, then the Magic Filter rips it apart, caches it perhaps (the ripped text), and then serves it up in a new shiny generic way as should have been done originally. If not, you let the Filter drop through and pass on so the container can handle the JSP requiest and go its merry way. Ideally, this can be done with very little change to the site, and certainly no change to the production CMS, it's ignorant of the change. Your web.xml "simply" has a new Filter element, and voila! I am not suggesting that this is easy or painless, but if your generated JSPs are able to be easily torn apart, then I think it's viable and practical. Perhaps your CMS JSP template can be recoded to add some comments (like <!-- ARTICLE START TEXT--> This is my article <!-- ARTICLE END TEXT -->) that makes it easy to find articles. Perhaps you can then REGENERATE all of your older content to use the new template. You'd like to think that your CMS will allow this (kind of the whole point of a CMS at some level, isn't it?). Or, if you can not regenerate your older content, have the Magic Filter "detect" old vs. new and simply "execute" the old ones (as noted above), and filter the new ones. If your site is mostly newer content, then the old stuff will drift away and be less of a problem over time. Once up and running, your Magic Filter can cache and keep track of the status of various JSPs it encounters so as not to have to rework them each time. I would vote for the "restart nightly" technique and look into ways of tweaking the CMS system to spit out something you can use, but that's me. The other problem with this technique is that it is doing processing at run time which could best be done statically. For example, the fact that your CMS is generating JSPs, perhaps you can place a step in the production step that performs the parsing for you, that way it is done at production time versus request time. It depends on how easily you can modify your work flow from the CMS into the site proper. Regards, Will Hartung ([EMAIL PROTECTED]) ----- Original Message ----- From: "Luc Foisy" <[EMAIL PROTECTED]> To: "Tomcat Users List" <[EMAIL PROTECTED]> Sent: Tuesday, December 31, 2002 7:44 AM Subject: RE: reducing tomcat & jasper memory footprint Since your jsp's are generated, they should all have the same formatting. Write some code to rewrite your own non jsp pages that you can then later insert into a single jsp. Sure its some work, but in the long run it will save you some aggravation. If tomcat can rewrite your jsp on the fly, so can you. -- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
