On Thu, 2008-07-31 at 12:34 +1000, David Crossley wrote: > Thorsten Scherler wrote: > > > > lately I saw a couple of OutOfMemoryError reports around forrest. Brain > > and Gavin reported them around the dispatcher plugin. Actually today I > > ran to an OutOfMemoryError within our solr plugin. > > > > The code is not used with the dispatcher but directly. I am using > > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.output.solr/src/java/org/apache/forrest/solr/client/SolrSearchGenerator.java?view=log > > > > Doing a "top" on the box I see that the memory never got cleaned up. It > > keeps on growing till it reaches the max. You can see it quite fast in > > limiting the max memory to a real small amount (e.g. 128M) the memory > > usage is keep on growing till forrest fails with Heaps size error. > > > > My first impression would be that is caused by the above generator but > > seeing all the recent threads about similar situations it makes me > > wander whether that may be a general problem. > > I don't use the solr plugin. > > I do see the issue with one of my complex sites that > uses dispatcher plugins (not your other whiteboard work). > Also with the forrest/site-author site which is skins-based. > > The memory usage does not increase quickly, as for yours, > but yes it does increase. Only manual tests so not yet > reached a crash. > > However, this seems to be consistent with the bugs that > we have already identified. > https://issues.apache.org/jira/browse/FOR-591 > "Memory Leak with XMLFileModule" > and there was some other known dispatcher issues. >
Thank you very much David to point me to this issue. I will now investigate whether this can be the cause (but it really sounds like it) and see how we can fix this. > Aside: Oh it would be blissful to get our internal > Cocoon upgraded to at least use Cocoon-2.1 head. > Ralph Goers has fixed that problem which causes FOR-591. I first need to find a quick workaround for the project based on forrest in my work (need to deliver till Monday a new version of this work project), but then I need to fix this issue once and for all in forrest and will try to update our cocoon. I personally prefer to 2.2 but I fear the pass-through feature (missing in 2.2. AFAIK and extensively used in forrest) will be a showstopper but 2.1 should work just fine. > > > Does anybody see an obvious leak in the above generator? Does anybody > > has an idea how we can trigger the leak? > > I have not looked - perhaps someone else can. > > Does the solr plugin make extensive use of XML File Module? > You might reach crisis more quickly. > > Also see Tim's comments in FOR-591 regarding perhaps > "compounded this issue with similar simple caching in the locationmap". > Maybe that part has already been addressed. > There was recently some other locationmap changes. > > Dunno, sorry, just some ideas. Thank you very much for your thoughts. As usual most valuable. salu2 > > -David -- Thorsten Scherler thorsten.at.apache.org Open Source Java consulting, training and solutions
