Veit, This wasn't just a stop-gap, but it may be incomplete. I only had a few minutes to review this code, but from what I could see there may be even a simpler solution to what you suggested. You could create a new virtual function for Analyzer, which will be called from its destructor; in this function you'll use reinterpret_cast to delete the tokenStreams properly. Or you can do this directly in the constructor for the overrided Analyzer class.
Or perhaps using a specialized ThreadLocal object which allows to lazily load a deletor. This deletor, in turn, needs to be able to work with void pointers and to properly cast them before deletion, of course. As far as I could see the only problematic deletion is in the destructor; in the rest of the places where deletion occurs there a proper casting is being done. Just a few ideas from the top of my head, to try and avoid adding more classes, and keep this implementation generalized. IIRC, I had to use void pointers because of very different implementation of token streams in code (for example in StopAnalyzer where there is a SavedStreams class instead of just TokenStream). HTH, Itamar. -----Original Message----- From: Veit Jahns [mailto:nuncupa...@googlemail.com] Sent: Friday, March 19, 2010 2:41 PM To: clucene-developers@lists.sourceforge.net Subject: Re: [CLucene-dev] PerFieldAnalyzerWrapper memory leak Itamar, I have a question to one of your commits done in the course of this discussion. The commit is the commit 364c21....6fb93 [1]. You replaced a TokenStream pointer with a void pointer in the thread local storage of Analyzer::Internal. Now everything can be saved as a previous stream, but this lead to two problems: (1) This changed lead to a delete of void pointers, which is undefined. (2) We wrote an own Analyzer and for this analyzer we defined a class own class SavedStream like you did it for the StandardAnalyzer. Our SavedStream class has some fields. Because of the void pointer these fields aren't delete correctly, because the destructor isn't called (because of the void pointer). Is this commit just a quick fix and will be changed back to TokenStream later? If not, I propose to define an class, which has to be used as a super class for all class, saved as previous token stream. This what we have done to avoid these problems. Kind regards, Veit [1] http://clucene.git.sourceforge.net/git/gitweb.cgi?p=clucene/clucene;a=commit ;h=364c21b6c3f54fbb90df223621b660197366fb93 ---------------------------------------------------------------------------- -- Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers
