SOLR-4872 and LUCENE-2145 (or, how to clean up a Tokenizer)

2013-06-12 Thread Benson Margulies
Could I have some help on the combination of these two? Right now, it appears that I'm stuck with a finalizer to chase after native resources in a Tokenizer. Am I missing something?

Re: SOLR-4872 and LUCENE-2145 (or, how to clean up a Tokenizer)

2013-06-12 Thread Lance Norskog
In 4.x and trunk is a close() method on Tokenizers and Filters. In currently released up to 4.3, there is instead a reset(stream) method which is how it resets a TokenizerFilter for a following document in the same upload. In both cases I had to track the first time the tokens are consumed,