I already removed the unmodifiable iterator, so one new instance is removed (see the JIRA issue). But you are right, the CharTokenizer should only clear the TermAttribute, as it is only using this attribute.
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik > Seeley > Sent: Monday, August 10, 2009 6:01 PM > To: java-dev@lucene.apache.org > Subject: who clears attributes? > > CharTokenizer.incrementToken() clears *all* attributes in the entire > tokenizer chain. > StandardTokenizer.incrementToken() clears only the term attribute. > > So... which is right? Seems like the tokenizer should be responsible? > > On a performance related note, CharTokenizer.clearAttribtes() could be > more efficient - 2 new objects (the unmodifiable map and the iterator > object) are created for every incrementToken. > > -Yonik > http://www.lucidimagination.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
