CharTokenizer.incrementToken() clears *all* attributes in the entire tokenizer chain. StandardTokenizer.incrementToken() clears only the term attribute.
So... which is right? Seems like the tokenizer should be responsible? On a performance related note, CharTokenizer.clearAttribtes() could be more efficient - 2 new objects (the unmodifiable map and the iterator object) are created for every incrementToken. -Yonik http://www.lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
