On Aug 13, 2009, at 10:29 AM, Yonik Seeley wrote:
I'm liking the new attribute based analysis (in conjunction with reusability), but I'm running into some questions... Is it valid for tokenizers or token filters add new attributes after their constructor (after they have processed some tokens)?
I don't know if it explicitly prohibits it, but I believe the docs discourage it for performance reasons.
Should restoreState() be able to add attributes (it currently throws an exception)? If not, does that mean that it's not supported/advised to use state across different TokenStreams?
Not sure
We've previously seen that the native java clone() can be much slower than implementing it ourselves in Java. Should we have our own clone() method on Attribute? Or just implement clone() ourselves and require that subclasses override if needed? This is inner-loop per-token stuff, and a single captureState() will invoke many clone operations (6 attributes make up the legacy Token object).
Sounds reasonable, but I haven't benchmarked. --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
