I'm liking the new attribute based analysis (in conjunction with reusability), but I'm running into some questions...
Is it valid for tokenizers or token filters add new attributes after their constructor (after they have processed some tokens)? Should restoreState() be able to add attributes (it currently throws an exception)? If not, does that mean that it's not supported/advised to use state across different TokenStreams? We've previously seen that the native java clone() can be much slower than implementing it ourselves in Java. Should we have our own clone() method on Attribute? Or just implement clone() ourselves and require that subclasses override if needed? This is inner-loop per-token stuff, and a single captureState() will invoke many clone operations (6 attributes make up the legacy Token object). -Yonik http://www.lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
