On Thu, Aug 13, 2009 at 4:32 PM, Michael Busch<busch...@gmail.com> wrote: > On 8/13/09 7:29 AM, Yonik Seeley wrote: >> >> I'm liking the new attribute based analysis (in conjunction with >> reusability), but I'm running into some questions... >> >> Is it valid for tokenizers or token filters add new attributes after >> their constructor (after they have processed some tokens)? >> > At the moment we're saying in the javadocs of TokenStream that all > Attributes should be > added up front.
Hmmm, OK... in which case, token producers using restoreState() would not have to call clearAttributes() first. > We could change these semantics. I had some thoughts about > it in the original > JIRA issue (LUCENE-1422). Apologies if I'm rehashing anything - it's hard to keep up with some of those monster (high volume) issues. > So back to your question if we should allow restoreState() to add attributes > and use a state across different AttributeSources: the complication is that > we can only > allow that if the different AttributeSource were filled using the same > AttributeFactory, otherwise > different AtttributeImpls could be in the sources and the copying wouldn't > work anymore. Hmmm, so perhaps just an assertion that the factories are equal... and documentation saying that moving state from one stream to the other requires identical factories? Anyway, I don't currently have a use case for this... I was just wondering. Another thing I was wondering about was the opacity of State - one can't inspect or change the attributes w/o restoring it first. Undesirable limitation, or feature allowing more flexible state implementations? -Yonik http://www.lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
