"Yonik Seeley" <[EMAIL PROTECTED]> wrote: > On Nov 19, 2007 11:38 AM, Michael McCandless <[EMAIL PROTECTED]> > wrote: > > > If we opt to treat payload like termBuffer and copy the bytes, then we > > > need no offset member. > > > > I think I'd lean towards leaving payload "by reference"? > > It seems difficult to allow the payload setter to reuse their byte[], > unless we break back compatibility with other token filters. Do you > have a solution in mind?
I think I must be missing something. The payload API is marked as experimental? And, since we are changing it anyway (& removing or deprecating the current experimental one), how would using the "by reference" approach break backwards compatibility? Maybe you mean that each Token must be fully independent because there are plenty of filters that hold onto each Token long after next() is called again, and then serve them up again later, eg Grant's new LUCENE-1058? But this is why we have the Token next() method (which returns a "stable" Token that the callee can't change later) vs the Token next(Token) method which allows the callee to re-use the passed in Token on the next call. So eg in TokenStream.next() base implementation we would have to make a new copy of the payload byte[] and set it in the copied token. This way when a caller calls next() they get a full private copy, even of the by-reference payload. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
