I hadn't kept up with the payload discussion/patch, and just got
around to looking at Token.
public class Token implements Cloneable {
String termText; // the text of the term
int startOffset; // start in source text
int endOffset; // end in source text
String type = "word"; // lexical type
Payload payload;
It almost feels like we are going down the road of Field, adding more
and more to the base class instead of using some other mechanism like
inheritance.
A bigger problem, however, is that payloads will be lost by filters
that aren't payload aware, and create new Tokens. We had the same
problem with position increments being lost.
For this latter problem, I think the answer is to *not* create new
tokens, and make all the properties of Token settable.
Thoughts?
-Yonik
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
