I'm trying to work through the logic of reading ahead until I've seen marker for the end of a sentence, then applying some analysis to all of the tokens of the sentence, and then changing some attributes of each token to reflect the results.
The queue of tokens for a position is just a State, so there isn't an API there to set any values. So do I need to subclass Position for myself, store the additional information in there, and set the attributes as each token comes by on the output side? I would be grateful for a bit more explanation of afterPosition versus incrementToken; some of the mock classes call peek from afterPosition, and I expected to see peek called in incrementToken based on the javadoc.
