I was doing some tokenizer/filter analysis attempting to fix a bug I have in highlighting under 4.0. I was running the displayTokensWithFullDetails code from LIA2. I would get an exception with a bad index value of -1.
I fixed the problem by doing a reset() immediately after creating my TokenStream object. Shouldn't a new TokenStream object be essentially "reset" when created? This was with SimpleAnalyzer() (since I suspect that may be relevant).