Sorry Steven
that change is in WhitespaceTokenizer not WhiteSpaceAnalyzer but in Analyzer
I had to call the tokenizer
On 5/24/07, Mohammad Norouzi <[EMAIL PROTECTED]> wrote:
Hi Steven
Thank you so much for your thorough comments about Analyzer
I write that class a couple of months ago, now I take a look at my
customized Analyzer
the only change I've made as follows:
the original class has this method:
protected boolean isTokenChar(char c) {
return !Character.isWhitespace(c);
}
And my class override that method as this:
protected boolean isTokenChar(char c) {
return !((int)c==32);
}
I think the Character.isWhitespace consider the unicodes as space :))
so everything will mess up.
what do you think?
--
Regards,
Mohammad
--------------------------
see my blog: http://brainable.blogspot.com/
--
Regards,
Mohammad
--------------------------
see my blog: http://brainable.blogspot.com/
