I need to construct a Tokenizer that tokenizes at word/number boundaries, so
that "IBM Deskstar IC35L060AVER07" would result in the following tokens:
IBM
Deskstar
IC
35
L
060
AVER
07

Has anybody solved this with the StandardTokenizer?

Christian


--
To unsubscribe, e-mail:   <mailto:[EMAIL PROTECTED]>
For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>

Reply via email to