my simple tests with some japanese text shows this to be a major improvement. thank you, john mcnally
On Mon, 2003-12-22 at 14:12, [EMAIL PROTECTED] wrote: > cutting 2003/12/22 14:12:24 > > Modified: . CHANGES.txt > src/java/org/apache/lucene/analysis/standard > StandardTokenizer.java StandardTokenizer.jj > StandardTokenizerConstants.java > StandardTokenizerTokenManager.java > Log: > Fix StandardTokenizer's handling of CJK characters. > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
