Great !
Thanks a lot Otis and Cheolgoo. Youngho ----- Original Message ----- From: "Otis Gospodnetic" <[EMAIL PROTECTED]> To: <java-dev@lucene.apache.org>; "Youngho Cho" <[EMAIL PROTECTED]> Sent: Wednesday, October 05, 2005 12:55 PM Subject: Re: [jira] Created: (LUCENE-444) StandardTokenizer loses Korean characters > Try the version from SVN, I just applied Cheolgoo's patch. > > Otis > > --- Youngho Cho <[EMAIL PROTECTED]> wrote: > > > Hello, > > > > Is there any plan to add this patch into lucene core ? > > I am using CJKAnalyzer but I hope to switch to the StanadardAnalyzer. > > > > Thanks, > > > > Youngho > > > > ----- Original Message ----- > > From: "Cheolgoo Kang (JIRA)" <[EMAIL PROTECTED]> > > To: <java-dev@lucene.apache.org> > > Sent: Tuesday, October 04, 2005 11:26 PM > > Subject: [jira] Created: (LUCENE-444) StandardTokenizer loses Korean > > characters > > > > > > > StandardTokenizer loses Korean characters > > > ----------------------------------------- > > > > > > Key: LUCENE-444 > > > URL: http://issues.apache.org/jira/browse/LUCENE-444 > > > Project: Lucene - Java > > > Type: Bug > > > Components: Analysis > > > Reporter: Cheolgoo Kang > > > Priority: Minor > > > > > > > > > While using StandardAnalyzer, exp. StandardTokenizer with Korean > > text stream, StandardTokenizer ignores the Korean characters. This is > > because the definition of CJK token in StandardTokenizer.jj JavaCC > > file doesn't have enough range covering Korean syllables described in > > Unicode character map. > > > This patch adds one line of 0xAC00~0xD7AF, the Korean syllables > > range to the StandardTokenizer.jj code. > > > > > > -- > > > This message is automatically generated by JIRA. > > > - > > > If you think it was sent incorrectly contact one of the > > administrators: > > > http://issues.apache.org/jira/secure/Administrators.jspa > > > - > > > For more information on JIRA, see: > > > http://www.atlassian.com/software/jira > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > > For additional commands, e-mail: [EMAIL PROTECTED]
