-- Sematext --
http://sematext.com/ -- Lucene - Solr -
Nutch - Original Message From:
Walter Underwood [EMAIL PROTECTED] To:
solr-user@lucene.apache.org Sent: Tuesday,
November 27, 2007 2:41:38 PM Subject: Re: CJK
Analyzers for Solr Dictionaries are
surprisingly expensive
Underwood [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Wednesday, November 28, 2007 5:43:32 PM
Subject: Re: CJK Analyzers for Solr
With Ultraseek, we switched to a dictionary-based segmenter for Chinese
because the N-gram highlighting wasn't acceptable to our Chinese
customers.
I guess
://sematext.com/ -- Lucene -
Solr - Nutch - Original Message From:
Walter Underwood [EMAIL PROTECTED] To:
solr-user@lucene.apache.org Sent: Wednesday,
November 28, 2007 5:43:32 PM Subject: Re: CJK
Analyzers for Solr With Ultraseek, we switched
to a dictionary-based segmenter for Chinese
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Walter Underwood [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Tuesday, November 27, 2007 2:41:38 PM
Subject: Re: CJK Analyzers for Solr
Dictionaries are surprisingly expensive to build
Is there any specific reason why the CJK analyzers in Solr were chosen to be
n-gram based instead of it being a morphological analyzer which is kind of
implemented in Google as it considered to be more effective than the n-gram
ones?
Regards,
Eswar
On Nov 27, 2007 7:57 AM, Eswar K [EMAIL
] wrote:
Is there any specific reason why the CJK analyzers in Solr were chosen to be
n-gram based instead of it being a morphological analyzer which is kind of
implemented in Google as it considered to be more effective than the n-gram
ones?
Regards,
Eswar
On Nov 27, 2007 7:57 AM, Eswar
On 27-Nov-07, at 8:54 AM, Eswar K wrote:
Is there any specific reason why the CJK analyzers in Solr were
chosen to be
n-gram based instead of it being a morphological analyzer which is
kind of
implemented in Google as it considered to be more effective than
the n-gram
ones?
The CJK
:
Is there any specific reason why the CJK analyzers in Solr were chosen to be
n-gram based instead of it being a morphological analyzer which is kind of
implemented in Google as it considered to be more effective than the n-gram
ones?
Regards,
Eswar
On Nov 27, 2007 7:57 AM, Eswar K [EMAIL
of these languages.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Walter Underwood [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Tuesday, November 27, 2007 2:41:38 PM
Subject: Re: CJK Analyzers for Solr
Dictionaries are surprisingly expensive
PM
Subject: Re: CJK Analyzers for Solr
I don't think NGram is good method for Chinese.
CJKAnalyzer of Lucene is 2-Gram.
Eswar K:
if it is chinese analyzer,,i recommend
hylanda（www.hylanda.com）,,,it is
the best chinese analyzer and it not free.
if u wanna free chinese analyzer, maybe u can
: Monday, November 26, 2007 9:27:15 PM
Subject: Re: CJK Analyzers for Solr
thanks james...
How much time does it take to index 18m docs?
- Eswar
On Nov 27, 2007 7:43 AM, James liu [EMAIL PROTECTED] wrote:
i not use HYLANDA analyzer.
i use je-analyzer and indexing at least 18m docs.
i
Subject: Re: CJK Analyzers for Solr
John,
There were two parts to my question,
1) n-gram vs morphological analyzer - This was based on what I read at
a few
places which rate morphological analysis higher than n-gram. An example
being (
http://www.basistech.com/knowledge-center/products/N-Gram-vs
Hi,
Does Solr come with Language analyzers for CJK? If not, can you please
direct me to some good CJK analyzers?
Regards,
Eswar
: Does Solr come with Language analyzers for CJK? If not, can you please
: direct me to some good CJK analyzers?
Lucene has a CJKTokenizer and CJKAnalyzer in the contrib/analyzers jar.
they can be used in Solr. both have been included in Solr for a while
now, so you can specify CJKAnalyzer
Hoss,
Thanks a lot. Will look into it.
Regards,
Eswar
On Nov 26, 2007 11:55 PM, Chris Hostetter [EMAIL PROTECTED] wrote:
: Does Solr come with Language analyzers for CJK? If not, can you please
: direct me to some good CJK analyzers?
Lucene has a CJKTokenizer and CJKAnalyzer in the
into 'C1 C2'. I hope someone who speaks Mandarin or
Cantonese understands what this should do.
Lance
-Original Message-
From: Eswar K [mailto:[EMAIL PROTECTED]
Sent: Monday, November 26, 2007 10:28 AM
To: solr-user@lucene.apache.org
Subject: Re: CJK Analyzers for Solr
Hoss,
Thanks a lot
: I notice this is in the future tense. Is the CJKTokenizer available yet?
CJKTokenizer and CJKAnalyzer are both available in Solr 1.2, but no
TokenizerFactory was provided for CJKTokenizer in 1.2, so it wasn't
possible to use out of the box without writing a 3 line java plugin.
that 3 line
://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Eswar K [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Monday, November 26, 2007 8:30:52 AM
Subject: CJK Analyzers for Solr
Hi,
Does Solr come with Language analyzers for CJK? If not, can you please
direct me to some
Analyzers for Solr
Hoss,
Thanks a lot. Will look into it.
Regards,
Eswar
On Nov 26, 2007 11:55 PM, Chris Hostetter [EMAIL PROTECTED]
wrote:
: Does Solr come with Language analyzers for CJK? If not, can you
please
: direct me to some good CJK analyzers?
Lucene has a CJKTokenizer
looked at Chinese
and CJK Analzyers, so I could be off.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Eswar K [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Monday, November 26, 2007 8:30:52 AM
Subject: CJK Analyzers for Solr
Hi
understands what this should do.
Lance
-Original Message-
From: Eswar K [mailto:[EMAIL PROTECTED]
Sent: Monday, November 26, 2007 10:28 AM
To: solr-user@lucene.apache.org
Subject: Re: CJK Analyzers for Solr
Hoss,
Thanks a lot. Will look into it.
Regards,
Eswar
Subject: CJK Analyzers for Solr
Hi,
Does Solr come with Language analyzers for CJK? If not, can you please
direct me to some good CJK analyzers?
Regards,
Eswar
--
regards
jl
.
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Eswar K [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Monday, November 26, 2007 8:30:52 AM
Subject: CJK Analyzers for Solr
Hi,
Does Solr come
: Eswar K [EMAIL PROTECTED]
To: solr-user@lucene.apache.org
Sent: Monday, November 26, 2007 8:30:52 AM
Subject: CJK Analyzers for Solr
Hi,
Does Solr come with Language analyzers for CJK? If not, can you
please
direct me to some good CJK analyzers?
Regards
24 matches
Mail list logo
