Just sharing my experience and asking another question.
I tried the analyzer suggested here:
http://www.ruby-forum.com/topic/72086#101764. It works fine if you
specify the search field you want to use (anyway, it seems that's how
it's suppose to work).
# CODE
analyzer =
Ferret::Analysis::PerFieldAnalyzer.new(Ferret::Analysis::StandardAnalyzer.new)
analyzer["chinese"] = Ferret::Analysis::RegExpAnalyzer.new(/./, false)
index = Index::Index.new(:path => '/var/index', :analyzer => analyzer,
:default_field => "*")
...
index.search_each("chinese: #{val}") do |doc, score| #val is a chinese char
puts "#{doc} - #{score}"
end
# END CODE
This works OK. However, if you try searching like this:
# CODE
index.search_each(val) do |doc, score| #val is a chinese char
puts "#{doc} - #{score}"
end
# END CODE
I get in my lighttpd error log:
/var/www/localhost/htdocs/cgi-bin/search_chinese.ruby:19:in
`search_each': : Error occured at <analysis.c>:701 (StandardError)
Error: exception 2 not handled: Error decoding input string. Check
that you have the locale set correctly
from /var/www/localhost/htdocs/cgi-bin/search_chinese.ruby:19
Which MAKES SENSE, since the docs I created before are created like this:
doc = { "author" => "englishchars", "title" => "more regular chars",
"chinese" => "新闻"}
index << doc
and I think search_each is going through all the fields (since I
explicitly said it should when I issued :default_field => "*" up
there), finding english chars, and trying to match them against the
chinese ones I supplied as a search query.
So alright, I can use the suggested analyzer. But my question is: is
there a way to use an analyzer that would work with both character
types (english, and asian) simply by not returning matches them as
opposed to giving me an error?
Thanks a ton for any help.
On 7/19/06, Julio Cesar Ody <[EMAIL PROTECTED]> wrote:
> Thanks, and sorry. I checked the documentation for Index::Index and
> found it right after I asked the question. My bad.
>
> I'm getting segfauls when trying to initialize an index using a
> different analyzer other than the default one (but it works
> otherwise). But as I can see in this thread
>
> http://www.ruby-forum.com/topic/71620
>
> It ain't stable yet for 64 bit. So I'll wait.
>
> Thanks again.
>
>
> On 7/19/06, David Balmain <[EMAIL PROTECTED]> wrote:
> > On 7/19/06, Julio Cesar Ody <[EMAIL PROTECTED]> wrote:
> > > Yep, it did. Thanks tons!
> > >
> > > But I'm not getting any results now. I take this is because of the
> > > default analyzer being used, right?
> > >
> > > How can I use a whitespace analyzer in my query? (or something that
> > > could work effectively with asian languages).
> > >
> > > For my needs, I suppose the whitespace one could do...
> >
> > index = Index::Index.new(:path => '/var/index', :default_field => "*",
> > :analyzer => Ferret::Analysis::WhiteSpaceAnalzyer.new)
> >
> > Although you should probably use the same analyzer I gave you for indexing;
> >
> > http://www.ruby-forum.com/topic/72086#101764
> >
> > Cheers,
> > Dave
> > _______________________________________________
> > Ferret-talk mailing list
> > [email protected]
> > http://rubyforge.org/mailman/listinfo/ferret-talk
> >
>
>
> --
> Julio C. Ody
> http://rootshell.be/~julioody
>
--
Julio C. Ody
http://rootshell.be/~julioody
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
