J�r�me Charron wrote:
I was thinking about it for a while: Multi-Lingual support in Nutch.
After looking at Nutch code, I write a proposal on the Wiki (
http://wiki.apache.org/nutch/MultiLingualSupport).
+1
I think this is a good design.
This was discussed recently on the lucene-dev mailing list.
http://www.mail-archive.com/[email protected]/msg01197.html
False positives are sometimes a concern, where different words in
different languages analyze to the same string. For example, a French
analyzer might normalize "th�" to "the" which would then match many
English documents. I think such issues are best resolved by restricting
search results to the desired language. (In the case of "th�", even
this is not enough, since many French documents include bits of English.
So, while converting "th�" to "the" might aid French recall a bit, it
really kills precision. Thus "th�" is probably best analyzed as "th�",
or perhaps as both "th�" and "the".)
Doug
-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games. How far can you shotput
a projector? How fast can you ride your desk chair down the office luge track?
If you want to score the big prize, get to know the little guy.
Play to win an NEC 61" plasma display: http://www.necitguy.com/?r=20
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
