[Zope-dev] ZCatalog : UTF-8 Chinese

[Sin Hang Kin]

HI,

I have a project which need search with Chinese. I think I can make Zcatalog
to search Chinese in utf8. So I change Voodoo Kludge Splitter.py to convert
the input string to unicode (assume it is utf8) and make a version of split
of my own (see the attached splitter.py). I borrow (stolen) from Interscript
the utf8 encoding conversion scheme. I separate the chinese with space by
hand hoping the Zcatalog will work.

After changing these, I have a catalog which looks good : I can see from the
volucably the chinese were actually there (except with some which have html
encode like  inside the utf8.

I generate the search interface, and test it. However, the search of the
index terms return nothings. I search most entries found in the vocubalury
but none works, those work will return many unwanted results also.

What is causing this failure? What I can do to go further?


Rgs,

Kent Sin
-
kentsin.weblogs.com
kentsin.imeme.net

 Splitter.py

Re: [Zope-dev] ZCatalog : UTF-8 Chinese

[Zope mailing lists]

On Mon, 25 Sep 2000, Sin Hang Kin wrote:
 I generate the search interface, and test it. However, the search of the
 index terms return nothings. I search most entries found in the vocubalury
 but none works, those work will return many unwanted results also.
 
 What is causing this failure? What I can do to go further?

It is possible you are having an issue with the way the splitter
is used on the search term input side.  Several of us have found
bugs in that area.  We've fixed the ones we've found, but there
may be more wry grin.

Run zope in debug mode ("the debugger is your friend" howto), and
watch what UnTextIndex does with the search terms.  (Hint: instead
of trying to set breakpoints per the howto, just uncomment the
appropriate calls to the debugger in the UnTextIndex or Lexicon
source file...)

--RDM


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )