I think there shouldn't be 2 components that deal with this. If a user wants to use its functions then he should either deal with the performance or let it be. I think a good documentation is the right way here.
On 9/22/06, Gavin Vess <[EMAIL PROTECTED]> wrote:
Hi André,
André, are you still planning a Zend_Utf8 class with different
functionality than the UTF8 helper class for Zend_Locale*?
I know the i18n/locale team is working on a flyweight UTF8 helper class
for private use by Zend_Locale* related classes. This helper class will
include only the function absolutely needed in order for the locale
classes to work. For those new to the ZF, a few weeks ago, after a long
discussion on this list, we decided to not attempt to duplicate the UTF8
functionality coming in PHP6, and not attempt to make the entire ZF work
with UTF8 strings (note: mbstring extension helps with UTF8).
http://framework.zend.com/wiki/display/ZFDEV/i18n+Locale+Team
Cheers,
Gavin
Alexander Veremyev wrote:
> Yes. That's a problem.
>
> Hm... Two solutions may be here.
> 1. Wait until Zend_UTF8 may help with this.
> 2. Move translation (current work around) to other place to keep
> stored fields unchanged.
>
> What is better???
>
> With best regards,
> Alexander Veremyev.
>
>
> Christer Edvartsen wrote:
>
>> Converting to ASCII//TRANSLIT is done in the Zend_Search_Lucene_Field
>> constructor as far as I can see, so what I have to do is to convert
>> the search string in the same fashion and then convert the search
>> hits before I display them. This is where I start getting problems.
>>
>> If I do a var_dump(iconv('ISO-8859-1', 'ASCII//TRANSLIT',
>> 'æ,ø,å,Æ,Ø,Å')); I get string(13) "ae,o,a,AE,O,A"
>>
>> The problem is that I can not seem to be able to translate the search
>> hits back to ISO-8859-1 to get back my precious norwegian characters.
>> Any tips?
>>
>> Alexander Veremyev wrote:
>>
>>> Hi Christer,
>>>
>>> UTF-8 can be completely handled with 'ascii//translit' conversion.
>>> Take a look at
>>> http://framework.zend.com/manual/en/zend.search.charset.html
>>>
>>> iconv('ISO-8859-1', 'ASCII//TRANSLIT', $docText) converts umlauts to
>>> two-symbol representation.
>>> Ex. ü -> ue, æ -> ae, å -> aa, ö -> oe.
>>> (I am not sure on ø)
>>>
>>> Thus 'für' will be translated to 'fuer'.
>>> If the same translation is applied to search query, you will get
>>> search result as expected.
>>>
>>>
>>> I don't like this solution, but it works.
>>>
>>> Zend_Search_Lucene completely supports utf-8 internally (for index
>>> files), but the problem is in the document tokenizer and query parser.
>>>
>>> We need utf-8 versions of ctype_alphe()/ctype_digit() functions
>>> (mbstring extension can't help with this).
>>>
>>>
>>> As I see Zend_UTF8 can help with this (http://www.utf8-chartable.de/
>>> can give this information). And, I hope, will do :)
>>> (There are no performance issues for Zend_Search_Lucene)
>>>
>>>
>>> With best regards,
>>> Alexander Veremyev.
>>>
>>>
>>>
>>>
>>> Christer Edvartsen wrote:
>>>
>>>> I guess the main problem is that utf8 is not fully implemented
>>>> yet... Maybe you know some more about when this will happen? Could
>>>> you also give me some tips about how to handle the characters I am
>>>> having problems with? (æ, ø and å in ISO-8859-1)
>>>>
>>>> Alexander Veremyev wrote:
>>>>
>>>>> Hi Facundo,
>>>>>
>>>>> I think that we have not a lot of discussions, because everything
>>>>> is almost clear there.
>>>>> It's just a port. We only should move functionality from Java
>>>>> Lucene with enough accurate and understand, when we should stop :)
>>>>>
>>>>> But if you have any thoughts, you are welcome!
>>>>>
>>>>>
>>>>> I heard, that it's used in some projects now, but don't know
>>>>> details. That would be great to find it out.
>>>>>
>>>>> As I see Zend_Search_Lucene is stable enough and I work on
>>>>> automatic index optimization just now.
>>>>> It will allow to be independent from Java tools (ex. Luke tool)
>>>>> and also will close memory usage issue
>>>>> (http://framework.zend.com/issues/browse/ZF-88).
>>>>>
>>>>>
>>>>> With best regards,
>>>>> Alexander Veremyev.
>>>>>
>>>>>
>>>>> Facundo Pagani wrote:
>>>>>
>>>>>> Hi there ppl!
>>>>>> What about Zend_Search_Lucene? I dont see any1 talking about it
>>>>>> ... Has any1 doing some serious/production work/project with it?
>>>>>> Can u share ur xperiences?
>>>>>> Be in touch!
>>>>>> Thanks in advance.
>>>>>>
>>>>>> --
>>>>>> ---------------------------------------------------
>>>>>> Facundo M. Pagani
>>>>>> Ingeniería | Sectorial de Informática
>>>>>> Ministerio de Hacienda y Finanzas
>>>>>> Santa Fe - (C.P.3000 ) - Argentina
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
>
--
best regards,
André Hoffmann
Germany
