I have just returned from vacation and am looking at this issue in more detail. Dieter's explanation seems logical, especially considering the following traceback when I try and move a folder containing these specific CMF object types.

Looking at the objects, they have plain text ids (I assume by looking only), but unicode titles (again by just looking in ZMI). The latter were probably copied and pasted from somehwere into a Cmf product's editting screen, which I presume is the problem here.

So following Dieter's explanation, is it possible to find and identify which objects have the non-unicode/non-ascii ids/titles using some python? I'm assuming that I could then edit the offending object's id/title in the ZMI to "re-unicode" it.

Changing the default character encoding of my production server for this one application just isn't an option unfortunately.

Thanks, nick

Traceback (innermost last):

    * Module ZPublisher.Publish, line 101, in publish
    * Module ZPublisher.mapply, line 88, in mapply
    * Module ZPublisher.Publish, line 39, in call_object
    * Module OFS.CopySupport, line 231, in manage_renameObjects
    * Module OFS.CopySupport, line 260, in manage_renameObject
    * Module OFS.ObjectManager, line 276, in _setObject
    * Module Products.CMFCore.CMFCatalogAware, line 148, in manage_afterAdd
    * Module Products.CMFCore.CMFCatalogAware, line 177, in __recurse
    * Module Products.CMFCore.CMFCatalogAware, line 148, in manage_afterAdd
    * Module Products.CMFCore.CMFCatalogAware, line 177, in __recurse
    * Module Products.CMFCore.CMFCatalogAware, line 148, in manage_afterAdd
    * Module Products.CMFCore.CMFCatalogAware, line 177, in __recurse
    * Module Products.CMFCore.CMFCatalogAware, line 147, in manage_afterAdd
    * Module Products.CMFCore.CMFCatalogAware, line 42, in indexObject
    * Module Products.CMFPlone.CatalogTool, line 56, in indexObject
    * Module Products.CMFCore.CatalogTool, line 235, in catalog_object
    * Module Products.ZCatalog.ZCatalog, line 528, in catalog_object
    * Module Products.ZCatalog.Catalog, line 381, in catalogObject
    * Module Products.ZCTextIndex.ZCTextIndex, line 163, in index_object
    * Module Products.ZCTextIndex.ZCTextIndex, line 176, in _index_object
    * Module Products.ZCTextIndex.OkapiIndex, line 58, in index_doc
    * Module Products.ZCTextIndex.BaseIndex, line 108, in index_doc
    * Module Products.ZCTextIndex.Lexicon, line 69, in sourceToWordIds
    * Module Products.ZCTextIndex.Lexicon, line 135, in _getWordIdCreate

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 5: ordinal not in range(128)



Dieter Maurer wrote:
Nick Bower wrote at 2004-10-8 16:41 +0200:

...
Module Products.ZCTextIndex.Lexicon, line 69, in sourceToWordIds
Module Products.ZCTextIndex.Lexicon, line 135, in _getWordIdCreate

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 5: orginal not in range(128).


In your lexicon operation a unicode and a non-unicode string is
put together (this can happen internally during BTree traversal).

Whenever such a thing happens, Python tries to convert the
non unicode to unicode -- using its default encoding.
This fails as the non unicode string contains bytes not convertable
this this encoding.

In a later message you reported that setting Python's default
encoding to "utf-8" gave you an unexpected end exception.
This means that your non unicode string is not utf-8 encoded.


You should use as default encoding the encoding that is used for your non unicode strings.

If you do not know it, use an encoding that can map any 8 bit byte.
Windows has a few of them (called "cpXXX" (for CodePage);
I do not know the correct XXX).


_______________________________________________
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Reply via email to