yuppie wrote:

Yves Bastide wrote:
yuppie wrote:

3.) GenericSetup is not tested with non-ASCII UTF-8 site settings. AFAIK import works, but not export. I consider this a bug.
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf4 in position 20: ordinal not in range(128)

This traceback just confirms that export does not work. Is import also broken?

Differently: it may or may not raise ... And Zope treats properties as iso8859-15 anyway.

Fresh install of Zope trunk (after a long struggle; make instance now works but make install is broken?) and CMF trunk, with

~/src/CMF$ svn diff
Index: CMFDefault/profiles/default/properties.xml
--- CMFDefault/profiles/default/properties.xml  (revision 68514)
+++ CMFDefault/profiles/default/properties.xml  (working copy)
@@ -1,6 +1,6 @@
 <?xml version="1.0"?>
- <property name="title">Portal</property>
+ <property name="title">Portàl</property>
  <property name="description"></property>
  <property name="email_from_address"
     type="string">[EMAIL PROTECTED]</property>

Fails when CMFDefault.factory.addConfiguredSite calls createSnapshot.

Here's a minimal patch for GenericSetup not to raise on the previous case (Demonstration product. Not for sale.)

[EMAIL PROTECTED]:~/src/CMF$ svn diff GenericSetup/
Index: GenericSetup/context.py
--- GenericSetup/context.py     (revision 68514)
+++ GenericSetup/context.py     (working copy)
@@ -475,7 +475,7 @@
         if isinstance( body, unicode ):
             encoding = self.getEncoding()
             if encoding is None:
-                body = body.encode()
+                body = body.encode('UTF-8')
                 body = body.encode( encoding )

Index: GenericSetup/utils.py
--- GenericSetup/utils.py       (revision 68514)
+++ GenericSetup/utils.py       (working copy)
@@ -625,6 +625,8 @@
                 if prop_map.get('type') == 'boolean':
                     prop = str(bool(prop))
+                elif isinstance(prop, str):
+                    prop = prop.decode('UTF-8')
                 elif not isinstance(prop, basestring):
                     prop = str(prop)
                 child = self._doc.createTextNode(prop)

With this applied, Portàl (u'Port\xe0l'), which becomes 'Port\xc3\xa0l', is displayed as Portà l ... Zope does input--output properties in utf-8, but stores them in iso8859. Sigh.

Thanks for setting me right. What's the usefulness of getEncoding()? As you say, exported files don't need to be other than utf-8 encoded.

I guess it just exists for historical reasons.

Might it be removed, or default'ed to utf-8? Do people already rely on it?

Well, I think I can wriggle out of most of my problems using translation. And I'll try and write UTF-8 unit tests if nobody beats me to it.

That would be great.

Hmm, by adding to an existing test suite, or creating a new one?





Zope-CMF maillist  -  Zope-CMF@lists.zope.org

See http://collector.zope.org/CMF for bug reports and feature requests

Reply via email to