yuppie wrote:

With this applied, Portàl (u'Port\xe0l'), which becomes 'Port\xc3\xa0l', is displayed as Portà l ... Zope does input--output properties in utf-8, but stores them in iso8859. Sigh.

I was afraid this would be complex :(
That's why I only use ASCII in configuration data.

Can you find out why it stores them in iso8859? Is this hardcoded or configurable somewhere?

I know of at least one point, ZPublisher.Converters (field2string). However by the time a supposedly unicode string (say title:UTF-8:string) comes here, it's already iso8859. Will look deeper ...

[About getEncoding()]

Don't know if third party products use it. I guess if CPS doesn't nobody does.

It does, though I suspect incorrectly. Florent?

AFAICS it could be deprecated at least for export contexts.

Well, I think I can wriggle out of most of my problems using translation. And I'll try and write UTF-8 unit tests if nobody beats me to it.

That would be great.

Hmm, by adding to an existing test suite, or creating a new one?

In general the unit tests have a module / class structure similar to the tested units. E.g. tests for utils.PropertyManagerHelpers should be added to test_utils.PropertyManagerHelpersTests. But sometimes there are reasons to add a new test suite, e.g. if you need a different setup.

I did modify test_utils's properties suite (see attached patch), but it passes with GenericSetup current version :-)



Index: GenericSetup/tests/test_utils.py
--- GenericSetup/tests/test_utils.py	(revision 68520)
+++ GenericSetup/tests/test_utils.py	(working copy)
@@ -24,7 +24,7 @@
 from Products.GenericSetup.testing import DummySetupEnviron
 <?xml version="1.0"?>
  <property name="foo_boolean" type="boolean">False</property>
@@ -34,6 +34,7 @@
  <property name="foo_lines" type="lines"/>
  <property name="foo_long" type="long">0</property>
  <property name="foo_string" type="string"></property>
+ <property name="foo_unicode_string" type="string"></property>
  <property name="foo_text" type="text"></property>
  <property name="foo_tokens" type="tokens"/>
  <property name="foo_selection" select_variable="foobarbaz"
@@ -45,9 +46,9 @@
  <property name="foo_float_nodel">0.0</property>
  <property name="foo_boolean_nodel">False</property>
 <?xml version="1.0"?>
  <property name="foo_boolean" type="boolean">True</property>
@@ -60,6 +61,7 @@
  <property name="foo_long" type="long">1</property>
  <property name="foo_string" type="string">Foo String</property>
+ <property name="foo_unicode_string" type="string">\u0080</property>
  <property name="foo_text" type="text">Foo
  <property name="foo_tokens" type="tokens">
@@ -78,9 +80,9 @@
  <property name="foo_float_nodel">3.1415</property>
  <property name="foo_boolean_nodel">True</property>
 <?xml version="1.0"?>
  <property name="foo_boolean">True</property>
@@ -93,6 +95,7 @@
  <property name="foo_long">1</property>
  <property name="foo_string">Foo String</property>
+ <property name="foo_unicode_string" type="string">\u0080</property>
  <property name="foo_text">Foo
  <property name="foo_tokens">
@@ -109,7 +112,7 @@
  <property name="foo_float_nodel">3.1415</property>
  <property name="foo_boolean_nodel">True</property>
 <?xml version="1.0"?>
@@ -240,6 +243,7 @@
         obj.manage_addProperty('foo_lines', '', 'lines')
         obj.manage_addProperty('foo_long', '0', 'long')
         obj.manage_addProperty('foo_string', '', 'string')
+        obj.manage_addProperty('foo_unicode_string', '', 'string')
         obj.manage_addProperty('foo_text', '', 'text')
         obj.manage_addProperty('foo_tokens', '', 'tokens')
         obj.manage_addProperty('foo_selection', 'foobarbaz', 'selection')
@@ -264,6 +268,7 @@
         obj._updateProperty('foo_lines', 'Foo\nLines')
         obj._updateProperty('foo_long', '1')
         obj._updateProperty('foo_string', 'Foo String')
+        obj._updateProperty('foo_unicode_string', u'\u0080'.encode('utf-8'))
         obj._updateProperty('foo_text', 'Foo\nText')
         obj._updateProperty( 'foo_tokens', ('Foo', 'Tokens') )
         obj._updateProperty('foo_selection', 'Foo')
@@ -303,6 +308,7 @@
         self.assertEqual(getattr(obj, 'foo_lines', None), None)
         self.assertEqual(getattr(obj, 'foo_long', None), None)
         self.assertEqual(getattr(obj, 'foo_string', None), None)
+        self.assertEqual(getattr(obj, 'foo_unicode_string', None), None)
         self.assertEqual(getattr(obj, 'foo_text', None), None)
         self.assertEqual(getattr(obj, 'foo_tokens', None), None)
         self.assertEqual(getattr(obj, 'foo_selection', None), None)
Zope-CMF maillist  -  Zope-CMF@lists.zope.org

See http://collector.zope.org/CMF for bug reports and feature requests

Reply via email to