On Fri, Dec 14, 2007 at 01:20:52AM +0100, Justin Fletcher wrote:
> I am trying to migrate some data into the ZODB.  I am storing this data
> in a string, and later presenting the data in a ZPT similar to this:
> <span tal:replace="context/content" />
> This works fine for text that does not have any characters outside of
> the ascii range, but for text that does I receive this error:
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
> 3283: ordinal not in range(128)

You are mixing non-ASCII str objects with Unicode strings somewhere.

> The characters are typically things like what the HTML code &eacute;
> would generate, but it is a fairly significant amount of text so I am
> unsure specifically what characters are causing the problem.

Any that are non-ASCII.

> My understanding is that Zope3 strings are Unicode so I don't understand
> why the ascii range is a restriction.  Shouldn't these characters be
> stored and presented transparently?

There's no magic.  "Zope 3 strings are Unicode" is a convention, and
Zope makes it easy to follow by decoding all HTTP request strings into
Unicode objects.  If you're migrating existing non-Unicode data into
ZODB with a simple Python script, you'll have to take care of converting
your binary strings to Unicode yourself.

> Am I misunderstanding something or doing something wrong?

I think so.  If you could show us how you're migrating "some data" into
the ZODB, we could give you more advice.

Marius Gedminas
lg_PC.gigacharset (lg = little green men language, PC = proxima centauri)
        -- Markus Kuhn provides an example of a locale

Attachment: signature.asc
Description: Digital signature

Zope3-users mailing list

Reply via email to