Hi,
I'm having trouble converting an application that uses psycopg2
directly. The Postgres database is encoded in UTF-8. As an example, it
has a column called title with the value 'Wilhelm R\xc3\xb6pke', i.e.,
'Wilhelm Röpke'. With psycopg2, the connection and retrieval is done
more or less as follows:
# -*- coding: utf-8 -*-
from psycopg2 import connect
from psycopg2.extensions import register_type, UNICODE
register_type(UNICODE)
db = connect('host=...')
curs = db.cursor()
curs.execute('select title from tablex')
row = curs.fetchone()
title = row[0]
print(row)
print(title)
The output from this test program is:
(u'Wilhelm R\xf6pke',)
Wilhelm Röpke
In other words, psycopg2 hands the app a Unicode string, which is ideal
because the templating engine (Mako) apparently expects Unicode, even
when told the input_encoding is utf-8. If the register_type() is
removed, a plain string is delivered but that doesn't help Mako.
The comparable SQLAlchemy program is as follows:
# -*- coding: utf-8 -*-
from sqlalchemy.engine import create_engine
engine = create_engine('postgres://...')
result = engine.execute('select title from tablex')
row = result.fetchone()
title = unicode(row.title, encoding='utf-8')
print(row)
print(title)
The output of the SA test program is:
('Wilhelm R\xc3\xb6pke',)
Wilhelm Röpke
In other words, SA hands the app a plain string instead of Unicode, and
if the unicode() conversion is removed, the print(title) (and Mako)
complain with:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 9:
ordinal not in range(128)
I've tried various combinations of convert_unicode, assert_encoding and
encoding parameters to the create_engine() call and I have not been able
to get Unicode strings from SA, which I find somewhat surprising. Is it
possible, or do I have to filter every string retrieved from the
database through encoding(value, encoding='utf-8)?
For reference, this is on Debian, using Python 2.5.4 and SA 0.5.5.
Joe
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
