On Mar 23, 2011, at 11:24 PM, ccooper wrote:
> I am trying to store and retrieve unicode text in an sqlite database
> and repeatedly run into UnicodeEncodeErrors.
>
> To start I am specifying the encoding in create_engine even though I
> believe UTF-8 is default:
> create_engine('sqlite:///db/rnews.db', encoding='utf-8',
> echo=True)
The pysqlite adapter is all unicode, always. The "encoding" parameter has no
effect since SQLAlchemy (assuming a modern version) doesn't do any encoding
with this backend. Python unicodes are sent directly and received back.
>
> All my column types are Unicode. All my text is Unicode. I have also
> tried adding the encoding to my unicode strings:
> u'Some string'.encode('utf-8')
That converts the Python unicode object to an encoded bytestring. You don't
want to be doing that, you want to keep all strings as u'xyz'. The pysqlite
backend itself (as of recent versions) will reject the value if it contains any
non-ascii characters.
>
> When adding a new object to the session I get:
>
> UnicodeEncodeError: 'ascii' codec can't encode character
> u'\u201c' in position 48: ordinal not in range(128)
If you're using a modern version of SQLAlchemy, the SQLA will not be doing any
encoding with the SQLite dialect. It passes Python unicode objects straight
in. If it was doing any encoding, it would be using the 'utf-8' codec, not
'ascii'
So this issue might be on your end. Calling str() on the value, or trying to
print it, invokes the Python default 'ascii' codec for example. Its not
possible to know without a stack trace and code which reproduces the issue.
>
> How can I change the fact that an ascii codec is still being used?
>
> Any help would be much appreciated,
>
> Chad
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
