it depends greatly on if you are using SQLAlchemy 0.6 or not. The short answer is that you should be using the Unicode type for columns that store unicode data, or you can set this globally using create_engine(...., convert_unicode=True), which encodes strings into utf-8 before being sent.
However, if you're using SQLAlchemy 0.6, we don't actually do that conversion with psycopg2 anymore and instead use it's native unicode support, which only works if you've configured the client encoding of your PG database to "utf-8", i.e. in postgresql.conf. If you're on the default of "SQL_ASCII", and its not an option to change that (you really should change it if possible), then SQLA's usual "encode to utf-8" logic can be re-established on 0.6 by also adding "use_native_unicode=False". Docs at http://www.sqlalchemy.org/docs/reference/dialects/postgresql.html#unicode (as of just now) On May 11, 2010, at 12:04 PM, Oliver Beattie wrote: > I have some code that is taking values out of one database (a MySQL > latin-1 database) and inserting them into another (postgres, utf-8) > database, and I can't for the life of me work out what is going wrong > here. The traceback looks slightly fishy, so I can only assume > something is going horribly wrong in C-land: http://dpaste.org/afO4/ > > So, from this, I guess the problem must be in psycopg2? The only thing > is that it actually works just fine on one of my machines, but on the > machine I need to run it on, it fails miserably. > > I'm going slightly mad here trying to find a solution to this, but so > far nothing I've tried (short of an .encode('ascii', 'ignore') does > the trick. > > Is this something anyone's seen before, or am I doing something > monumentally stupid? > > Any help in tracking this down would be really appreciated… I'm > seriously going insane trying to fix this. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
