I can confirm that on:
MySQL 4.1.11-Debian_4sarge2-log
With the slightly modified test case:
============================================================================
#!/usr/bin/env python2.4
# -*- coding: utf-8 -*-
import sqlobject
connection = sqlobject.connectionForURI("mysql://*:[EMAIL PROTECTED]/*")
sqlobject.sqlhub.processConnection = connection
class MyTestClass(sqlobject.SQLObject):
class sqlmeta:
table = 'test_unicode_table'
test_column = sqlobject.UnicodeCol(length = 5)
#MyTestClass.dropTable()
#MyTestClass.createTable()
test123 = MyTestClass(test_column = u'\xe1\xe9\xed\xf3\xfa')
print test123
print test123.test_column
============================================================================
I get:
[EMAIL PROTECTED]:/www/tg$ ./test.py
Traceback (most recent call last):
File "./test.py", line 18, in ?
test123 = MyTestClass(test_column = u'\xe1\xe9\xed\xf3\xfa')
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/declarative.py",
line 92, in _wrapper
return_value = fn(self, *args, **kwargs)
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/main.py",
line 1197, in __init__
self._create(id, **kw)
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/main.py",
line 1224, in _create
self._SO_finishCreate(id)
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/main.py",
line 1251, in _SO_finishCreate
self._init(id)
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/main.py",
line 958, in _init
self._SO_selectInit(selectResults)
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/main.py",
line 1149, in _SO_selectInit
colValue = col.to_python(colValue, self._SO_validatorState)
File
"/usr/local/lib/python2.4/site-packages/SQLObject-0.7.1dev_r1588-py2.4.egg/sqlobject/col.py",
line 538, in to_python
return unicode(value, self.db_encoding)
File "/usr/lib/python2.4/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc3 in position 4:
unexpected end of data
[EMAIL PROTECTED]:/www/tg$
However if I change the column length (using MySQL) to something even I get:
[EMAIL PROTECTED]:/www/tg$ ./test.py
<MyTestClass 2L test_column=u'\xe1\xe9\xed\xf3'>
áéíó
(Truncated) or the full version at column length 10
This was with my tables running as latin1 and utf8_unicode_ci
So at least with this version, it's a MySQL problem, which isn't
surprising if I'm honest. MySQL is a great database, but this is one of
those areas where people bitch about it being shit.
I think if you want to reliably run with Unicode, at least in MySQL 4.1,
you should use TEXT columns, or be prepared not to take in data more
than 127 characters in length for varchar(255).
I got this with InnoDB and MyISAM. MySQL 5 seems to make changes to
varchar (max length is now 2^32), so this may be fixed, but the manuals
for both major versions mention nothing about character sets.
This should be filed as a bug against MySQL, but you almost certainly
won't get it fixed in 4.1 (as MySQL can only use 1 byte to store the
length of a varchar column), and I bet you stand a slim chance of
getting it fixed in 5.x. I think internal functions make assumtions
about character offsets that don't adhere to character sets. This is why
MySQL columns have a collation, but not a characer set. A collation
means "this is what I will use when I want to do case insensitive
sorting", not "this *is* what format my data is in". MySQL is a high
performance DB that makes sacrifices for speed - this is one of those
sacrifices. I suggest you use PGSQL if you really need this, or use TEXT
columns instead (drop the length from your model definition).
Best of luck
-Rob
Baruch wrote:
> Tried to change the database to utf8 with your commands and there is no
> change with regard to my problems, the test program still fails for me.
>
> I am using MyISAM, if only because that was the default. Can you try
> the test program and see if it works for you on MySQL? I'd like to know
> if it's my problem or a MySQL problem.
>
> Baruch
>
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"TurboGears" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at http://groups.google.com/group/turbogears
-~----------~----~----~----~------~----~------~--~---
