Christian Boos wrote:
Jonas Borgström wrote:
Hi,
I've been experimenting with using real unicode strings instead of
normal utf-8 encoded strings on the sandbox/unicode branch.
...
The branch currently passes all unit-tests and seems to work well
but I would appreciate some additional testing and feedback on the
idea in general.
I've tested it a bit.
I had some issues initially, but it It appeared that I had a mix
of ISO-8859-1 and utf8 encoded values in my enum table
(tests values for when I worked on #1821, IIRC).
After cleaning that up (remove the "très" encoded with "tr<8A>s"),
it seems to work.
I also tested "the Milestone with national language characters" (#1239),
and that still works.
So for me (pysqlite-2.1.x - Windows), it works.
PS: Something I'm wondering... if everything is Unicode,
shouldn't the regexps (using \w, \W, \b, \B, \d, \D, \s and \S)
have the UNICODE flag set?
Probably, but we meed to be a bit careful about that since regexps with
the UNICODE flag set will work slightly different (\w for example will
match _all_ alphanumeric characters and '_' and not just [a-zA-Z0-9_]).
Regexps without the UNICODE-flag will work the same way with
unicode strings as with unicode strings.
Also, in general, a quick overview of how it works would be
useful: what strings are unicode (i.e. those coming from the db
layer, those coming from the req.args, those coming from the
config...), which aren't (those coming from the svn repos?).
Thanks!
The basic idea is to use unicode strings everywhere internally.
Strings are converted into unicode strings as soon as possible
and only converted back to utf-8 when sent to the browser.
For example:
* The database driver returns unicode strings
* The Request class stores the query string as unicode strings
in req.args.
* clearsilver.py converts unicode strings to utf-8 when they are
stored in the hdf object.
* The vc-layer should return things such as filenames as unicode
strings (not done yet).
Cheers,
Jonas
_______________________________________________
Trac-dev mailing list
[email protected]
http://lists.edgewall.com/mailman/listinfo/trac-dev
