On Apr 21, 2008, at 9:16 AM, Stefan Behnel wrote: > Hi, > > Stefan Behnel wrote: >> just a quick note that I started working on PEP 263 (source code >> encoding >> declaration) and PEP 3120 (UTF-8 as default source encoding). > > One problem I noticed: doc-strings are currently handled as part of > the parent > node (class/property/function) and not as normal StringNodes. So the > information if it is supposed to be a unicode string or a byte > string is lost. > Before I start hacking into this, is there any reason why doc- > strings should > not just *all* be unicode? Because this would be really simple to > implement, > and I don't think we would loose anything, except for a certain > level of > compatibility to Python 2. In Python 3, doc-strings are unicode by > default > anyway, and you would have to make them byte strings explicitly. I > think the > cases where you would not want unicode here are really rare, if any...
The only place it should make a difference is for string literals-- anything beyond the ascii charset is illegal elsewhere. Whether it makes it easier to make the entire parser operate on unicode I don't know (would there be a performance impact?) As for making docstrings unicode in Python 2, I think this is a bad idea because some programs extract and manipulate them at runtime and will be expecting Strings. I would have them be unicode throughout the compile process, but have the C output either instantiate unicode or str objects depending on PY_VERSION_HEX. - Robert _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
