On 9/18/07, Guido van Rossum <[EMAIL PROTECTED]> wrote: > On 9/18/07, Jim Jewett <[EMAIL PROTECTED]> wrote: > > On 9/18/07, Stephen J. Turnbull <[EMAIL PROTECTED]> wrote:
> > > There's no UTF-8 in Python's internal string encoding. > > (At least as of a few days ago) > > In Python 3 there is; strings are unicode. A PyUnicodeObject object > > has two encodings that you can grab from a pointer (which means > > they have to be there; you don't have time to generate them like > > you would with a function pointer). > Incorrect. The pointer can be NULL. I had missed that comment, but I do see it now; thank you. > The API for getting the UTF-8 encoding is a function Thank you. But given that defenc is now always UTF-8, won't exposing it in the public typedef then just be an attractive nuisance? > (moreover a function whose name starts with _Py). That I still don't see. http://svn.python.org/view/python/branches/py3k/Include/unicodeobject.h?rev=57656&view=markup PyAPI_FUNC(PyObject*) PyUnicode_AsUTF8String( PyObject *unicode /* Unicode object */ ); PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF8( const Py_UNICODE *data, /* Unicode char buffer */ Py_ssize_t length, /* number of Py_UNICODE chars to encode */ const char *errors /* error handling */ ); Later, the same file shows me: /* --- Unicode Type ------------------------------------------------------- */ typedef struct { PyObject_HEAD Py_ssize_t length; /* Length of raw Unicode data in buffer */ Py_UNICODE *str; /* Raw Unicode buffer */ long hash; /* Hash value; -1 if not set */ int state; /* != 0 if interned. In this case the two * references from the dictionary to this object * are *not* counted in ob_refcnt. */ PyObject *defenc; /* (Default) Encoded version as Python string, or NULL; this is used for implementing the buffer protocol */ } PyUnicodeObject; I would be happier with: typedef struct { PyObject_VAR_HEAD /* Length in code points, not chars */ } PyUnicodeObject; And, in unicodeobject.c (*not* in a public header) typedef struct { PyUnicodeObject ob_unicodehead; Py_UNICODE *str; /* Raw Unicode buffer */ long hash; /* Hash value; -1 if not set */ int state; /* != 0 if interned. In this case the two * references from the dictionary to this object * are *not* counted in ob_refcnt. */ PyObject *defenc; /* (Default) Encoded version as Python string, or NULL; this is used for implementing the buffer protocol */ } _PyDefaultUnicodeObject; As this would allow 3rd parties to create implementations specialized for (and saving space on) smaller alphabets, without breaking C extensions that stick to the public header files. (Moving hash or even state to the public header might be OK too, but they seemed to get ignored for subclasses anyhow.) -jJ _______________________________________________ Python-3000 mailing list Python-3000@python.org http://mail.python.org/mailman/listinfo/python-3000 Unsubscribe: http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
