* Florian Pflug:

> On Nov24, 2011, at 10:54 , Florian Weimer wrote:
>>> Or is it not only about being able to *store* NULs in a text field?
>> No, the entire core should be NUL-transparent.
> That's unlikely to happen.

Yes, with the type input/output functions tied to NUL-terminated
strings, that seems indeed unlikely to happen.

> A more realistic approach would be to solve this only for UTF-8
> encoded strings by encoding the NUL character not as a single 0 byte,
> but as sequence of non-0 bytes.

0xFF cannot occur in valid UTF-8, so that's one possibility.

> Java, for example, seems to use it to serialize Strings (which may contain
> NUL characters) to UTF-8.

Only internally in the VM.  UTF-8 produced by the I/O encoder/decoders
produces and consumes NUL bytes.

> Should you try to add a new encoding which supports that, you might also
> want to allow CESU-8-style encoding of UTF-16 surrogate pairs. This means
> that code points representable by UTF-16 surrogate pairs may be encoded by
> separately encoding the two surrogate characters in UTF-8.

I'm not sure if this is a good idea.  The motivation behind CESU-8 is
that it sorts byte-encoded strings in the same order as UTF-16, which is
a completely separate concern.

Florian Weimer                <fwei...@bfk.de>
BFK edv-consulting GmbH       http://www.bfk.de/
Kriegsstra├če 100              tel: +49-721-96201-1
D-76133 Karlsruhe             fax: +49-721-96201-99

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to