On 2016-02-10 16:19, Tom Lane wrote:
I wrote:
Artur Zakirov <a.zaki...@postgrespro.ru> writes:
I think this is not a bug. It is a normal behavior. In Mac OS
sscanf()
with the %s format reads the string one character at a time. The size
of
letter 'х' is 2. And sscanf() separate it into two wrong characters.
That argument might be convincing if OSX behaved that way for all
multibyte characters, but it doesn't seem to be doing that. Why is
only 'х' affected?
I looked into the OS X sources, and found that indeed you are right:
*scanf processes the input a byte at a time, and applies isspace() to
each byte separately, even when the locale is such that that's a
clearly
insane thing to do. Since this code was derived from FreeBSD, FreeBSD
has or once had the same issue. (A look at the freebsd project on
github
says it still does, assuming that's the authoritative repo.) Not sure
about other BSDen.
I also verified that in UTF8-based locales, isspace() thinks that 0x85
and
0xA0, and no other high-bit-set values, are spaces. Not sure exactly
why
it thinks that, but that explains why 'х' fails when adjacent code
points
don't.
So apparently the coding rule we have to adopt is "don't use *scanf()
on data that might contain multibyte characters". (There might be
corner
cases where it'd work all right for conversion specifiers other than
%s,
but probably you might as well just use strtol and friends in such
cases.)
Ugh.
regards, tom lane
Definitive FreeBSD Sources:
https://svnweb.freebsd.org/base/
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: l...@lerctr.org
US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
