On Sat, Jul 15, 2023 at 10:41:49PM +0000, Thorsten Glaser via austin-group-l at The Open Group wrote: > Hi, > > I get that the POSIX locale must be a single-byte character locale > where all 256 octets are characters. I’ve got a question about the > wide character representation. > > Assuming my POSIX locale uses ASCII as encoding, I’ve got the whole > portable character set (and then some) in the first 128 codepoints, > which have the ASCII code as both octet SBCS value and wchar_t value. > In this scenario, is it permissible to map the other 128 codepoints > “high” i.e. to wchar_t values > 0x0100?
You're not the first to ask this question. Here's a link to a proposed patch to glibc on the same topic just this month, after noting that musl has already dealt with it: https://sourceware.org/pipermail/libc-alpha/2023-July/149588.html https://sourceware.org/pipermail/libc-alpha/2023-July/150021.html https://www.openwall.com/lists/musl/2022/11/10/2 The conclusion in those links appears to be that it is compliant to have the 8-bit characters map to wchar_t codepoints that are not valid Unicode characters, but which are distinct enough to preserve all other properties needed to treat the POSIX locale as a single-byte locale with 256 "characters" and proper collation sequence without encoding errors. Whether the mapping is to the 0xdcXX or 0xdfXX range of reserved codepoints in Unicode is a matter of implementation choice; both choices exist in implementations already out there. > > I’m reading the standard as yes, but not asking already landed me > in trouble in the past so I’d rather… That's a wise course of action. And while maybe the standard could make this easier, the fact that there are already two commonly chosen ranges already in play is not going to make it easy to mandate a specific mapping. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
