[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 Yuri Pankov changed: What|Removed |Added CC||yur...@freebsd.org

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #25 from commit-h...@freebsd.org --- A commit references this bug: Author: yuripv Date: Thu Dec 6 11:52:08 UTC 2018 New revision: 341631 URL: https://svnweb.freebsd.org/changeset/base/341631 Log: MFC r339827: localedef:

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #24 from commit-h...@freebsd.org --- A commit references this bug: Author: yuripv Date: Thu Dec 6 11:49:53 UTC 2018 New revision: 341630 URL: https://svnweb.freebsd.org/changeset/base/341630 Log: MFC r339311, r339313:

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #23 from commit-h...@freebsd.org --- A commit references this bug: Author: yuripv Date: Wed Dec 5 17:10:07 UTC 2018 New revision: 341596 URL: https://svnweb.freebsd.org/changeset/base/341596 Log: MFC r339827: localedef:

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #22 from commit-h...@freebsd.org --- A commit references this bug: Author: yuripv Date: Sat Oct 27 23:31:43 UTC 2018 New revision: 339827 URL: https://svnweb.freebsd.org/changeset/base/339827 Log: localedef: define

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #21 from commit-h...@freebsd.org --- A commit references this bug: Author: yuripv Date: Thu Oct 11 18:30:13 UTC 2018 New revision: 339313 URL: https://svnweb.freebsd.org/changeset/base/339313 Log: Restore some of the ctype

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #20 from Yuri Pankov --- Those have names as well, though pretty simple ones :-) I did it like the following (where wc2mb is simple code snippet printing out multibyte code suitable to be grep'ped in UTF-8.cm): $ grep

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #19 from Hank Hampel --- I just realized that it might not be as easy to add the code points "U+E000 - U+F8FF : Private Use Area" as they obviously don't have any definitive names like the other symbols - and those seem to be

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #18 from Hank Hampel --- (In reply to Yuri Pankov from comment #15) Thanks for providing this deep insight into the root of the problem! My above addition to this bug report is covering a different part of the Unicode ctypes

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #17 from Conrad Meyer --- (In reply to Yuri Pankov from comment #15) Hey, that's really cool! Thanks for investigating this. I'll look at the reviews. -- You are receiving this mail because: You are the assignee for the

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #16 from Yuri Pankov --- The ctype update is our for review as well: review D17471. -- You are receiving this mail because: You are the assignee for the bug. ___

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #15 from Yuri Pankov --- Properly fixing U+FF08 and other full width characters is more involved. A bit of background: we have common ctype definitions file for all UTF-8 locales, src/share/ctypedef/en_US.UTF-8.src (all other

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 Yuri Pankov changed: What|Removed |Added CC||yur...@yuripv.net --- Comment #14

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 Conrad Meyer changed: What|Removed |Added Version|11.1-RELEASE|CURRENT --- Comment #13 from

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #12 from Conrad Meyer --- CLDR v33.1 was released 2018-06-20, maybe we can update to that. I see v34 is in alpha as well. And Unicode 11.0.0 2018-06-05: http://www.unicode.org/versions/Unicode11.0.0/ -- You are receiving

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 Hank Hampel changed: What|Removed |Added CC||free...@niamodnikufesin.de ---

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 lich...@gmail.com changed: What|Removed |Added CC||lich...@gmail.com --- Comment

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #9 from Conrad Meyer --- (In reply to jkerian+freebsdbugs from comment #8) I believe it is an automated tool, but bapt knows much more about this process than I do. -- You are receiving this mail because:

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #8 from jkerian+freebsdb...@gmail.com --- (In reply to Conrad Meyer from comment #7) Is UTF-8.src pulled from CLDR in some scripted way? Or is that a custom file? If custom, I have other codepoints that should probably be

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #7 from Conrad Meyer --- Notably, FULLWIDTH LEFT PARENTHESIS (\uFF08) is missing from every relevant copy of UTF-8.src I see. -- You are receiving this mail because: You are the assignee for the bug.

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 Conrad Meyer changed: What|Removed |Added CC||b...@freebsd.org

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #5 from Conrad Meyer --- RuneLocale data comes from _Read_RuneMagi() on a path derived from the encoding and LC_CTYPE, e.g., /usr/share/locale/en_US.UTF-8/LC_CTYPE . Where does this data come from? -- You

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #4 from Conrad Meyer --- Oh, I see, I didn't properly setlocale(). With: setlocale(LC_ALL, getenv("LANG")); and an appropriate LANG environment variable value set (e.g., en_US.UTF-8), I do see characters

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #3 from Conrad Meyer --- I find that in the default C locale, output stops after 126 ('~'). Curiously, also with LANG or LC_CTYPE set. H. -- You are receiving this mail because: You are the assignee

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #2 from Conrad Meyer --- By the way, it might be interesting to play with nextwctype(3) to enumerate the current table. wint_t wc = 0; wctype_t wct = wctype("print"); while (wc < 0x10 && wc >= 0) { wc

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 --- Comment #1 from Conrad Meyer --- iswprint(wc) is a thin shim around __istype(wc, _CTYPE_R); __istype(wc, type) is a thin shim in include/_ctype.h: return (!!__maskrune(wc, _CTYPE_R)); __maskrune() is defined

[Bug 225692] iswprint() wrong for some FULL WIDTH characters in UTF-8 locale

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692 Bug ID: 225692 Summary: iswprint() wrong for some FULL WIDTH characters in UTF-8 locale Product: Base System Version: 11.1-RELEASE Hardware: Any OS: