hvdijk wrote: > That being said, I understand that there might be other use cases you are > trying to solve,
I've been working on [llvm-strings's --encoding option](https://github.com/llvm/llvm-project/pull/221794), where with `--encoding=S` I would use `std::iswprint()` to determine whether a character is printable, with `--encoding=utf8` I would use `llvm::sys::unicode::isPrintable()`, and it did not make sense that in an UTF-8 locale, `--encoding=S` and `--encoding=utf8` would return hugely different results. > And our definition of isPrintable is consistent with that of ICU, Python, Go. That can't be right when one of those three is different from the other two, but we didn't match any of them. We defined isPrintable as: L|M|N|P|S|Zs. ICU defines isPrintable as: not unassigned, not Cc|Cf|Cn|Co|Cs. (per https://github.com/unicode-org/icu/blob/560e6db559a4a836a016b762051eee00d56e3d76/icu4j/main/core/src/main/java/com/ibm/icu/lang/UCharacter.java#L6966) Python defines isprintable as: L|M|N|P|S|space. (per https://docs.python.org/3/builtins/stdtypes.html#str.isprintable) Go defines IsPrint as: L|M|N|P|S|space. (per https://pkg.go.dev/unicode#IsPrint) We currently differ from Python and Go in that we consider e.g. U+00A0 printable. We currently differ from ICU in that we consider e.g. U+2028 non-printable. > We are very much intentionally deviating from JF tweet (a tweet isn't a > design document) I specifically referred to D108469 where you wrote in the commit messages (870b6d21839707a3e4c40a29b526995f065a220f, 64ab2b1dcc5136a744fcac21d3d2c59e9cce040a): > This make clang more consistent with other implementation and fixes this > tweet https://twitter.com/jfbastien/status/1298307325443231744 :) If you were intentionally not doing what that tweet said to do, then it was not exactly a good idea to put in the commit message that you did, that just leads to unnecessary confusion. I agree with that tweet, so it's only natural that I considered it an accident that the implementation did not match the comment. > and to prevent abuses inherent with the use of format characters, that could > mess up the terminal or be used to missled users (as diagnostics contain > strings outside of our control) That's why this PR still prevents the control characters that could mess up the terminal from coming through. This is covered by the tests in that tweet. > I think this is a regression of diagnostics, for example I wasn't convinced about most of the character constants and wrote as such earlier, I've updated the PR so that character constants don't show formatting characters. The change in `warn-implicit-unicode-conversions.cpp` though for U+E000, I think that's an improvement, if you've got a system that gives some meaning to U+E000 (as mine does), the compiler shouldn't prevent that from being used. https://github.com/llvm/llvm-project/pull/231223 _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
