On Saturday, December 7, 2019 5:23:30 AM MST Joseph Rushton Wakeling via Digitalmars-d-learn wrote: > On Saturday, 7 December 2019 at 03:23:00 UTC, Jonathan M Davis > > wrote: > > The module to look at here is std.utf, not std.encoding. > > Hmmm, docs may need updating then -- several functions in > `std.encoding` explicitly state they are replacements for > `std.utf`. Did you mean `std.uni`?
> It is honestly a bit confusing which of these 3 modules to use, > especially as they each offer different (and useful) tools. For > example, `std.utf.validate` is less useful than > `std.encoding.isValid`, because it throws rather than returning a > bool and giving the user the choice of behaviour. `std.uni` > doesn't seem to have any equivalent for either. > > Thanks in any case for the as-ever characteristically detailed > and useful advice :-) There may have been some tweaks to std.encoding here and there, but for the most part, it's pretty ancient. Looking at the history, it's Seb who marked some if it as being a replacement for std.utf, which is just plain wrong. Phobos in general uses std.utf for dealing with UTF-8, UTF-16, and UTF-32, not std.encoding. std.encoding is an old module that's had some tweaks done to it but which probably needs a pretty serious overhaul. The only thing that I've ever use it for is BOM stuff. std.utf.validate does need a replacement, but doing so gets pretty complicated. And looking at std.encoding.isValid, I'm not sure that what it does is any better from simply wrapping std.utf.validate and returning a bool based on whether an exception was thrown. Depending on the string, it would actually be faster to use validate, because std.encoding.isValid iterates through the entire string regardless. The way it checks validity is also completely different from what std.utf does. Either way, some of the std.encoding internals do seem to be an alternate implementation of what std.utf has, but outside of std.encoding itself, std.utf is what Phobos uses for UTF-8, UTF-16, and UTF-32, not std.encoding. I did do a PR at one point to add isValidUTF to std.utf so that we could replace std.utf.validate, but Andrei didn't like the implementation, so it didn't get merged, and I haven't gotten around to figuring out how to implement it more cleanly. - Jonathan M Davis