On Friday, December 6, 2019 9:48:21 AM MST Joseph Rushton Wakeling via Digitalmars-d-learn wrote: > Hello folks, > > I have a use-case that involves wanting to create a thin struct > wrapper of underlying string data (the idea is to have a type > that guarantees that the string has certain desirable properties). > > The string is required to be valid UTF-8. The question is what > the most useful API is to expose from the wrapper: a sliceable > random-access range? A getter plus `alias this` to just treat it > like a normal string from the reader's point of view? > > One factor that I'm not sure how to address w.r.t. a full range > API is how to handle iterating over elements: presumably they > should be iterated over as `dchar`, but how to implement a > `front` given that `std.encoding` gives no way to decode the > initial element of the string that doesn't also pop it off the > front? > > I'm also slightly disturbed to see that `std.encoding.codePoints` > requires `immutable(char)[]` input: surely it should operate on > any range of `char`? > > I'm inclining towards the "getter + `alias this`" approach, but I > thought I'd throw the problem out here to see if anyone has any > good experience and/or advice. > > Thanks in advance for any thoughts!
The module to look at here is std.utf, not std.encoding. decode and decodeFront can be used to get a code point if that's what you want, whereas byCodeUnit and byUTF can be used to get a range over code units or code points. There's also byCodePoint and byGrapheme in std.uni. std.encoding is old and arguably needs an overhaul. I don't think that I've ever done anything with it other than for dealing with BOMs. If you provide a range of UTF-8 code units, then it will just work with any code that's written to work with a range of any character type, whereas if you specifically need to have it be a range of code points or graphemes, then using the wrappers from std.utf or std.uni will get you that. And there really isn't any reason to restrict the operations on a range of char the way that std.range.primitives does for string. If you're dealing with a function that was specifically written to operate on any range of characters, then it's unnecessary, and if it's just a normal range-based function which isn't specialized for ranges of characters, then it's going to iterate over whatever the element type of the range is. So, you'll need to use a wrapper like byUTF, byCodePoint, or byGrapheme to get whatever the correct behavior is depending on what you're trying to do. The main hiccup is that a lot of Phobos is basically written with the idea that ranges of characters will be ranges of dchar. Some of Phobos has been fixed so that it doesn't, but plenty of it hasn't been. However, what that usually means is that the code just operates on the element type and special-cases for narrow strings, or it's specifically written to operate on ranges of dchar. For cases like that, byUTF!dchar or byCodePoint will likely work; alternatively, you can provide a way to access the underlying string and just have them operate directly on the string, but depending on what you're trying to do with your wrapper, exposing the underlying string may or may not be a problem (given that string has immutable elements though, it's probably fine so long as you don't provide a reference to the string itself). In general, I'd strongly advise against using alias this with range-based code (or really, generic code in general). Depending, it _can_ work, but it's also an easy source of bugs. Unless the code forces the conversion, what you can easily get is some of the code operating directly on the type and some of it doing the implicit conversion to operate on the type. Best case, that results in compilation errors, but it could also result in subtle bugs. It's far less error-prone to require that the conversion be done explicitly. So, if all you're really trying to do is provide some guarantees about how the string was constructed but then are looking to essentially just have it be a string after that, it would probably be simplest to make it so that your wrapper type doesn't have much in the way of operations and that it just provides a property to access the underlying string. Then the type itself isn't a range, and any code that wants to operate on the data can just use the property to get the underlying string and use it as a string after that. That approach basically completely sidesteps the issue of how to treat the data as a range, since you get the normal behavior for strings for any code that does much more than just pass around the data. You _do_ lose the knowledge that the wrapper type gave you about the state of the string once you start actually operating on the data, but once you start operating on it, that knowledge is probably no longer valid anyway (especially if you're passing it to a function which is going to return a wrapper range to mutate the elements in the range rather than something like find which just looks at the range). - Jonathan M Davis
