On 6/3/16 8:06 AM, Jonathan M Davis via Digitalmars-d wrote:
On Friday, June 03, 2016 07:37:59 Steven Schveighoffer via Digitalmars-d wrote:
But consider the case where you are searching the string: "cassé"
for the letter 'e'. If é is encoded as 'e' + U+0301, then you will
succeed when you should fail! However, it may be that you actually want
to find specifically any code points with 'e', including ones with
combining characters. This is why we really need more discretion from
Phobos, and less hand-holding.
There are certainly searches that will be correct. For example,
searching for newline should always work in code-point space. Actually,
what happens when you use a combining character on newline? Is it an
invalid unicode sequence? Does it matter? :)
A nice function to determine whether code points or graphemes are
required for comparison given a needle may be useful.
Well, if you know that you're dealing with a grapheme that has that problem,
you can just iterate by graphemes rather than code units like find would
normally.
Yes, I agree. This is exactly the point. Don't assume anything, just
treat a type as it is written. And tell the user this!
If you are going to search a range of code points with a code point, you
may not get what you expect. If you want to do a grapheme-aware search,
change it to a range of graphemes, and do a grapheme search.
What I was trying say with my example is that searching by code points,
even for graphemes that definitively fit into one code point, may still
not be correct in all use cases.
-Steve
