On Monday, 18 May 2020 at 13:44:15 UTC, Martin Tschierschke wrote:
Hi,
I have to find a certain line in a file, with a text containing
umlauts.
How do you do this?
The following was not working:
foreach(i,line; file){
if(line=="My text with ö oe, ä ae or ü"){
writeln("found it at line",i)
}
}
I ended up using line.canFind("with part of the text without
umlaut").
It solved the problem, but what is the right way to use umlauts
(encode them) inside the program?
Your code should have already worked like that, assuming your
input file is a UTF-8 file. Check with an editor like Notepad++
or Visual Studio Code what the actual encoding of your text file
is. In D all strings you specify in source are UTF-8 bytes in the
end and a byte-by-byte comparison like with your line == "..."
will cause it to fail if line is not UTF-8.
My guess is that your file is most likely gonna be encoded in
Windows-1251 or Windows-1252. To quickly check if it is UTF-8,
print out your strings but with separators between each character
like using `writefln("%(%s, %)", line.byUTF!dchar);` and see if
it is actually 'M', 'y', ' ', 't', 'e', 'x', 't', ' ', 'w', 'i',
't', 'h', ' ', 'ö', ' ', 'o', 'e', ',', ' ', 'ä', ' ', 'a', 'e',
' ', 'o', 'r', ' ', 'ü'
If you have identified that the character encoding is indeed your
problem, interpret your line with the correct character encoding
using
import std.encoding;
Windows1252String win1252Line = cast(Windows1252String)line;
and then convert that to utf8:
string utf8Line;
transcode(win1252Line, utf8Line);
and then compare that with your input string:
if (line == "My text with ö oe, ä ae or ü") { ... }
Alternatively you can also change your comparison string to be in
Windows 1251/1252 encoding if you know that all your files will
have this encoding, but I would advise against that and instead
somehow figure out the encoding based on common German characters
or an external library/program and always convert to UTF-8 for
all text operations.
Another tip: if you perform case-insensitive comparision with
UTF-8, use std.uni : sicmp or icmp (sicmp is faster / less
advanced) and use like `sicmp(strA, strB) == 0` where you replace
== with < or > if you want to sort. Note that this is not bound
to any locale and is rather the invariant locale. You will
probably want to use OS APIs or third party libraries to do
locale based text operations (like text in UI)
