On 08/23/2012 03:50 PM, Stefan Sperling wrote:
On Thu, Aug 23, 2012 at 08:58:53PM +0200, Eivind Evensen wrote:
Since version 1.10 of lib/libc/gen/ctype_.c, I've been
unable to use localized characters in mg properly (they're printed
as an octal value only).
I've been using the below change to regain support for printing them
normally.
Best regards, Eivind Evensen
Index: main.c
===================================================================
RCS file: /data/openbsd/src/usr.bin/mg/main.c,v
.....
Eivind
This kind of change has been proposed before.
In my opinion it is not the right way of solving this problem.
It won't work correctly with multi-byte files (like UTF-8). E.g. typing
backspace to delete one character will delete one byte instead of the
entire character, which messes up the display. To properly support multi-byte
encodings mg needs to learn the difference between a byte and a character.
The locales mechanism and wchar_t are only useful for applications that do
not care about details of character encodings, and which only need to deal
with a single character set at a time. It is not very useful for editors
because they need to handle files in various encodings and be aware of
the current encoding in use.
Some applications in base (less and tmux, for example) have special
support code for UTF-8. This could be done for mg as well, so that
it can support single-byte character sets (ASCII, latin1) and also
UTF-8 (but no other multi-byte character set). You'd activate the
special UTF-8 mode if nl_langinfo(CODESET) returns "UTF-8".
To properly support arbitrary multi-byte character sets (UTF-8, UTF-16,
special asian language encodings etc) mg needs iconv which we don't have
in base. I have some work-in-progress iconv code but it's not ready for
the tree yet and I'm not actively working on it at the moment.
If you want to help out with this let me know.
Using iconv in an editor is EXTREMELY dangerous without complex precautions.
Given a file containing characters not valid in the current locale,
it will at minimum prevent viewing the file.
If the file is written out, the file is destroyed.
IMnsHO, that is fatally flawed.
Returning an error for an "impossible" character translation is specified
in the archaic version of the Unicode standard I read.
That is not useful in an editor.
Geoff Steckel
