Bug#477366: linking ncurses-ruby against libncursesw5
> ncurses-ruby should be linked against libncursesw? (Namely, it would
> enable multibyte Ruby curses programs,
No. It would enable ruby curses programs that use "wide character"
strings. Programs that use multi-byte character strings such as UTF-8
strings would still not run correctly. See my description of display
errors in sup.
The concept of wide characters and wide character strings is not
supported by ruby. Not on the language level, not in the standard
library, not in any library I know of. Ruby 1.9 supports string
encodings, but "wide character" is not one of them. Depending on
compiler, operating system, processor architecture, and locale setting,
wide character width and encoding may be 1 byte (e.g. US-ASCII), 2 or 4
bytes (e.g. UCS-2, UCS-4 with architecture-dependent endianness). The
only portable way to convert a string to wide characters is to use
mbsrtowcs. And this works only for strings encoded in the current locale.
It does not mix well with ruby. Applications would have to store
integers in arrays to emulate wide character strings. I doubt that any
ruby application will ever do this.
> and it wouldn't break anything.)
It should not break anything, but apparently, it does. The non-"wide
character" fuctions of ncurses should behave exactly the same in
libncurses and libncursesw. The effect that you are after is apparently
caused by a bug in ncursesw.
I suppose the difference is caused by the
#if USE_WIDEC_SUPPORT
sections in waddch_nosync() in lib_addch.c, but I don't want to further
investigate in detail.
> Or are you, or anyone else, planning on producing a ncursesw-ruby?
I am not going to implement it myself. It would require wrapping of the
wide character functions of ncursesw, nothing too difficult. But nobody
is ever going to use them, since ruby and the wide character concept do
not work well together, so why bother.
If you want to do this, send me ncursesw_wrap.{h,c} files with wrappers
for add_wch() etc, and I am going to include it if it is complete,
LGPL2, and uses a convincing wstring implementation (e.g. arrays of
integers).
Your other options to get sup working:
1) You may not be aware that ncurses-ruby works fine with non-ascii
characters as long as they are contained in an 8 bit charset. Starting
sup as
LANG=de_DE.ISO-8859-15 xterm -e sup-mail
works for all the characters that I need. (You may need to generate the
locale using locale-gen)
2) Fork ncurses-ruby and distribute your modified version with the
extconf.rb patch applied. It's free software after all. It will work
until the bug in ncursesw that this solution relies on is fixed, albeit
with some display errors.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#477366: linking ncurses-ruby against libncursesw5
What's the status of this? Do you not buy my arguments about why ncurses-ruby should be linked against libncursesw? (Namely, it would enable multibyte Ruby curses programs, and it wouldn't break anything.) Or are you, or anyone else, planning on producing a ncursesw-ruby? -- William -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#477366: linking ncurses-ruby against libncursesw5
Does linking libcurses-ruby against ncursesw break legacy ASCII-only ncurses applications? I suspect not, but I could be wrong. Using a ruby ncurses library linked against ncursesw isn't *sufficient* for an app to magically work with non-ASCII encodings, as you illustrate. However, it is *necessary*. If legacy apps aren't broken, then I don't see a reason to produce a separate libncursesw-ruby package. Also, to clarify the situation wrt Sup, and other Ruby ncurses programs: under Ruby 1.9, they can finally correctly slice, modify, determine the length in characters of, etc. non-ASCII strings. But even under 1.9, Ruby ncurses applications won't be able to correctly display non-ASCII strings without access to the wcwidth and wcswidth functions, which AFAIK aren't actually provided by any common Ruby library anywhere. FWIW. -- William -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#477366: linking ncurses-ruby against libncursesw5
Adeodato Simó suggests to follow through with the suggestion of this bug report and link ncurses-ruby against ncursesw instead of ncurses because he has observed that the sup email program will display non-ascii characters better on a utf-8 terminal when linked like that. I am the upstream author of ncurses-ruby. I admit that until today I had no clear idea what the difference was between ncurses and ncursesw, apart from ncursesw "somehow" enabling "wide characters". I have investigated the matter today and I recommend not to link ncurses-ruby against ncursesw. Reasoning: I agree that it would be a good thing to have a ruby ncurses binding that links against ncursesw. Conventional ncurses (without the trailing w) only works well with 8 bit charsets. A few years ago, this has not been a problem for most users, as it was common then for linux distributions to configure local 8 bit charsets like ISO-8859-1. Now however, virtually every linux installation defaults to UTF-8 character encoding. With the consequence that non-ascii characters require more than one byte for encoding them. NCurses programs that worked fine in the old environment will no longer display non-ascii characters reliably. Is ncursesw the rescue? Yes, but its not that simple. You cannot simply link an ncurses program against ncursesw and expect it to magically work with UTF-8 Strings. In the email program mentioned above, you will still notice display errors when you use the cursor keys to highlight a line in the message body that contains non-ascii characters: Not the whole line is highlighted, a few character cells will remain black. If an email runs over several pages, then flipping the pages may cause some garbage from the previous page remain on the screen in lines containing non-ascii characters. What is happening? The email program still calls mvaddstr with an utf-8 encoded string. As far as ncurses(w) is concerned, the multiple bytes that make up a single non-ascii character are distributed to different character cells on the screen. The only reason why the user can recognise the original non-ascii character on the screen is that ncurses probably also happens to "print" the sub-character bytes in the correct sequence to the terminal, which then interprets the resulting UTF-8 encoding. However, after the printing, there is a disagreement on the horizontal position of the cursor between the terminal and the ncurses(w) library. The correct way to use ncursesw to print non-ascii, utf-8 encoded characters on a utf-8 terminal is for the application to split the string to print into (possibly multibyte) characters, compute the unicode codepoint for each character, and call the wide character functions of ncursesw (e.g. mvadd_wch, mvaddwstr). This requires a ncursesw-ruby wrapper as well as changes to the application. Looking at the source code of the mailer I'd say that it is not really suited for UTF-8 encoded strings yet, as it still assumes that the length of a string in bytes is equal to the number of characters in the string. Conclusions: - The switch from 8 bit character sets to UTF-8 requires serious modifications to applications using ncurses. Ncursesw cannot be used as a drop-in replacement. - A separate ncursesw-ruby wrapper is desirable. It has to export the additional wide character functions. Tobias -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Bug#477366: linking ncurses-ruby against libncursesw5
Hello, I don't really use the ncurses-ruby library, but I wanted to try the "sup" mailed which is written in Ruby and makes use of this library. Unfortunately, non-ASCII characters were not rendered correctly. Applying the patch provided by Micah Anderson to ncurses-ruby 1.1-3 solved the rendering problems in sup. It'd be great if this patch could be applied to the ncurses-ruby Debian package. We're in the beginning of a release cycle, and any regressions can be ironed out, or the patch reverted. Thanks, -- - Are you sure we're good? - Always. -- Rory and Lorelai -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
