Screen corruption when unicode emoji are present
Hey folks, I run mutt 1.9.0 on Debian (linked against ncursesw) and 1.9.0-r1 from Portage on Gentoo. On each of these systems, I typically run mutt within a session of tmux. I usually connect to those sessions with mosh (1.3.2). The starting terminal is almost always iTerm2 on a macOS 10.12 system. I've tried ruling this problem out six ways to Sunday. When my maildir has messages that have unicode emoji in their subject lines, the screen will slowly start to corrupt itself as I scroll through the index. This corruption is usually subtle in tmux and rather severe in "screen." Yet, interestingly enough, it doesn't happen outside of tmux/screen. Yes, TERM is set properly inside of both (screen-256color). I have: - Self-compiled 1.9.0 with ncursesw on both systems - Tried a recent master copy of tmux on both systems - Tried mosh 1.3.2 and ssh interchangeably on both systems - Switched from iTerm2 to Terminal and back again In the process of trying to figure this out, I broke tmux's cardinal FAQ of having the right TERM setting inside tmux. When it's set to xterm-256color, I notice virtually no corruption. When set correctly (i.e., screen-256color), the emoji wreak havoc with the screen. I'm at a loss and, unfortunately, a bit out of my depth in troubleshooting much further. Has anyone else seen this / know of a workaround? Cheers, Matt
Re: Good Unicode support in fonts (was: Re: Just converted to UTF-8. Line graphics don't work. :-()
Quoth Aaron Toponce on Thursday, 12 May 2011: On Thu, May 12, 2011 at 04:08:24PM +, Alan Mackenzie wrote: I suspect the font I'm using is lacking support for the line graphics, and the driver for the screen is helpfully outputting an ASCII representation of the 3 UTF-8 bytes which code up the line graphic code. This is just an FYI based on personal experience, so take it as you will, but I've personally found the following fonts to have good overall Unicode support for my needs (your needs might be different): 0) DejaVu 1) Liberation 2) Courier Some fonts that I have found lack good (if any at all) Unicode support are: 0) Calibri, Cambria, Candara, Constantia, Consolas and Corbel 1) Bitstream 2) Most standard free fonts installed by default I was surprised that the Vista fonts were as lacking as they were. They're essentially just Latin, Greek and Cyrillic fonts, which is disappointing. I would expect more from a major organization with deep pockets. Awards or no awards, they are seriously lacking. I prefer DejaVu for my font rendering almost everywhere. It's clean, supports a substantial number of glyphs, is updated frequently, and is licensed under a free license. Anyway, thought I would share. -- . o . o . o . . o o . . . o . . . o . o o o . o . o o . . o o o o . o . . o o o o . o o o +1 on DejaVu. For the unusual unicode glyphs, I use Code2000 as a backup font. But line-drawing characters are included in DejaVu. -- .O. | Sterling (Chip) Camden | http://camdensoftware.com ..O | sterl...@camdensoftware.com | http://chipsquips.com OOO | 2048R/D6DBAF91 | http://chipstips.com pgpBSN7Zdp9pd.pgp Description: PGP signature
Re: unicode
On Sun, Nov 28, 2010 at 02:52:22PM -0800, Chip Camden wrote: Someone must have solved this problem before, but all the Googling in the world isn't helping me so far. on my FreeBSD system, which i believe you are using, i managed to get it to display these characters by setting the locale as Derek pointed out; but, the values i used for $LANG and $MM_CHARSET are en_GB.ISO8859-1 and ISO-8859-1 respectively. This is on the FreeBSD web faq/manual i believe where it explains about localisation. You shouldn't need to set any of the other locale values - at least i didn't need to but i'm no expert. Jamie
Re: unicode
On Mon, Nov 29, 2010 at 10:58:12AM +, Jamie Paul Griffin wrote: On Sun, Nov 28, 2010 at 02:52:22PM -0800, Chip Camden wrote: Someone must have solved this problem before, but all the Googling in the world isn't helping me so far. on my FreeBSD system, which i believe you are using, i managed to get it to display these characters by setting the locale as Derek pointed out; but, the values i used for $LANG and $MM_CHARSET are en_GB.ISO8859-1 and ISO-8859-1 respectively. This is on the FreeBSD web faq/manual i believe where it explains about localisation. You shouldn't need to set any of the other locale values - at least i didn't need to but i'm no expert. Setting a ISO-8859 locale will mostly work but it's not so all encompassing as using UTF-8 so if you can use UTF-8 it's better. ISO-8859 character sets are basically only the 'Roman' character sets of western[ish] Europe. Using UTF-8 will show almost anything, I get to see chinese spam in all its chinese glory sometimes! :-) -- Chris Green
Re: unicode
Setting a ISO-8859 locale will mostly work but it's not so all encompassing as using UTF-8 so if you can use UTF-8 it's better. ISO-8859 character sets are basically only the 'Roman' character sets of western[ish] Europe. Using UTF-8 will show almost anything, I get to see chinese spam in all its chinese glory sometimes! :-) Out of interest Chris, do you use FreeBSD? I only ask because oddly enough i had better results using the ISO character sets than with UTF-8. I don't fully understand why and perhaps it's a BSD issue/thing. With my OpenBSD system i have even more trouble getting unicode characters to display properly. Jamie
Re: unicode
On Mon, Nov 29, 2010 at 11:38:43AM +, Jamie Paul Griffin wrote: Setting a ISO-8859 locale will mostly work but it's not so all encompassing as using UTF-8 so if you can use UTF-8 it's better. ISO-8859 character sets are basically only the 'Roman' character sets of western[ish] Europe. Using UTF-8 will show almost anything, I get to see chinese spam in all its chinese glory sometimes! :-) Out of interest Chris, do you use FreeBSD? I only ask because oddly enough i had better results using the ISO character sets than with UTF-8. I don't fully understand why and perhaps it's a BSD issue/thing. With my OpenBSD system i have even more trouble getting unicode characters to display properly. No I don't, I use Linux (Xubuntu). I only moved from ISO-8859 to UTF-8 a little while ago though, mainly because until a year or two go I did a lot of work on legacy Sun systems which, as regards characters sets etc. were back in the dark ages and for cross compatibility with them ISO-8859 made things easier. My basic needs are really only western European character sets (I do actually read and write news and E-Mail in French as well as English and read, rarely, Polish) so ISO-8859 does most of what I want. I just found that so much software is now defaulting to UTF-8 that it was easier to go with that now that I don't have to deal with ancient Sun systems. -- Chris Green
Re: unicode
No I don't, I use Linux (Xubuntu). I only moved from ISO-8859 to UTF-8 a little while ago though, mainly because until a year or two go I did a lot of work on legacy Sun systems which, as regards characters sets etc. were back in the dark ages and for cross compatibility with them ISO-8859 made things easier. My basic needs are really only western European character sets (I do actually read and write news and E-Mail in French as well as English and read, rarely, Polish) so ISO-8859 does most of what I want. I just found that so much software is now defaulting to UTF-8 that it was easier to go with that now that I don't have to deal with ancient Sun systems. Ok thanks Chris, I see your point. I've just checked again and for FreeBSD using the UTF-8 locale does not work for me, especially when using mutt and reading man pages. Using the ISO character sets and setting $LANG and $MM_CHARSET as I mentioned earlier works. Also, on my OpenBSD system setting $LC_CTYPE to use one of the ISO character sets works well. Setting $LC_CTYPE to UTF-8 however has very poor results. The thread tree in mutt's index is all screwed up and the messages overlap as the indicator moves over them. $LC_CTYPE is the only variable option available on the default installation for localisation. Jamie
Re: unicode
On Sun, Nov 28, 2010 at 07:50:08PM -0600, Derek Martin wrote: On Sun, Nov 28, 2010 at 02:52:22PM -0800, Chip Camden wrote: inside rxvt-unicode (urxvt) v9.07 and I can't seem to get unicode characters to display properly. I have: set charset=utf-8 This comes up often enough that it should probably be a FAQ... First off, don't set charset. You shouldn't need to, and -- unless you're doing something very funky and you really, really know what you're doing -- having to do this means your environment is not set up properly. Most likely, setting this manually will only work against you. [...] +1 In general, using an off-the-shelf desktop on Linux/*BSD/Solaris should cause everything to be in order, particularly if you use a UTF-8 locale to begin with. When everything's in order (you have the necessary locales and fonts installed, and you're using blessed desktops / start scripts) then you will have the locale environment variables properly setup and your tools will find their fonts/renderers and codeset conversion modules and so on. Mutt too will be able to do codeset conversions and thus display foreign characters to the best of your locale's ability. If you must use a non-UTF-8 locale yet want to be able to use UTF-8 for your mutt instances (e.g., to be able to display more characters than your locale allows, or to be able send mail using UTF-8 as your locale), then you'll want to run a terminal emulator that allows you to pick an encoding: set the encoding of the terminal where you run mutt to UTF-8, make sure to change the locale env vars accordingly in that session, and start mutt. For example, gnome-terminal allows you to set the encoding on a per-tab basis. But it's better to just use a UTF-8 locale for all your sessions and work. Nico --
Re: unicode
Quoth Nicolas Williams on Monday, 29 November 2010: On Sun, Nov 28, 2010 at 07:50:08PM -0600, Derek Martin wrote: On Sun, Nov 28, 2010 at 02:52:22PM -0800, Chip Camden wrote: inside rxvt-unicode (urxvt) v9.07 and I can't seem to get unicode characters to display properly. I have: set charset=utf-8 This comes up often enough that it should probably be a FAQ... First off, don't set charset. You shouldn't need to, and -- unless you're doing something very funky and you really, really know what you're doing -- having to do this means your environment is not set up properly. Most likely, setting this manually will only work against you. [...] +1 In general, using an off-the-shelf desktop on Linux/*BSD/Solaris should cause everything to be in order, particularly if you use a UTF-8 locale to begin with. When everything's in order (you have the necessary locales and fonts installed, and you're using blessed desktops / start scripts) then you will have the locale environment variables properly setup and your tools will find their fonts/renderers and codeset conversion modules and so on. Mutt too will be able to do codeset conversions and thus display foreign characters to the best of your locale's ability. If you must use a non-UTF-8 locale yet want to be able to use UTF-8 for your mutt instances (e.g., to be able to display more characters than your locale allows, or to be able send mail using UTF-8 as your locale), then you'll want to run a terminal emulator that allows you to pick an encoding: set the encoding of the terminal where you run mutt to UTF-8, make sure to change the locale env vars accordingly in that session, and start mutt. For example, gnome-terminal allows you to set the encoding on a per-tab basis. But it's better to just use a UTF-8 locale for all your sessions and work. Nico -- Thanks to everyone who offered advice. As usual in a case like this, I was overcomplicating the situation. All I needed to do was export LC_CTYPE=en_US.UTF-8 prior to starting urxvt, and everything works as expected. Funny, uxterm worked OK with specifying the charset in .muttrc, but urxvt needed LC_CTYPE to be correct instead. Thanks again! -- Sterling (Chip) Camden| sterl...@camdensoftware.com | 2048D/3A978E4F http://camdensoftware.com | http://chipstips.com| http://chipsquips.com pgpFutz1N4DUK.pgp Description: PGP signature
unicode
Someone must have solved this problem before, but all the Googling in the world isn't helping me so far. I'm running Mutt 1.4.2.3i (2007-05-26) Copyright (C) 1996-2002 Michael R. Elkins and others. Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'. Mutt is free software, and you are welcome to redistribute it under certain conditions; type `mutt -vv' for details. System: FreeBSD 8.2-PRERELEASE (amd64) [using slang 20202] Compile options: -DOMAIN -DEBUG -HOMESPOOL +USE_SETGID +USE_DOTLOCK +DL_STANDALONE -USE_FCNTL +USE_FLOCK +USE_POP +USE_IMAP +USE_GSS +USE_SSL -USE_SASL +HAVE_REGCOMP -USE_GNU_REGEX +COMPRESSED +HAVE_COLOR -HAVE_START_COLOR -HAVE_TYPEAHEAD -HAVE_BKGDSET -HAVE_CURS_SET -HAVE_META -HAVE_RESIZETERM +HAVE_PGP -BUFFY_SIZE -EXACT_ADDRESS -SUN_ATTACHMENT +ENABLE_NLS -LOCALES_HACK +HAVE_WC_FUNCS +HAVE_LANGINFO_CODESET +HAVE_LANGINFO_YESEXPR +HAVE_ICONV -ICONV_NONTRANS +HAVE_GETSID +HAVE_GETADDRINFO -ISPELL SENDMAIL=/usr/sbin/sendmail MAILPATH=/var/mail PKGDATADIR=/usr/local/share/mutt SYSCONFDIR=/usr/local/etc EXECSHELL=/bin/sh -MIXMASTER vvv.initials 1.3.28.nr.threadcomplete rr.compressed inside rxvt-unicode (urxvt) v9.07 and I can't seem to get unicode characters to display properly. I have: set charset=utf-8 in my .muttrc. My fonts for urxvt are: URxvt*font: xft:Deja Vu Sans Mono:pixelsize=12, xft:Code2000 Sans Mono:pixelsize=11;antialias=false What else do I need? Many TIA, -- Sterling (Chip) Camden| sterl...@camdensoftware.com | 2048D/3A978E4F http://camdensoftware.com | http://chipstips.com| http://chipsquips.com pgpv20C9nC0V2.pgp Description: PGP signature
Re: unicode
On Sun, Nov 28, 2010 at 02:52:22PM -0800, Chip Camden wrote: inside rxvt-unicode (urxvt) v9.07 and I can't seem to get unicode characters to display properly. I have: set charset=utf-8 This comes up often enough that it should probably be a FAQ... First off, don't set charset. You shouldn't need to, and -- unless you're doing something very funky and you really, really know what you're doing -- having to do this means your environment is not set up properly. Most likely, setting this manually will only work against you. Make sure your LANG environment variable (or its many cousins) is properly set to a UTF-8 locale. For example, this is my environment (on a redhat-ish system): $ locale LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 LC_NUMERIC=en_US.UTF-8 LC_TIME=en_US.UTF-8 LC_COLLATE=C LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=en_US.UTF-8 LC_ADDRESS=en_US.UTF-8 LC_TELEPHONE=en_US.UTF-8 LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=en_US.UTF-8 LC_ALL= Note that values in quotes are inherited from the $LANG environment variable, and those without quotes are directly set. Also note that this all needs to be done *BEFORE* whatever launches your rxvt windows starts; e.g. if you're using some window manager gizmo to launch your windows, it needs to be done before the window manager starts. This may mean you need to set the system-wide default to a UTF-8 encoding, or hack your start-up scripts, depending on what system you're running on, and how its X startup stuff works (or doesn't). Next, you need to make sure you're using a proper font that has all the UTF-8 glyphs. I'm not familiar with rxvt's font configuration, but the font you listed may or may not have them... strikes me it wouldn't, depending on what you're trying to see. Until I became too lazy to not use whatever my system provided as its default terminal window, I configured my xterm with this font: -misc-fixed-medium-r-semicondensed-*-13-*-*-*-*-*-iso10646-* In the past I had to install some unicode font package to make sure this font was present; these days I don't recall having to do that, but it's been a while since I installed a fresh system. This font is (or should be) the universal unicode font, which contains very many of the Unicode glyphs, including most for asian character support. In any event, if your locale settings ($LANG) is set properly, and your terminal window is using proper fonts that contain the glyphs you want to display, Mutt should just work without you having to set *anything* in Mutt's config. Oh, also... if you hand-compile Mutt, you probably need to make sure you're compiling against ncursesw(-dev) instead of ncurses(-dev) for Unicode to work properly (note the 'w' at the end). These days I use gnome-terminal primarily, which uses newer font rendering libraries (bonobo, I believe) to magically guess which fonts contain characters for the charset you're trying to display. In some sense this is better because you don't have to futz with stuff to get any given subset of unicode characters to display properly; though when you end up with a font that you hate for some sub-charset you use, figuring out how to fix that can be very painful. :( On the whole I still prefer xterm, but I haven't ever figured out how to make its modern font-rendering features work in a way that's nice, and the bitmapped fonts really don't look as good IMO, so I gave in and reluctantly switched. I'm not ranting, per se... I mention this mainly because if you struggle to get this working, and if there's no strongly compelling reason for you to stick with rxvt, then you may find that switching to a more modern terminal emulator solves your problem rather immediately. Maybe you care, maybe you don't. I hope at least some of this was helpful. =8^) -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpQXxB7ncOp3.pgp Description: PGP signature
Re: Unicode and Mutt
* Dave Feustel [EMAIL PROTECTED] [20081103 14:03]: I saved a few messages to mutt folders that had Russian or Asian Characters in the subject line. Now when I look at those saved messages the Russian/Asian characters are no longer displayed. Is there any way to make mutt work with UTF-8 so that email message text does not get screwed up? Thanks. mutt (at least the version I have) works fine with Unicode characters as long as your terminal and LANG settings handle it. I run mutt in urxvt, and with the font xft:DejaVu Sans Mono-10. This does display most kanji, chinese and cyrillic characters I have come across. LANG is set to en_GB.utf8. HTH, -- Anders Karlsson [EMAIL PROTECTED] All-Round Linux Tinkerer RHCE
Re: Unicode and Mutt
On Mon, Nov 03, 2008 at 02:50:40PM +0100, Anders Karlsson wrote: * Dave Feustel [EMAIL PROTECTED] [20081103 14:03]: I saved a few messages to mutt folders that had Russian or Asian Characters in the subject line. Now when I look at those saved messages the Russian/Asian characters are no longer displayed. Is there any way to make mutt work with UTF-8 so that email message text does not get screwed up? Thanks. mutt (at least the version I have) works fine with Unicode characters as long as your terminal and LANG settings handle it. I run mutt in urxvt, and with the font xft:DejaVu Sans Mono-10. This does display most kanji, chinese and cyrillic characters I have come across. LANG is set to en_GB.utf8. HTH, Did you try saving the messages and verifying that the saved messages still display properly? Where is LANG set? I am using xterm and I wonder if a yum update could have changed those settings. I am currently reading the book _Unicode Explained_ by Jukka K. Korpela, and getting Unicode to work across the desktop is more involved than I originally thought.
Re: Unicode and Mutt
On Mon, Nov 03, 2008 at 10:14:15AM -0500, Dave Feustel wrote: Did you try saving the messages and verifying that the saved messages still display properly? Lots of people have been using Mutt with UTF-8 for years, and yes, this is known to work if your environment is set up correctly. Where is LANG set? In the environment. See the man pages for locale(1) and environ(7) on your system, as well as the bash man page if you don't know how to set environment variables. You need to make sure it is set correctly so that iconv will convert (or not convert) the messages properly. You'll also need to use a unicode font with your xterm, AND make sure it has all the glyphs that you want to see... It sounds like you may already have that set up, but if you don't try adding either of these to your ~/.Xdefaults file: XTerm*font: -misc-fixed-medium-r-semicondensed-*-13-*-*-*-*-*-iso10646-* XTerm*font: -Misc-Fixed-Medium-R-Normal--18-120-100-100-C-90-ISO10646-1 Both of these fonts have a very large number of character glyphs from most of the world's major languages, though in each font, at least a (different) subset of the Asian characters is missing. Then run this command: xrdb -merge ~/.Xdefaults And after that, any new xterms you open should display characters properly. You may need to install the fonts first... Alternately, try running Mutt in a gnome-terminal window, which is built with libraries to select and display an appropriate font based on the natural language character set from which the characters originate. [Of course, you still need to have appropriate fonts installed for the language you want to see...] If you still need more help, google is rife with pages which describe how to configure your locale properly to display unicode correctly. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. pgpFZ7WJpLxvQ.pgp Description: PGP signature
Re: Unicode and Mutt
On Mon, Nov 03, 2008 at 12:51:03PM -0500, Derek Martin wrote: On Mon, Nov 03, 2008 at 10:14:15AM -0500, Dave Feustel wrote: Did you try saving the messages and verifying that the saved messages still display properly? Lots of people have been using Mutt with UTF-8 for years, and yes, this is known to work if your environment is set up correctly. Where is LANG set? In the environment. See the man pages for locale(1) and environ(7) on your system, as well as the bash man page if you don't know how to set environment variables. You need to make sure it is set correctly so that iconv will convert (or not convert) the messages properly. You'll also need to use a unicode font with your xterm, AND make sure it has all the glyphs that you want to see... It sounds like you may already have that set up, but if you don't try adding either of these to your ~/.Xdefaults file: XTerm*font: -misc-fixed-medium-r-semicondensed-*-13-*-*-*-*-*-iso10646-* XTerm*font: -Misc-Fixed-Medium-R-Normal--18-120-100-100-C-90-ISO10646-1 Both of these fonts have a very large number of character glyphs from most of the world's major languages, though in each font, at least a (different) subset of the Asian characters is missing. Then run this command: xrdb -merge ~/.Xdefaults And after that, any new xterms you open should display characters properly. You may need to install the fonts first... Alternately, try running Mutt in a gnome-terminal window, which is built with libraries to select and display an appropriate font based on the natural language character set from which the characters originate. [Of course, you still need to have appropriate fonts installed for the language you want to see...] If you still need more help, google is rife with pages which describe how to configure your locale properly to display unicode correctly. -- Derek D. Martinhttp://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. Thanks very much for the info I thought I had everything set up for Unicode, but the loss of the Russian and Asian glyphs in the saved emails prove otherwise
rxvt-unicode tip Re: Charset issue?
On 14May2007 12:24, Alain Bench [EMAIL PROTECTED] wrote: | On Monday, May 14, 2007 at 8:49:16 +1200, Roland Hill wrote: | | $ printf L1: won\xB4t \xA8reply\xA8\nU8: won\xC2\xB4t \xC2\xA8reply\xC2\xA8\n | - L1 line I get won't correctly followed by a quote mark. | - U8 line I get won followed by an A with a caret on top followed |by an apostrophe and a t. | | Fine: Those were indeed Latin-1 terminals. The day you don't see the | A circumflex but a correct U8 line, this will mean UTF-8 term. Might I say that printf line is enormously useful! I have just now fixed my own unicode setup with its help. For anyone using rxvt-unicode, this FAQ entry is very important: http://cvs.schmorp.de/rxvt-unicode/doc/rxvt.7.html#unicode_does_not_seem_to_work For historic reasons (now bogus) I've been running with LC_CTYPE=C and LC_COLLATE=C in my environment, still expecting UTF-8 to render in my urxvt terminals. I've backed off to just the LC_COLLATE setting (so ls and shell globbing behave as I expect/want) and it's all good now! Alain, thank you! -- Cameron Simpson [EMAIL PROTECTED] DoD#743 http://www.cskk.ezoshosting.com/cs/ Hold up a one iron and walk. Even God can't hit a one iron. - Lee Travino, on how to not get stuck by lightning.
Re: Unicode
Gero Treuner: I acquired xterm-117, which seems to support utf-8 at least to the extent that it looks right when I cat a utf-8 file to the terminal. However, if I run mutt in the xterm, with charset=utf-8, it doesn't look right: Mutt assumes to have an 8-bit clean terminal for the best case, all characters are sent as byte wide numbers to the terminal and supposed to be displayed directly using the current font. With a utf-8 terminal characters 128 and above are interpreted as utf-8 sequences, which leads to unexpected results. Currently it isn't possible to make mutt use more than 8-bit character sets plus the few terminal graphic characters. The reason: The ncurses library currently don't support this. First there has to be a usuable utf-8 xterm implementation, then ncurses can be improved to handle wide characters (ready at version 6?), and finally mutt can be changed to be aware of this extension (it works with unicode internally already). All this may take a year at least, I think. I think you're being pessimistic! There is already a usable utf-8 xterm: patch level 117 seems to work, as I mentioned, and it will find its way into standard XFree86 releases before long. There's at least one editor that runs in a utf-8 xterm: mined98. There is information about this and related topics in the Unicode-HOWTO. I built mined linked with ncurses, but it's possible that it bypasses curses to some extent. It's true that ncurses and slang don't understand wide characters, but what really stops mutt from running in a utf-8 xterm is bugs in mutt. I have corrected some of those and am sending a patch to mutt-dev. It does now seem to work to an acceptable extent. Presumably tabs and line breaks are wrong, and sometimes slang gets confused by utf-8 and redraws some screen lines incorrectly, but you can just press C-l to tidy things up. It's usable. And I can see Russian, Greek, Hungarian, etc all on the screen at once ... Edmund
Unicode
I acquired xterm-117, which seems to support utf-8 at least to the extent that it looks right when I cat a utf-8 file to the terminal. However, if I run mutt in the xterm, with charset=utf-8, it doesn't look right: Characters 0xa0..0xff in an iso-8859-X attachment are not being converted to utf-8; they are all displayed as "?", except for a very few, which are displayed as " ". Characters in a utf-8 attachment are sometimes displayed correctly, sometimes displayed as "? ", sometimes as " " (two spaces). Do I need a newer slang or ncurses for utf-8 to work? Has anyone had utf-8 working with mutt? If anyone else wants to try this, it's very easy. You don't even need any new fonts because there are enough problems getting utf-8 encoded 8859-1 characters to work. Just get http://www.clark.net/pub/dickey/xterm/xterm.tar.gz, ./configure --enable-wide-chars, and make. Then "xterm -u8". I'll be happy to send you an e-mail for you to test with ... If you do want some Unicode fonts, there are instructions in: ftp://ftp.ilog.fr/pub/Users/haible/utf8/Unicode-HOWTO.html Edmund
Re: Unicode
Hi! On Fri, Oct 01, 1999 at 03:00:54PM +0100, Edmund GRIMLEY EVANS wrote: I acquired xterm-117, which seems to support utf-8 at least to the extent that it looks right when I cat a utf-8 file to the terminal. However, if I run mutt in the xterm, with charset=utf-8, it doesn't look right: Mutt assumes to have an 8-bit clean terminal for the best case, all characters are sent as byte wide numbers to the terminal and supposed to be displayed directly using the current font. With a utf-8 terminal characters 128 and above are interpreted as utf-8 sequences, which leads to unexpected results. Currently it isn't possible to make mutt use more than 8-bit character sets plus the few terminal graphic characters. The reason: The ncurses library currently don't support this. First there has to be a usuable utf-8 xterm implementation, then ncurses can be improved to handle wide characters (ready at version 6?), and finally mutt can be changed to be aware of this extension (it works with unicode internally already). All this may take a year at least, I think. Gero
