Re: problem with Mime headers

2009-05-06 Thread Kyle Wheeler
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tuesday, May  5 at 06:19 PM, quoth Michael Tatge:
* On Tue, May 05, 2009 05:33PM +0200 Christoph Kukulies (k...@kukulies.org) 
muttered:
 Yes, in .muttrc I have :
 set charset=iso-8859-1# character set for your terminal

 That's my output of locale:

 LANG=de_DE.UTF-8
 LANGUAGE=de_DE:de:en_GB:en
 LC_CTYPE=de_DE.UTF-8

Why charset iso-8859-1 but utf-8 locales? Let mutt autodetect $charset,
or at least don't let charset differ from your locales.

Michael's right. The reason you don't want your locale and charset to 
disagree is because it confuses the libraries that mutt relies on. For 
example, when displaying characters, mutt needs to know what 
characters are printable and what characters are non-printable (e.g. 
control characters). A standard way to do this is to use the isprint() 
function in libc (this isn't the way mutt does it, but it's a simple 
example). isprint() relies on LANG and LC_CTYPE to figure out what 
characters are printable and what aren't. It assumes that the input 
character is in the charset specified in the locale environment 
variables, which mutt can do. But if you forced mutt to assume that 
the terminal only accepts iso-8859-1 characters, mutt believes 
something different than what isprint() believes, so when handed a 
given character, isprint() may say sure, that's printable when it's 
definitely NOT printable in the character set you specified with the 
$charset setting. Does that make sense? In other words: the libraries 
that mutt relies on for handling strings (such as libc) do not 
understand mutt's configuration.

I'm *guessing* that the reason you set $charset to be ISO-8859-1 is 
because your terminal (putty) freaks out when you try to display 
anything else. As it happens, recent versions of putty CAN understand 
UTF-8 output, if you click the right box in their configuration dialog 
box. If you don't want to use UTF8 for whatever reason (e.g. your 
version of putty really really can't handle it), then the right way to 
change it is to change your LANG and/or LC_CTYPE settings rather than 
to change mutt's $charset setting. That way all the software on the 
remote system can agree on what characters are displayable and what 
characters are not displayable.

Does that make sense?

~Kyle
- -- 
In the game of life and evolution there are three players at the 
table: Human Beings,  Nature, and Machines. I am firmly on the side of 
Nature. But Nature, I suspect, is on the side of the Machines.
-- George Dyson
-BEGIN PGP SIGNATURE-
Comment: Thank you for using encryption!

iEYEARECAAYFAkoAihoACgkQBkIOoMqOI16jvgCgs9MtU+2Si4SkEwx75xuyzqah
LkUAoNMsiaoMCav0vRPKHdeCODpN6trh
=zbzs
-END PGP SIGNATURE-


problem with Mime headers

2009-05-05 Thread Christoph Kukulies
During searching for a way to pretty print my Emails from mutt I found, 
that all messages (or at least the message under concern)
are stored from ISO-8859-1 to quoted-printable. I tried various 
converters, especially recode and recode is complaing about some
ungueltige Eingabe in data. (when I type =FC into stdin, recode behaves 
brave and issues an u-umlaut), so it's not the needs getting used to 
command line syntax of recode.


Anyway, looking closer to the stored message I find this:

|This message is in MIME format. Since your mail reader does not understand
|this format, some or all of this message may not be legible.
|
|--Boundary_(ID_nSHaJBsO42WhPK/6brziZw)
|Content-type: multipart/alternative;
|boundary=Boundary_(ID_MKUMWPz0I4J+Cb9RfTtFtg)
|
|
|--Boundary_(ID_MKUMWPz0I4J+Cb9RfTtFtg)
|Content-type: text/plain; charset=iso-8859-1
|Content-transfer-encoding: quoted-printable


Then the quoted printable message text follows
with the next message boundary.

My questions are:

\1 What mechanism causes mutt to produce a quoted printable encoded copy 
of the message, when I save the file?


\2 When I view the message in mutt, umlauts are shown as two byte 
characters, e.g. an u-umlaut is represented as \374.
   Would be nice, mutt would do that right away with the correct 
character set. Only when invoking the editor (vi) to

  write a reply, the characters are shown correctly.


--
Christoph







Re: problem with Mime headers

2009-05-05 Thread Kyle Wheeler
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tuesday, May  5 at 01:32 PM, quoth Christoph Kukulies:
 During searching for a way to pretty print my Emails from mutt I found, 
 that all messages (or at least the message under concern)
 are stored from ISO-8859-1 to quoted-printable.

I don't understand... you mean they're *converted* into 
quoted-printable from ISO-8859-1?

 I tried various converters, especially recode and recode is 
 complaing about some ungueltige Eingabe in data. (when I type =FC 
 into stdin, recode behaves brave and issues an u-umlaut), so it's 
 not the needs getting used to command line syntax of recode.

I'm pretty sure recode is not what you want.

My *guess* is that you have unset $print_decode, so mutt is passing 
raw message data to your $print_command. If you set $print_decode, 
mutt should convert all output into $charset as it gets passed to 
$print_command.

 My questions are:

 \1 What mechanism causes mutt to produce a quoted printable encoded copy 
 of the message, when I save the file?

None.

Let me clarify: when you save a copy of a message somewhere, mutt 
leaves it AS IS (unless you use something like decode-save). So if 
the message was encoded with quoted-printable to begin with, it will 
stay that way.

 \2 When I view the message in mutt, umlauts are shown as two byte 
 characters, e.g. an u-umlaut is represented as \374.
Would be nice, mutt would do that right away with the correct 
 character set. Only when invoking the editor (vi) to
   write a reply, the characters are shown correctly.

Technically, that's not a question. ;)

But if mutt is displaying umlauts as byte-codes (e.g. \374) instead of 
as an umlaut or even as a masked unprintable character (displayed as a 
question mark), then your charset environment is incorrectly 
configured.

Let me guess, you set $charset manually, don't you. (Hint: that's 
almost always the wrong thing to do, for lots of reasons.)

Step 1: stop setting $charset
Step 2: set up your LANG environment variable correctly
Possible Step 3: correctly configure your terminal

~Kyle
- -- 
I think we ought always to entertain our opinions with some measure of 
doubt. I shouldn't wish people dogmatically to believe any philosophy, 
not even mine.
-- Bertrand Russell
-BEGIN PGP SIGNATURE-
Comment: Thank you for using encryption!

iEYEARECAAYFAkoAVQwACgkQBkIOoMqOI14LxACgw/qx8gVs3NDg8276Y2EUXom6
Mz0AmQEVnCdDTDoefCbXHNWUPbzcHLvn
=dg0V
-END PGP SIGNATURE-


Re: problem with Mime headers

2009-05-05 Thread Christoph Kukulies

Kyle Wheeler schrieb:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Tuesday, May  5 at 01:32 PM, quoth Christoph Kukulies:
  
During searching for a way to pretty print my Emails from mutt I found, 
that all messages (or at least the message under concern)

are stored from ISO-8859-1 to quoted-printable.



I don't understand... you mean they're *converted* into 
quoted-printable from ISO-8859-1?
  


Yes, they are converted to quoted printable in the saved file.

  
I tried various converters, especially recode and recode is 
complaing about some ungueltige Eingabe in data. (when I type =FC 
into stdin, recode behaves brave and issues an u-umlaut), so it's 
not the needs getting used to command line syntax of recode.



I'm pretty sure recode is not what you want.

My *guess* is that you have unset $print_decode, so mutt is passing 
  


Yes, that's the case. I don't have $print_decode.

raw message data to your $print_command. If you set $print_decode, 
mutt should convert all output into $charset as it gets passed to 
$print_command.


  

My questions are:

\1 What mechanism causes mutt to produce a quoted printable encoded copy 
of the message, when I save the file?



None.

Let me clarify: when you save a copy of a message somewhere, mutt 
leaves it AS IS (unless you use something like decode-save). So if 
  


I don't have anything special WRT saving. But mutt definitely doesn't 
save the message as is.
At least I think so. I would have to fish out the message next time from 
my inbox before mut gets hold on it.


OK, here is the content of the message from /var/spool/mail:
Dies ist ein Test mit Ãmläuten

This is how it shows in a mutt terminal window (putty):

Dies ist ein Test mit \334ml\344uten

And in the save message
Dies ist ein Test mit Ãmläuten

OK, no conversion took place, you're right. Strange though that the 
multipart mime-encoded message
was converted (or did it arrive already that way?) I cannot reproduce 
that case at the moment.






the message was encoded with quoted-printable to begin with, it will 
stay that way.


  
\2 When I view the message in mutt, umlauts are shown as two byte 
characters, e.g. an u-umlaut is represented as \374.
   Would be nice, mutt would do that right away with the correct 
character set. Only when invoking the editor (vi) to

  write a reply, the characters are shown correctly.



Technically, that's not a question. ;)

But if mutt is displaying umlauts as byte-codes (e.g. \374) instead of 
as an umlaut or even as a masked unprintable character (displayed as a 
question mark), then your charset environment is incorrectly 
configured.


Let me guess, you set $charset manually, don't you. (Hint: that's 
  


Yes, in .muttrc I have :
set charset=iso-8859-1# character set for your terminal

That's my output of locale:

LANG=de_DE.UTF-8
LANGUAGE=de_DE:de:en_GB:en
LC_CTYPE=de_DE.UTF-8
LC_NUMERIC=de_DE.UTF-8
LC_TIME=de_DE.UTF-8
LC_COLLATE=de_DE.UTF-8
LC_MONETARY=de_DE.UTF-8
LC_MESSAGES=de_DE.UTF-8
LC_PAPER=de_DE.UTF-8
LC_NAME=de_DE.UTF-8
LC_ADDRESS=de_DE.UTF-8
LC_TELEPHONE=de_DE.UTF-8
LC_MEASUREMENT=de_DE.UTF-8
LC_IDENTIFICATION=de_DE.UTF-8
LC_ALL=



almost always the wrong thing to do, for lots of reasons.)

Step 1: stop setting $charset
Step 2: set up your LANG environment variable correctly
Possible Step 3: correctly configure your terminal

~Kyle
  

Thanks.

--
Christoph



Re: problem with Mime headers

2009-05-05 Thread Michael Tatge
* On Tue, May 05, 2009 05:33PM +0200 Christoph Kukulies (k...@kukulies.org) 
muttered:
 Yes, in .muttrc I have :
 set charset=iso-8859-1# character set for your terminal

 That's my output of locale:

 LANG=de_DE.UTF-8
 LANGUAGE=de_DE:de:en_GB:en
 LC_CTYPE=de_DE.UTF-8

Why charset iso-8859-1 but utf-8 locales? Let mutt autodetect $charset,
or at least don't let charset differ from your locales.

HTH,

Michael
-- 
To the systems programmer, users and applications serve only to provide a
test load.

PGP-Key-ID: 0xDC1A44DD
Jabber: init...@amessage.de