Re: setting mutt to charset UTF-8 ?

2010-10-05 Thread Matthias Apitz
El día Monday, October 04, 2010 a las 11:06:22PM -0500, Kyle Wheeler escribió:

 I've been using mutt as a UTF-8 enabled program for... gosh, 
 probably four years now. So, it works, and it works well. Here are 
 some things to consider, though:
 
 1. As has been said, mutt uses the smallest necessary charset (of the 
 options listed in $send_charset, in order). This makes it very
 compatible; for the most part, I generally use the UTF-8 capability
 only for displaying emails (with some rare exceptions).
...

Thanks for all the hints from you and the others.

When I set

set send_charset=us-ascii:iso-8859-1:utf-8

mutt (1.5.19) says:

Error in /home/guru/.muttrc, line 70: Invalid value for option
send_charset: us-ascii:iso-8859-1:utf-8

I double checked this against the man page and even cutpaste the value
from there...

What does this mean?

matthias
-- 
Matthias Apitz
t +49-89-61308 351 - f +49-89-61308 399 - m +49-170-4527211
e g...@unixarea.de - w http://www.unixarea.de/


Re: setting mutt to charset UTF-8 ?

2010-10-05 Thread Kyle Wheeler
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Tuesday, October  5 at 11:58 AM, quoth Matthias Apitz:
Error in /home/guru/.muttrc, line 70: Invalid value for option
send_charset: us-ascii:iso-8859-1:utf-8

I double checked this against the man page and even cutpaste the value
from there...

What does this mean?

Literally, it means that one of those charsets wasn't in the list of 
preferred MIME names (i.e. isn't supported by your version of iconv) 
AND your mutt was compiled without utf8 support.

Check your mutt -v output; you may need to recompile mutt to enable it 
to do fancy character set stuff.

~Kyle
- -- 
All men dream; but not equally.
  -- T. F. Lawrence (of Arabia)
-BEGIN PGP SIGNATURE-
Comment: Thank you for using encryption!

iQIcBAEBCAAGBQJMq0IBAAoJECuveozR/AWeXq4QAJtS8o12EiZWX9ooUsF4+MX4
ivl2pTMDuV6CCUxqkLX+ikOOu7yCHwfIuTEyIwAnNwLJfhGt1uAwDYbWwSKOFnZ5
/sHrcpD/49qk8JFtx/IpqqrAvW2UI8jUALqeSsn46WMMjwW0ytZDNrC9JYUiQ9br
Tr4LCEC6SP+TcoD1rJbwKqtO0xyQpaMltYfS2ZOtvOlCwRMbzjI4qDaPAWSVFIiI
zoxR6tZDjxZu8cNqZ5XLszwvUrE+gefcU+mexvd8xz6gIXoTcijjlQwja++gyW04
9NfHvwiNOOcmnbA59Td1V2szDdyjvSeIxdt+GDACIt9L5n2f/8U5rue6iW68Rc4L
btgbM21WWJwAbpmghU04Wrq9mtz4t95BY4+FiT+nx9puXlaQlsqyQLkwy7MJ5IEg
pyeIsLJl1z6fn2YTpD5YQDYaeWSTbnSo6LiXJsuYWvnfCx5UrlUQfprVTpK4e+Wi
sqJ2zIikhvcoJU8iqWl8LkY7OhoXZ2Habz9hYbYCETQj/LvmDkmziv0W0JsaeLSX
z0fgJwrCCGSUdHDePsyftF1NGZuXtQN4ehbJv0DVJecL2g8dbw6GN+e6hBfV2vsd
NmHuQ9MxoLZcZA7N+HI5DzEuoBL4B42waNYdGyJKItc7LmM+uoNR6EJYFonqnX6A
OWKaC9jLyBTx9zqIdWhs
=MdWw
-END PGP SIGNATURE-


Re: setting mutt to charset UTF-8 ?

2010-10-05 Thread Matthias Apitz
El día Tuesday, October 05, 2010 a las 10:19:29AM -0500, Kyle Wheeler escribió:

 On Tuesday, October  5 at 11:58 AM, quoth Matthias Apitz:
 Error in /home/guru/.muttrc, line 70: Invalid value for option
 send_charset: us-ascii:iso-8859-1:utf-8
 
 I double checked this against the man page and even cutpaste the value
 from there...
 
 What does this mean?
 
 Literally, it means that one of those charsets wasn't in the list of 
 preferred MIME names (i.e. isn't supported by your version of iconv) 
 AND your mutt was compiled without utf8 support.

Hmm. It works if I only set

set send_charset=utf-8

 Check your mutt -v output; you may need to recompile mutt to enable it 
 to do fancy character set stuff.

In the output of mutt -v it says (among other stuff):

-HAVE_ICONV
-ICONV_NONTRANS

but nothing positiv/negativ about UTF-8. I can't check the
exact configure values (because the port is on my real work
laptop and on my netbook I only have the created package
installed). But it is curious, I can send UTF-8 messages
with this mutt :-)

Thanks anyway

matthias

-- 
Matthias Apitz
t +49-89-61308 351 - f +49-89-61308 399 - m +49-170-4527211
e g...@unixarea.de - w http://www.unixarea.de/


Spaces replaced by question marks?

2010-10-05 Thread Ed Blackman
In the past few weeks, I've been seeing lots of question marks in my 
mutt display, and finally took some time to track down what was 
happening so I could find out what's happening, and if I can get rid of 
them.


In the display, I'll see ? ? ? ? * Track your shipment.  If I pipe 
the part being displayed to od -a, I see runs of spaces where the 
display shows alternating question marks and spaces:

   0001160  sp   c   a   n   :  nl  sp  sp  sp  sp  sp  sp  sp  sp   *  sp
   0001200   T   r   a   c   k  sp   y   o   u   r  sp   s   h   i   p   m

Another example:
Suzie,
?
This email

od -a:
   000   S   u   z   i   e   ,  nl  sp  nl   T   h   i   s  sp

The messages I've seen affected have these MIME properties:
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

That makes the latter example understandable, if not desirable: having 
unquoted spaces at the end of the line is a violation of rule 3 of 
RFC2045, section 6.7: Octets with values of 9 and 32 MAY be 
represented as US-ASCII TAB (HT) and SPACE characters, respectively, but 
MUST NOT be so represented at the end of an encoded line.  Any TAB (HT) 
or SPACE characters on an encoded line MUST thus be followed on that 
line by a printable character.  But that rule concludes with 
Therefore, when decoding a Quoted-Printable body, any trailing white 
space on a line must be deleted, as it will necessarily have been added 
by intermediate transport agents.  I'd greatly prefer that to question 
marks.


I also don't see why leading runs of spaces, and runs of spaces in the 
middle of printable characters, also get the ?.


Mutt 1.5.20 (2009-06-14) on Ubuntu lucid, with Michael Elkin's lenient 
RFC2047 patch applied.  Other configuration details available on 
request.


Ed


signature.txt
Description: Digital signature


Re: Spaces replaced by question marks?

2010-10-05 Thread Derek Martin
On Tue, Oct 05, 2010 at 04:16:29PM -0400, Ed Blackman wrote:
 In the display, I'll see ? ? ? ? * Track your shipment.  If I pipe
 the part being displayed to od -a, I see runs of spaces where the
 display shows alternating question marks and spaces:

The -a transforms the data in the sense that it ignores the high order
bit of the character -- so what you're seeing may not actually be
what's in the data.  Try od -ba, and wherever you see spaces, see if
the octal byte matches the ASCII code for a space (040).  If they
don't, the problem is most likely that the e-mail contains
non-iso-8859-1 characters, and Mutt can't figure out how to display
them.

Failing that, there's a good chance that there's something wrong with
ME's patch, in which case you should post to the dev list, not here.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpVYvkzP3W8n.pgp
Description: PGP signature


Re: Spaces replaced by question marks?

2010-10-05 Thread Michael Elkins

On Tue, Oct 05, 2010 at 03:30:18PM -0500, Derek Martin wrote:

The -a transforms the data in the sense that it ignores the high order
bit of the character -- so what you're seeing may not actually be
what's in the data.  Try od -ba, and wherever you see spaces, see if
the octal byte matches the ASCII code for a space (040).  If they
don't, the problem is most likely that the e-mail contains
non-iso-8859-1 characters, and Mutt can't figure out how to display
them.


I agree with this assessment--most likely a conversion problem.


Failing that, there's a good chance that there's something wrong with
ME's patch, in which case you should post to the dev list, not here.


That patch only applies to decoding message headers, not to the bodies of 
messages.


Re: Spaces replaced by question marks?

2010-10-05 Thread Michael Elkins

On Tue, Oct 05, 2010 at 04:16:29PM -0400, Ed Blackman wrote:
In the display, I'll see ? ? ? ? * Track your shipment.  If I pipe 
the part being displayed to od -a, I see runs of spaces where the 
display shows alternating question marks and spaces:

  0001160  sp   c   a   n   :  nl  sp  sp  sp  sp  sp  sp  sp  sp   *  sp
  0001200   T   r   a   c   k  sp   y   o   u   r  sp   s   h   i   p   m

Another example:
Suzie,
?
This email

od -a:
  000   S   u   z   i   e   ,  nl  sp  nl   T   h   i   s  sp

The messages I've seen affected have these MIME properties:
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable


Need more information:

- what $LANG are you using?
- do you have iconv support enabled? (for conversion of iso-8859-1 to utf-8 if 
  using $LANG=*.utf-8)

- what is your $charset ?
- what is your $assumed_charset ?

That makes the latter example understandable, if not desirable: having 
unquoted spaces at the end of the line is a violation of rule 3 of 
RFC2045, section 6.7: Octets with values of 9 and 32 MAY be 
represented as US-ASCII TAB (HT) and SPACE characters, respectively, 
but MUST NOT be so represented at the end of an encoded line.  Any TAB 
(HT) or SPACE characters on an encoded line MUST thus be followed on 
that line by a printable character.  But that rule concludes with 
Therefore, when decoding a Quoted-Printable body, any trailing white 
space on a line must be deleted, as it will necessarily have been 
added by intermediate transport agents.  I'd greatly prefer that to 
question marks.


This most likely has nothing to do with quoted-printable encoding.  Mutt 
doesn't replace characters while decoding (it happens when displaying).


I also don't see why leading runs of spaces, and runs of spaces in the 
middle of printable characters, also get the ?.


Re: Spaces replaced by question marks?

2010-10-05 Thread Ed Blackman

On Tue, Oct 05, 2010 at 03:30:18PM -0500, Derek Martin wrote:

On Tue, Oct 05, 2010 at 04:16:29PM -0400, Ed Blackman wrote:

In the display, I'll see ? ? ? ? * Track your shipment.  If I pipe
the part being displayed to od -a, I see runs of spaces where the
display shows alternating question marks and spaces:


The -a transforms the data in the sense that it ignores the high order
bit of the character -- so what you're seeing may not actually be
what's in the data.  Try od -ba, and wherever you see spaces, see if
the octal byte matches the ASCII code for a space (040).  If they
don't, the problem is most likely that the e-mail contains
non-iso-8859-1 characters, and Mutt can't figure out how to display
them.


Ah!  I didn't know that.  And indeed, they aren't 040 spaces:
000 123 165 172 151 145 054 012 240 012 124 150 151 163 040 145 155
  S   u   z   i   e   ,  nl  sp  nl   T   h   i   s  sp   e   m

240 instead of 040.

Anything I can do?


Failing that, there's a good chance that there's something wrong with
ME's patch, in which case you should post to the dev list, not here.


I didn't think it would, since it only affected headers, but since it 
was non-standard for that version I thought I would mention it.


Ed


signature.txt
Description: Digital signature


Re: Spaces replaced by question marks?

2010-10-05 Thread Morris, Patrick

 On 10/5/2010 9:54 PM, Ed Blackman wrote:

Ah!  I didn't know that.  And indeed, they aren't 040 spaces:
000 123 165 172 151 145 054 012 240 012 124 150 151 163 040 145 155
  S   u   z   i   e   ,  nl  sp  nl   T   h   i   s  sp   e   m

240 instead of 040.


That should, indeed, be a non-breaking space (though I'm not shocked 
that a subject line with line breaks and non-breaking spaces in it would 
display strangely).  Sounds like a bug.