[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Nicolas Dumazet nicd...@gmail.com added the comment: Hello folks. (stumbling on this bug with Python2.7 release, noting that a few Mercurial tests broke with 2.7) I have no problem whatsoever with the fix itself (you know emails better than me), but you broke backwards compatibility for email.generator.Generator here, without propagating the changes to previous versions. 1) It would have been nice to flag this somewhere in the release notes, and/or the documentation for email.generator.Generator. Finding the reason of the Mercurial failures was... a bit trickier than necessary :) 2) What durable solutions can you propose for those of us that have to support several Python versions and share a similar behaviour? In a perfect world, I would simply do this: class CompatibleHeader(email.Header.Header.__class__): Python2.7 introduces a backwards incompatible change (Python issue1974, r70772) in email.generaor.Generator code: pre-2.7 code passed continuation_ws='\t' to the Header constructor, and 2.7 removed this parameter. Default argument is continuation_ws=' ', which means that the behaviour is different in 2.7 and 2.7 We consider the 2.7 behaviour to be preferable, but need to have an unified behaviour for versions 2.4 to 2.7 def __init__(self, **kw): # override continuation_ws kw['continuation_ws'] = ' ' email.Header.Header.__init__(self, **kw) email.Header.Header.__class__ = CompatibleHeader and get over it, but you'll notice that Header is still an old-style class... so that's not possible. Any suggestions? Thanks! -- nosy: +djc, nicdumz ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Chris Withers ch...@simplistix.co.uk added the comment: Maybe an old-fashioned monkey-patch would be the way to go? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Nicolas Dumazet nicd...@gmail.com added the comment: Sure, where was my head. So, a simple patch like this one: _oldheaderinit = email.Header.Header.__init__ def _unifiedheaderinit(self, *args, **kw): # override continuation_ws kw['continuation_ws'] = ' ' _oldheaderinit(self, *args, **kw) email.Header.Header.__dict__['__init__'] = _unifiedheaderinit fixes the issue for us, and might be helpful to others. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Chris Withers ch...@simplistix.co.uk added the comment:
It's probably worth noting that changing:
from email.mime.text import MIMEText
m = MIMEText('foo')
m['subject']='something long'
...to:
from email.header import Header
m = MIMEText('foo')
m['subject']=Header('something long')
...will do folding without the \t problem, even in Python 2.6
I guess the moral of the story is that all headers should really be
header objects. I think Barry has some ideas on that ;-)
--
keywords: -patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue1974
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Chris Withers ch...@simplistix.co.uk added the comment: Barry and I talked about this and he's is now working on it. We're literally going to remove the \t folding substitution and have it do the default space folding substitution instead. -- resolution: - accepted versions: +Python 2.6, Python 2.7, Python 3.1 -Python 2.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Barry A. Warsaw ba...@python.org added the comment: So astoundingly, this change has deep implications. The upshot is that it's difficult to fix this so that headers look nice for e.g. Subject headers, but so that splitting and wrapping work as expected for e.g. machine readability of Received headers. After discussion with other sprinters, I'm committing a change to Python 2.7 to fix this, but I am not back porting the change to 2.6. I think we should Do It Right for Python 3.1 but this requires (IMO) API changes. r70772 -- status: open - closed versions: -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Changes by Martin von Gagern martin.vgag...@gmx.net: -- nosy: +gagern ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Chris Withers [EMAIL PROTECTED] added the comment: Ori, I do agree with both you and Barry but is there any chance someone could make the one-character change to make the /t a space so we can stop seeing weirdness in common mail clients? Perhaps a separate issue could be raised to refactor this all correctly? ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Ori Avtalion [EMAIL PROTECTED] added the comment:
I think there's been a little misinterpretation of the standard in the
comments above.
It's important to note that RFC 2822 basically defines folding as
adding a CRLF before an existing whitespace in the original message.
See http://tools.ietf.org/html/rfc2822#section-2.2.3
It does *not* allow prepending folded lines with extra characters that
were not in the original message such as '\t' or ' '.
This is exactly what _encode_chunks does in header.py:
joiner = NL + self._continuation_ws
(Note that the email package docs and Header docstring use the word
'prepend' which is reflects the error in the code).
With a correct implementation, why would I want to choice of which type
of character to line-break on when folding?
The whole notion of controlling the value of continuation_ws seems wrong.
However, changing the default continuation_ws to ' ', as the patch
suggests, will output syntactically correct headers in the majority of
cases (due to other bugs that remove trailing whitespace and merge
consecutive whitespace into one character).
All in all, I agree with the change of the default continuation_ws due
to its lucky side-effects, but as Barry hinted, the algorithm needs some
serious work to really output valid headers.
Some examples of the good and bad behaviors:
from email.Header import Header
l = ['[EMAIL PROTECTED]' % i for i in range(8)]
# this turns out fine
Header(' '.join(l), continuation_ws=' ').encode()
'[EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL
PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED]\n [EMAIL PROTECTED] [EMAIL PROTECTED]'
# This does not fold even though it should
Header('\t'.join(l), continuation_ws=' ').encode()
'[EMAIL PROTECTED]\t[EMAIL PROTECTED]\t[EMAIL PROTECTED]\t[EMAIL
PROTECTED]\t[EMAIL PROTECTED]\t[EMAIL PROTECTED]\t[EMAIL
PROTECTED]\t[EMAIL PROTECTED]'
# And here the 4-char whitespace is shrinked into one
Header(''.join(l), continuation_ws=' ').encode()
'[EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL PROTECTED] [EMAIL
PROTECTED] [EMAIL PROTECTED]
[EMAIL PROTECTED]\n [EMAIL PROTECTED] [EMAIL PROTECTED]'
--
nosy: +salty-horse
___
Python tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1974
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Barry A. Warsaw [EMAIL PROTECTED] added the comment: I'm pretty convinced that this stuff is broken and needs serious attention. Unfortunately, I won't have time to address the email package before 2.6 and 3.0 get released. My current work lives at bzr+ssh://[EMAIL PROTECTED]/python/users/barry/30email but it hasn't been updated for a while. I highly suggest taking this to the email-sig to see if anybody else has spare cycles to work in this for now. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Chris Withers [EMAIL PROTECTED] added the comment: Again, in total agreement with Andi *EXCEPT*: Please use ' ' instead of '\t' for the continuation character. It's the \t that gets mis-rendered by Outlook and Thunderbird (at the very least!) and since ' ' is also valid according to the RFC, we make life much easier for ourselves if we just stick to that... ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Andi Albrecht [EMAIL PROTECTED] added the comment: FWIW, I've uploaded a patch to codereview: http://codereview.appspot.com/2407 It uses a space character as Chris suggested. ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Chris Withers [EMAIL PROTECTED] added the comment: Andi, I'm in total agreement with you :-) (so if this bug could get fixed, both issues could get closed) ___ Python tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Steve Baker [EMAIL PROTECTED] added the comment: I'm also hitting this problem. I'm including a demo script. The minimal condition for me to reproduce this bug is that the subject contain at least 76 characters of any kind, followed by a space, followed by at least 3 characters of any kind. -- nosy: +splorgdar Added file: http://bugs.python.org/file10418/test __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Changes by Steve Baker [EMAIL PROTECTED]: Added file: http://bugs.python.org/file10419/test.py __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Changes by Steve Baker [EMAIL PROTECTED]: Removed file: http://bugs.python.org/file10418/test __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
Changes by Guido van Rossum: -- assignee: - barry nosy: +barry __ Tracker [EMAIL PROTECTED] http://bugs.python.org/issue1974 __ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1974] email.MIMEText.MIMEText.as_string incorrectly folding long subject header
New submission from Chris Withers:
Somewhere in email.MIMEText.MIMEText.as_string (I'm not sure where) long
subject headers are folded using a newline followed by a tab:
from email.MIMEText import MIMEText
m = MIMEText('foo')
m['Subject']='AA '*40
m.as_string()
'Content-Type: text/plain; charset=us-ascii\nMIME-Version:
1.0\nContent-Transfer-Encoding: 7bit\nSubject: AA AA AA AA AA AA AA AA
AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA\n\tAA AA AA AA AA AA AA AA
AA AA AA AA AA AA AA AA AA \n\nfoo'
While RFC 822 section 3.1.1 doesn't forbid this type of folding, the
current behaviour mis-renders in both Thunderbird and Outlook.
Messages generated by Thunderbird and Outlook represent the above
subject as follows:
'Content-Type: text/plain; charset=us-ascii\nMIME-Version:
1.0\nContent-Transfer-Encoding: 7bit\nSubject: AA AA AA AA AA AA AA AA
AA AA AA AA AA AA AA AA AA AA AA AA AA AA AA\n AA AA AA AA AA AA AA AA
AA AA AA AA AA AA AA AA AA \n\nfoo'
Could the email library be adjusted to do the same? Years of wondering
why mails re-sent by mailman, mails from any number of python web
systems and recent complaints from customers of mine of mis-rendered
emails could be solved by doing so.
--
components: Library (Lib)
messages: 61866
nosy: cjw296
severity: normal
status: open
title: email.MIMEText.MIMEText.as_string incorrectly folding long subject header
versions: Python 2.4
__
Tracker [EMAIL PROTECTED]
http://bugs.python.org/issue1974
__
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
