[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Jens Troeger]

Jens Troeger  added the comment:

Any updates on this? Looks like the proposed change has not been merged into 
mainstream yet?

I’m having problems with Google rejecting emails:

(555, b'5.5.2 Syntax error, goodbye. r10-v6sm7321838qtj.41 - gsmtp', '…')

and using IETF’s message linter (https://tools.ietf.org/tools/msglint/) I get 
the following:

ERROR: missing mandatory header 'date' lines 1-7
ERROR: missing mandatory header 'return-path' lines 1-7

amongst a few others.

--
nosy: +_savage

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Change by Eric Lafontaine :


--
pull_requests: +5028

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Roundup Robot]

Changes by Roundup Robot :


--
pull_requests: +2720

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

Yes.  There's a chance someone else will review it if you do that.  I'm still 
not likely to have time for a while, myself, but ping me again in a month if I 
haven't gotten to it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi All,

Should I try to make this a github PR instead to accelerate the review?

Regards,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

Changes by R. David Murray :


--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi,

Could someone put this ticket on "waiting for review"?

Regards,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

Here you go :).

Regards,
Eric Lafontaine

--
Added file: http://bugs.python.org/file46459/issue28879_v5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Resent-heuristic

--
Added file: http://bugs.python.org/file46460/Resent_heuristic.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi,

I've implemented the heuristic, but it's messy with the issue of this ticket. 

I'm going to do some clean-up and separate the issue from the heuristic and 
post them separated.

Date issue ;
Test Case
Documentation
Implementation

Heuristic of Resent ;
Test Case
Documentation
Implementation


Regards,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

I think this constitutes the heuristic I was talking about in that comment, 
that will get it right 99+% of the time.  Strict mode should raise an error, 
but strict is not the default in the email package.

I probably won't have time to do any review for a while yet, I'm afraid.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

I've received an answer from the IETF Pete Resnick.  The answer does say 
however that there is no guaranteed way of getting the right headers if the 
message doesn't respect the standard; 

"[...] 
In this case, if the trace fields were not present, you would not be able to 
clearly distinguish. Certainly you know which blocks the Resent-Date and 
Resent-From belong to, and in this case you know which blocks the 
Resent-Message-ID belong to (since there are two of them and only two Resent-* 
blocks), and you know which block the Resent-Bcc belongs to (because it comes 
between the first Resent-Date and Resent-Message-ID), but you can't tell which 
block the Resent-To: belongs to. And if the either Resent-Message-ID was 
missing, you would be unable to tell where the Resent-Bcc or Resent-Message-ID 
belongs. This is simply a weakness in the standard.

However, the trace fields should exist, and that should divide the Resent-* 
blocks. As it says in 3.6:

header fields SHOULD NOT be reordered when a message is transported
or transformed. More importantly, the trace header fields and resent
header fields MUST NOT be reordered, and SHOULD be kept in blocks
prepended to the message.

I hope that helps.


pr
-- 
Pete Resnick http://www.qualcomm.com/~presnick/
Qualcomm Technologies, Inc. - +1 (858)651-4478
"
__

In other words, if we were to take an e-mail that would have more than one 
Resent- headers, there should be traces in between the resent block.  If there 
are no traces for a Resent- block, and we can detect there are 2 blocks in what 
is supposed to be a block , we should raise an error.  

However, if we were to do the implementation and make it to detect "blocks" of 
resents (with the position of the resent for example).  We could use the first 
block as the one we should be using and see if there are 2 Resent-Dates or 2 
Resent-From in that block only.

What do you think David?  I would like to know :).

P.s. I can forward the e-mail to does who want to have it :).

Regards,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

The IETF didn't answer yet :(.  

I'll await your news regarding this patch ("issue_28879_V4.patch").

I would like to have feedback if I need to change something.

Thanks a lot in advance,
Eric Lafontaine

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

I'm happy to comment on issues as far as mentoring goes, and yes email is my 
primary area of responsibility for CPython.  However, I don't have much free 
time, so getting to reviews is proving to be problematic.  There's a chance a 
might have some time this weekend, but no promises :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

overkill file

--
Added file: http://bugs.python.org/file46002/issue_28879_python2_overkill.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi David, Henning,

Sorry for the delay.  Here is the patch for python 2... but I find that adding 
an "example" just for RFC 5322 is kind of over-kill...

So I've made both :) please choose which one you prefer and let me know.  I 
personnally prefer to use the non-overkill (which is only a comment and a line 
of code).

David, I know you might not like it, but could you be my mentor?  I feel you 
have an understanding of messaging applications :).  I know you don't have much 
time, but I feel you're a big participant and you could delegate some tasks on 
to me.  

Regards,
Eric Lafontaine

- Implement a patch for the code to add a missing "Date" field if it doesn't 
exist .   (in review)
- Modify the documentation at the SMTPLib for the send_message to mention that 
it add missing date using the email.utils.formatdate 
- Modify the comment of the send_message code to mention RFC 5322 in there 
(ideally with the section of the RFC).  ( in review)
- Modify the example of Python 2 to present a RFC 5322 examples ( in review)

With this, I've done all I could... What else can I start to do to get reviewed?

--
Added file: http://bugs.python.org/file46001/issue_28879_python2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

The documentation is all in the Doc subdirectory of the checkout.

So, this should teach me to rely on vague memory instead of reading the code.  
I've probably still missed something because I don't have enough time to read 
it properly.

Yes, you are correct, the Resent headers need to be added by the user.  So 
adding a Resent-Date header is the only question, and we can assume that it 
should go before the first Resent-X header we find.

Here is my reconstruction of why I made that restriction and comment:  There is 
no specified order to the Resent- block headers.  So how do we know where the 
first block ends and the next begins?  It will almost always be at the 
Resent-Date header...but we can't assume that that will *always* be true.

For sendmail/send_message, I'm not going to look at a youtube video, so you'll 
have to make your argument in text :)  But you may not want to bother, it is 
indeed too late: we can't change the signature for backward compatibility 
reasons.  In any case IMO the signature is optimal as it is: the only required 
argument is msg, and that, therefore, has to come first.  Otherwise the 
arguments are already in the same order as they are for sendmail.

I spent a lot of time reading RFCs while working on these libraries, but it has 
been a while since my head was buried in that space :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

@David, the "Resent-" block should be added by the user and he needs to know 
what he's doing.  Hell, I work with emails all the time and never knew about 
this.  The Worse part of it is that I have to communicate with the IETF just to 
know how we can do the rules... that doesn't mean another programmer didn't 
interpret it wrong...

@Henning, this is my first contribution :P  I've just been brain-washed by 
Raymond Hettinger and his beautifuls speechs.  I suggest (no shame plug) you 
look at the first 5-6 videos of the following link if you got free time.  
https://www.youtube.com/playlist?list=PLRVdut2KPAguz3xcd22i_o_onnmDKj3MA

I've received his "call to arms" ; 
https://www.youtube.com/watch?v=-TdrFjDJn5E&list=PLRVdut2KPAguz3xcd22i_o_onnmDKj3MA&index=16
 

David, I said the API was awkward because of what is mention in this latest 
link.  I really believe we ought to change it ASAP if it's possible at all and 
make it more like sendmail... that's only an opinion though.  
https://youtu.be/-TdrFjDJn5E?list=PLRVdut2KPAguz3xcd22i_o_onnmDKj3MA&t=4231

Finally, Could I know how to modify the Doc of the Python2 ? 
Regards,
Eric Lafontaine


- Implement a patch for the code to add a missing "Date" field if it doesn't 
exist .   (in review)
- Modify the documentation at the SMTPLib for the send_message to mention that 
it add missing date using the email.utils.formatdate (in review)
- Modify the comment of the send_message code to mention RFC 5322 in there 
(ideally with the section of the RFC).  (in review)
- Modify the example of Python 2 to present a RFC 5322 examples

--
Added file: http://bugs.python.org/file45969/issue_28879_V4.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

There is at least one open ticket (maybe two, I forget) about providing a way 
to insert headers at arbitrary locations in the header list.  Our trouble has 
been deciding on an API.  There is a mailing list (email-sig) where such things 
can be discussed (it has been idle for a while now...).

I think you are right about the 'heuristic'.  Thinking about it now I believe 
the problem was partly the lack of an API for inserting headers at the top: if 
there is only one resent block we are OK appending the address headers, but if 
there is more than one block then that would be wrong.  The other aspect, to 
which the comment is referring, is that if we are "augmenting" headers, how do 
we decide if we are augmenting an existing block, or creating a new one?  
Looking at it now with fresh eyes I think we could just decide on the rules and 
document them.  And we probably want an option that says "this is a resend, add 
all the Resent headers as a new block".  Or maybe that is enough by itself.  No 
heuristics, refuse to guess :)  Either the user adds all the headers 
themselves, or none of them, and we construct them in send_message.  (I haven't 
reviewed the RFC lately, so I'm not sure that covers all the use cases.)

I don't have a strong opinion on the ehlo_helo issue, but I'm not clear on why 
its presence causes any difficulty for subclassing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Henning von Bargen]

Henning von Bargen added the comment:

I think Eric's To-Do list sums it up pretty well.

I just wanted to say how much I appreciate the Python community. It seems that 
every little detail is very well-thought-out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

forgot to re-post the to-do, and to correct the documentation :
List of things to do :
- Implement a patch for the code to add a missing "Date" field if it doesn't 
exist .   (in review)
- Modify the documentation at the SMTPLib for the send_message to mention that 
it add missing date using the email.utils.formatdate 
- Modify the comment of the send_message code to mention RFC 5322 in there 
(ideally with the section of the RFC).  ( in review)
- Modify the example of Python 2 to present a RFC 5322 examples 

Feedback welcome as always!

--
Added file: http://bugs.python.org/file45959/issue_28879_V3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

this is not a short answer, like I hoped it would be ;).  But skip to the last 
section if you don't want to read it all.

the heuristic problem :

For the heuristic of the resent headers, it's clearly say in the RFC5322 that 
all resent block should be PREPENDED to messages.  So only the first ones you 
see should be handled.  email.message keeps the order of the headers while 
reading and "get" always takes the first one if it exist :).

from the RFC: 
   Each new set of resent fields is prepended to the
   message; that is, the most recent set of resent fields appears
   earlier in the message.  No other fields in the message are changed
   when resent fields are added.

For the behavior of the send_message, we used Resent-Date as it should always 
be present for EACH set of resent : 
   When resent fields are used, the "Resent-From:" and "Resent-Date:"
   fields MUST be sent.  The "Resent-Message-ID:" field SHOULD be sent.
   "Resent-Sender:" SHOULD NOT be used if "Resent-Sender:" would be
   identical to "Resent-From:".
   
It's fantastic that things work well like this for coders :).

The issues is with the msg object being passed...  It does retain the order, 
but doesn't prepend new-headers... it only append new headers (putting them at 
the bottom of the e-mail).

If someone wants to prepend headers (i.e. the "Resend-"), they will have to 
take the msg obj and do the addition manually; 
msg._headers.insert(0,msg.policy.header_store_parse(name, val)).

They already have to do it though as the as_string function will print them at 
the bottom anyway.  Changing the "__setitem__" of message change the behavior 
too much...  even though it would be the right thing to do... email.message 
read e-mail from top to bottom and "set" each line.  In other words, for order 
dependant headers like the Resent, we're screwed.

i.e. from the test case about the multiple resent, here is what is sent down 
the "data" smtp command :
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: f...@bar.com
To: John
CC: Sally, Fred
Bcc: John Root , "Dinsdale" 
Resent-Date: Thu, 1 Jan 1970 17:42:00 +
Resent-From: h...@grail.net
Resent-To: Martha , Jeff
Resent-Bcc: d...@losthope.net
Resent-Date: Thu, 2 Jan 1970 17:42:00 +
Resent-To: h...@grail.net
Resent-From: Martha , Jeff
Date: Sun, 18 Dec 2016 20:32:11 -

A test message


___

Finally, for the "ehlo_or_helo_if_needed", I thought about it and here's what I 
think and what I believe we should be doing (I'm not attached to the idea 
though);
The send_message is a higher level of thinking and should be able to be 
modified if someone wants to act differently by sub-classing the smtplib.SMTP 
class.  in other words, if that someone wants to modify the SMTP behavior, he 
could modify the sendmail function, which would also impact the send_message 
function.

This leads the developpers to have to use ehlo and prepare the connection 
for send_message, but doesn't enforce it :).  Which would also already be the 
case of doing it with sendmail and that instead of having to over-ride 2 
methods, only one is over-written.  

If this is not desired, than I suggest "protecting our reference" by using 
thunder sendmail instead.

I prefer to remove the ehlo from the send_message and allow "flexibility" on 
the send_message itself.  
___

The issue I have right now, is what David Mentionned about the heuristic... I 
should raise an error when 2 Resent-Date are found, but that will prevent me 
from using it for production ( as I have more than one resent field present 
sometime...).  This would also prevent people from using it and I find it sad.  
For now, i'm guessing that adding a keyword parameter "guess" would be the 
right things to do, but will let that be in an other ticket.  David, I find 
that somehow the RFC5322 isn't clear enough on the "Resent-" header order, so 
I've written to the IETF to have some explanation on the rules in edge cases.  
The heuristic is implemented (takes the first one it finds), but just not 
activated yet for multiple Resent-Date.

I'll re-iterate my need for someone to review the code though :).

Eric Lafontaine

--
Added file: http://bugs.python.org/file45958/issue_28879_V2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

The signature of send_message cannot be changed for backward compatibility 
reasons.  It's signature was not intended to be an extension of the sendmail 
signature.

send_message cannot accept a dictionary, as then it would have to know how to 
format that dictionary into an RFC5322 compliant text string.  Nor can a 
dictionary represent a collection of email headers accurate.  Representing the 
headers and doing the serialization is the email package's job.

It is true that the ehlo_or_helo_if_needed is technically redundant, however, 
what it means is that once you've called any smtplib command you know that has 
been done, even if the subcommand raises an error.  This can make debuging 
easier (consistency of state), at a trivial cost.

The "more than one Resent block" error is explained in the comments.  If you 
want to write the heuristics to remove it, go ahead :)

Looking at that code, I see that I misremembered what it was doing.  I thought 
it was adding Resent-Date if it was missing, but it is instead determining what 
form of address headers it needs to add if Reset-Date is present.  That doesn't 
change the analysis in this issue, though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all, I was looking at the code of the function and also noticed multiple 
"itching".

1- Why raise an error when 2 resent-date exist?  it may exist and completely 
legitimate as per RFC5322.  The code should always take the first one it sees 
as per the RFC.
2- Also, there was an "ehlo_or_helo_if_needed", but it's done in sendmail 
function so makes it always irrelevant.
3- The signature of the function is different from the one of sendmail 
function.  My understanding is that send_message is a replacement of sendmail 
with more feature (praticality) and should be simple to change by just swaping 
the 2 of them. 
4- This "send_message" shouldn't be dependant on the email.message.Message 
class.  It should allow a dictionary to be passed as well.  So I removed the 
"get_all" to be replaced with a "get" method (which align with what 
dictionaries do. 


I have addressed 1,2 and 4 in my patch (modified the test case as well) and my 
efforts to make this better and clearer, but number 3 seems to be something 
that may be a big change for clients... Should we handle it at all?

Please review and give me feedbacks please, I'm open to negative ones as well 
;).  I need to know if I'm doing something wrong and I don't know all the 
principle (as you may have noticed).  

I've ran the "python -m test" & "./python -m smtplib" command, but is there 
something else I should've done?

List of things to do :
- Implement a patch for the code to add a missing "Date" field if it doesn't 
exist .   (in review)
- Modify the documentation at the SMTPLib for the send_message to mention that 
it add missing date using the email.utils.formatdate
- Modify the comment of the send_message code to mention RFC 5322 in there 
(ideally with the section of the RFC).  ( in review)
- Modify the example of Python 2 to present a RFC 5322 examples

Thanks Henning von Bargen, David, maciej.  I appreciate your support.  

Regards,
Eric Lafontaine

--
keywords: +patch
Added file: http://bugs.python.org/file45929/issue_28879.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Henning von Bargen]

Henning von Bargen added the comment:

OK, I understand the arguments. 
If I understand correctly, this will be fixed in one way or another in Python 
3.6 or 3.7.
For Python 2.7, this will not be fixed (so I have to work around this somehow, 
should be quite easy).
I think that at least the documentation and the examples for smtplib should be 
updated.

The doc should make it clear that the message is passed "as-is" and it's the 
caller's reponsibility to create a valid message string.

The examples at https://docs.python.org/2/library/email-examples.html should 
note which header fields are required by RFC 5322 and provide an example.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

Sure the comments can be updated.  Some of them elsewhere have been already.

The reason for the email library to have a validation function is that it has 
an 'SMTP' policy that is designed to produce valid SMTP messages when the 
message is serialized.  (It also has an HTTP policy that is designed to produce 
valid HTTP header blocks...though I'm sure there are bugs there as well).  
These policies are relatively new, even newer than the send_message method of 
smtplib.

If this gets done it time it could be fixed in 3.5...Larry is planning the 
final non-security-fix 3.5 release some time after 3.6.0 final goes out the 
door, as is our tradition.  After that it could only go into what will very 
shortly be the maintenance release (3.6) and the next feature release (3.7).

To contribute, create a diff against the tip of the default branch and post it 
here.  (We will be switching to github "soon", but posting a patch here will 
always work).  See docs.python.org/devguide for more details on contributing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi all,

Thanks for the enlightment.  I never figured that there was a send_message 
function XD.  Never needed it and it's true that the example in the email 
library use sendmail and not send_message.
https://docs.python.org/2/library/smtplib.html#smtplib.SMTP.sendmail
https://docs.python.org/3.5/library/smtplib.html#smtplib.SMTP.send_message

This function is fairly recent (python 3.2) from what I see.

Reading the documentation of the python 3.5 send_message function :
"[...] If from_addr is None or to_addrs is None, send_message fills those 
arguments with addresses extracted from the headers of msg as specified in RFC 
5322: from_addr is set to the Sender field if it is present, and otherwise to 
the From field. to_addrs combines the values (if any) of the To, Cc, and Bcc 
fields from msg. "

As we're already using this function for convenience of the RFC 5322, then I 
agree to add it.  We should also modify the doc & comment inside the code to 
make it clear that date is added if absent and following RFC 5322. (I've looked 
at the source and the send_message only mention RFC2822 in the comments, no RFC 
5322).

Finally, why would we want to add a validate fonction to the email library?  
What would it do ? validate that we respect a certain RFC?  Who other than 
SMTPlib would use it?  I would like to understand the reasonning behind it.  

Again, all this are opinions to let the discussion continue :).

For now, what I see we need to do (this bullet point list is intended to be 
expanded with what you think we need to do):
- implement a patch for the code to add a missing "Date" field if it doesn't 
exist
- Modify the documentation at the SMTPLib for the send_message to mention that 
it add missing date using the email.utils.formatdate
- Modify the comment of the send_message code to mention RFC 5322 in there 
(ideally with the section of the RFC).
- Fix it on all Python3 versions?  It should have been supported since 3.2 
right?

As it's my first time trying to contribute... I still don't know how to do so...

Regards,
Eric Lafontaine
eric.lafontai...@gmail.com <= if you can help me outside of this discussion to 
contribute, it would be my pleasure.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Maciej Szulik]

Maciej Szulik added the comment:

I've chatted a bit with David about this feature. Here are some thoughts:
- check what SMTP standard says about some validation rules
- add validate method, probably into email package

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

The sendmail function will never modify the RFC822+ content.  send_message, 
however, already does several manipulations of the message headers to make 
sending email simpler.  Practicality (make it easy to send messages without 
knowing the details of the SMTP/RFC5322 rules) beats purity (an SMTP library 
should not modify the content of the DATA) in this case, especially since 
smtplib *does* provide the purity version if you prefer to work that way.

That is, 'sendmail' is pure SMTP, while send_message is a practical enhancement 
that provides additional services to the caller, and can (and does and probably 
should) do checks for RFC required headers.

If someone wants to do a purity refactor (to which I would not object, and 
would in fact encourage), the RFC5322-aware code could be factored out into one 
or more functions or Message object methods in the email library that smtplib 
would call from the send_message method.

My visualization is that the email library should allow you to construct a 
valid email in whatever order you want (adding the Date header late in the 
process, for example), but should support validating the email before it is 
sent.  One way to do this would be to have the SMTP policy do the unambiguous 
fixups such as date headers, and raising errors for the rest (probably only if 
the strict flag is on, at this point) when the message is serialized.

Note that we previously fixed send_message to add a Resent-Date header if there 
are Resent- headers and no Resent-Date, so the precedent is already set (that 
is, smptlib send_message is already "not pure" :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Maciej Szulik]

Maciej Szulik added the comment:

I tend to agree with Eric Lafontaine, looking at the quote Henning von Bargen 
posted the originator address field is also required, but yet we don't 
explicitly check its presence in the code, but rely on the SMTP server to error 
out.

--
nosy: +maciej.szulik

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[Eric Lafontaine]

Eric Lafontaine added the comment:

Hi,  Not sure this is where the comment goes...

I work with the smtplib and email libraries.  I understand Henning von Bargen 
when he say that we should have a way to support RFC 5322 without asking the 
user to understand how to support it.  The issue is that the SMTP protocol is 
NOT the protocol that format the e-mail.  SMTP is the protocol that identify 
the "from", the "to", start encryption and finally transfert the message.  The 
actual e-mail content is all passed inside the SMTP "DATA" Command.  I strongly 
believe that an email should not be modified by a SMTP library.

the discussion should be focused on trying to make it available to the user 
WITHOUT changing the current behavior of email.message class.  


In other words, I disagree to change the SMTPlib module and suggest that it's 
how you construct your email in the first place that should consider it;
class MessageRfc5322(email.message.Message):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
if self.get('Date', None) is None:
self.add_header('Date', email.utils.formatdate())

msg = email.message_from_string(string_message, MessageRfc5322)



But, that's my opinion as someone who uses the smtplib and email library but 
also need to support rfc822 clients...

--
nosy: +Eric Lafontaine

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28879] smtplib send_message should add Date header if it is missing, per RFC5322

[R. David Murray]

R. David Murray added the comment:

As I implied but did not say explicitly, this is the expected behavior of 
smtplib.  You are responsible for adding any headers to the message that you 
want smtplib to send.  In particular, the 'sendmail' method takes a string to 
send, and smtplib does not modify it except for cr/lf transformation.  smtplib 
does not itself understand RFC5322 message syntax.

We can, however, add a Date header in the new send_message method of the 
python3 email library, because it accepts a Message object, and smtplib can use 
the knowledge the Message object encapsulates to check for the Date header and 
add one if it is missing.  That's why I've left this issue open.  I have now 
adjusted versions accordingly (ie: this is not a bug in python2.7, it is an 
enhancement request for Python3).  Sorry I wasn't clear about this earlier.

Hmm.  Actually, we can argue that it is an RFC compliance issue, as you have 
suggested, and change it in 3.6 as well, since it isn't likely to break 
anyone's working code.  So I'll put 3.6 in the versions unless someone objects 
to that logic.  But even after this is changed in python3, the smtplib sendmail 
method will not add a Date header, only the send_message method.

--
keywords: +easy
title: smtplib RFC 5322 date header missing -> smtplib send_message should add 
Date header if it is missing, per RFC5322
type:  -> behavior
versions: +Python 3.6, Python 3.7 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com