> focus it on purely the strictly necessary parts to update to the new API … before dealing with potentially contentious things like EmailMultiAlternatives, MIMEBase, etc.
Appreciate the advice. I'll try to keep EmailMultiAlternatives and see whether that makes the new code hard to follow. Sadly, MIMEBase attachments are a documented part of Django's EmailMessage API (and have been necessary for certain kinds of attachments), so they'll have to be dealt with one way or another. - Mike On Thursday, June 27, 2024 at 2:14:12 AM UTC-7 Tom Carrick wrote: > I'm in favour of this change, and nice that you're thinking about the > future, but if you're going to write a ticket for this I would focus it on > purely the strictly necessary parts to update to the new API, and when that > is done, make more proposals to simplify the API as you suggest. > > I say this just with the goal of getting something that is easy to agree > on merged, before dealing with potentially contentious things > like EmailMultiAlternatives, MIMEBase, etc. > > Ignore me if this is irrelevant and those things are indeed necessary, my > knowledge on the topic is a bit lacking. > > Tom > > On Thu, 27 Jun 2024 at 03:28, Mike Edmunds <medm...@gmail.com> wrote: > >> Since the early feedback seems positive (though we're still waiting for >> more votes), here's some additional detail on the changes I think would be >> involved to update django.core.mail to use Python's modern email API. >> >> (See my earlier message >> <https://groups.google.com/g/django-developers/c/2zf9GQtjdIk/m/GWjimue9FAAJ> >> for >> background on Python's legacy vs. modern email APIs and why updating is >> useful.) >> >> Note: Django and Python both have classes named EmailMessage. I'm using >> "Django EmailMessage" to refer to django.core.mail.message.EmailMessage, >> and "Python EmailMessage" (or sometimes just "modern API") to refer to >> Python's modern email.message.EmailMessage. >> Necessary work >> <https://gist.github.com/medmunds/07720063fafef9e6980cc9fba2760986#necessary-work> >> >> 1. >> >> Update tests.mail.tests to use modern email APIs >> >> Where a test relies on legacy implementation details, try to rewrite >> it to be implementation agnostic if possible; otherwise try to retain the >> spirit of the test using modern APIs. >> >> Retain all security related tests, with updates as appropriate. (Even >> where we know Python's modern API handles the security for us, it doesn't >> hurt to double check.) >> >> Probably need to add some cases for existing behavior not currently >> covered by tests, particularly around attachments and alternatives. >> >> Remove a test case *only* if it's truly no longer relevant and can't >> be usefully updated. (And perhaps leave behind a brief comment explaining >> why.) >> >> (Legacy email APIs used in tests.mail.tests: email.charset, >> email.header.Header, MIMEText, parseaddr. Also message_from_… currently >> defaults to legacy policy.compat32.) >> 2. >> >> Update django.core.mail.message.EmailMessage to use modern email APIs >> >> Change Django's EmailMessage.message() to construct a modern >> email.message.EmailMessage rather than a SafeMIME object (which is based >> on >> legacy email.message.Message with policy=compat32). Add a >> message(policy=default) param forwarded to Python's EmailMessage >> constructor. >> >> Hoist alternative part handling from Django's EmailMultiAlternatives >> into Django's base EmailMessage to simplify the code. (More on this >> below.) >> >> In _create_alternatives(), use modern add_alternative() >> >> <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.add_alternative> >> and >> friends to replace legacy SafeMIME objects. >> >> In _create_attachments(), use modern add_attachment() >> >> <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.add_attachment>. >> >> Handle (legacy Python) MIMEBase objects as deprecated, and convert to >> modern equivalent. This is a relatively complex topic; I'll post a >> separate >> message about it later. (I also have some questions about how best to >> handle content-disposition "inline" and whether we want to somehow >> support >> multipart/related.) >> >> Remove the private Django EmailMessage methods >> _create_mime_attachment() and _create_attachment() (without deprecation). >> >> (Legacy APIs used in django.core.mail.message: email.message.Message, >> email.charset, email.encoders, email.header, email.mime, email.utils) >> 3. >> >> Deprecate unused internal APIs from django.core.mail.message >> >> Django will no longer need these (Python's modern email API covers >> their functionality), but they may be in use by third-party libraries: >> - utf8_charset >> - utf8_charset_qp >> - RFC5322_EMAIL_LINE_LENGTH_LIMIT >> - BadHeaderError^ (more details below) >> - ADDRESS_HEADERS >> - forbid_multi_line_headers()^ >> - sanitize_address() (more details below) >> - MIMEMixin >> - SafeMIMEMessage >> - SafeMIMEText^ >> - SafeMIMEMultipart^ >> >> (Items marked ^ are exposed in django.core.mail via __all__. I >> haven't looked into the reason for that.) >> 4. >> >> Update django.core.mail.__init__ to avoid EmailMultiAlternatives, and >> django.core.mail.backend.smtp to replace sanitize_address. (More details >> below.) >> 5. >> >> Update docs >> - deprecation of legacy MIMEBase in attachments list, what to do >> instead >> - eliminate EmailMessage/EmailMultiAlternatives distinction >> - deprecation of internal legacy items from #3 >> >> Tricky bits and things requiring longer discussion >> <https://gist.github.com/medmunds/07720063fafef9e6980cc9fba2760986#tricky-bits-and-things-requiring-longer-discussion> >> >> - >> >> *Legacy MIMEBase attachments:* I'll post a separate message about >> this sometime later. (Maybe next week. It's a long topic, plus I'm still >> experimenting.) >> - >> >> *EmailMultiAlternatives:* I'd like to get rid of the distinction >> between Django's EmailMessage and EmailMultiAlternatives, and just move >> the >> logic for alternative parts into base EmailMessage. (And then simplify >> the >> docs.) >> >> EmailMultiAlternatives is additional complexity for users, and it >> isn't really helpful with Python's modern add_alternative() >> >> <https://docs.python.org/3/library/email.message.html#email.message.EmailMessage.add_alternative> >> API. >> >> We could deprecate EmailMultiAlternatives, but that would be a very >> noisy deprecation. (Everything sending plaintext + html email today >> *has* to use EmailMultiAlternatives.) My preference would just be to >> leave a stub class with a doc string: >> class EmailMultiAlternatives(EmailMessage): >> # TODO: add doc string explaining what this used to be, >> # and suggesting just using EmailMessage instead. >> pass >> - >> >> *BadAddressHeader:* is a ValueError subclass raised only by >> forbid_multi_line_headers()—which would be deprecated. (The modern email >> API raises a ValueError for CR/NL in headers.) >> >> Do we want to issue a deprecation warning for this? The warning has >> to be on import, not on use. (There's a way to do that with a >> module-level >> __getattr()__, which I see we've done before in db.models.enums.) >> >> Otherwise I'm inclined to just write BadAddressHeader = ValueError in >> the deprecated code. (forbid_multi_line_headers() will warn about >> deprecation when it's called.) >> >> Same question about all the deprecated constants (like >> RFC5322_EMAIL_LINE_LENGTH_LIMIT). >> - >> >> *sanitize_address():* This function is called from two places in >> Django: >> - forbid_multi_line_headers() (which would be deprecated) >> - Django's *SMTP EmailBackend* to process the "envelope" from and >> recipient addresses (which may not be the same as the *From*, *To*, >> etc. message headers—e.g., bcc is only in envelope recipients, not >> message >> headers) >> >> sanitize_address() does a bunch of things: >> - Raises ValueError some forms of invalid addresses >> - Checks again for CR/NL, and raises ValueError (not >> BadAddressHeader) >> - Encodes/re-encodes non-ascii display names to ascii RFC 2047 >> (using a mix of legacy and modern email APIs that may be the source of >> some >> bugs) >> - Encodes/re-encodes non-ascii addr-specs to ascii RFC 2047 >> mailbox + punycode domain >> >> The modern email APIs handle all of this *for message headers.* (Except >> the punycode part, which is arguably wrong in message header fields.) >> >> However, I believe at least some of this is necessary for smtplib. In >> particular, the envelope from and recipient addresses have to be ascii >> only. (Unless SMTP.sendmail() is called with the "SMTPUTF8" option—which >> is >> not currently possible with Django's SMTP EmailBackend—and the SMTP >> server >> supports that option.) >> >> I'd suggest we deprecate sanitize_address(), and create a simpler >> private function in the SMTP EmailBackend to perform the SMTP-specific >> ascii conversion. >> - >> >> *Other stuff:* I haven't yet investigated what to do with: >> - Django EmailMessage.encoding and settings.DEFAULT_CHARSET >> - Django EmailMultiAlternatives.alternative_subtype (when >> overridden) >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django developers (Contributions to Django itself)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to django-develop...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/django-developers/0041c2f3-2ba4-4e76-b7df-b3dc025eb4ean%40googlegroups.com >> >> <https://groups.google.com/d/msgid/django-developers/0041c2f3-2ba4-4e76-b7df-b3dc025eb4ean%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/0056e25b-37b4-4d99-98fb-af23ecab6dfen%40googlegroups.com.
