I want to propose updating django.core.mail to replace use of Python's legacy email.message.Message (and other legacy email APIs) with email.message.EmailMessage (and other modern APIs).
If there's interest, I can put together a more detailed proposal and/or ticket, but was hoping to get some initial feedback first. (I searched for relevant discussions in django-developers and the issue tracker, and didn't find any. Apologies if this has come up before.) [Note: I maintain django-anymail <https://github.com/anymail/django-anymail>, which implements Django integration with several transactional email service providers.] *Background* Since Python ~3.6, Python's email package has included two largely separate implementations: - a "modern (unicode friendly) API" based on email.message.EmailMessage and email.policy.default - a "legacy API" providing compatibility with older Python versions, based on email.message.Message and email.policy.compat32 (See https://docs.python.org/3/library/email.html, especially toward the end.) django.core.mail currently uses the legacy API. *Why switch?* There are no plans to deprecate Python's legacy email API, and it's working, so why change Django? - *Fewer bugs:* The modern API fixes a *lot* of bugs in the legacy API. My understanding is legacy bugs will generally not be fixed. (And in fact, there are some cases where the legacy API deliberately replicates earlier buggy behavior.) Django #35497 <https://code.djangoproject.com/ticket/35497> is an example of a legacy bug (with a problematic proposed workaround) which is just fixed in the modern API. - *Simpler, safer code:* A substantial portion <https://github.com/django/django/blob/stable/5.1.x/django/core/mail/message.py#L32-L190> of django.core.mail's internals implements workarounds and security fixes for problems in the legacy API. This would be greatly simplified—maybe eliminated completely—using the modern API. Examples: the modern API prevents CR/NL injections in message headers. It serializes and folds address headers properly—even ones with unicode names. It enforces single-instance headers where appropriate. - *Easier to work with:* The modern API adds some nice conveniences for developers working in django.core.mail, third-party library developers, and (depending on what we choose to expose) users of Django's mail APIs. Examples: populating the "Date" header with a datetime or an address header with Address <https://docs.python.org/3/library/email.headerregistry.html#email.headerregistry.Address> objects—without needing intricate knowledge of email header formats. Using email.policy to generate a 7-bit clean serialization (without having to muck about with the MIME parts <https://github.com/anymail/django-anymail/blob/v11.0/anymail/backends/amazon_ses.py#L168-L179> ). *Concerns & risks* Compatibility and security, of course… - *Backwards compatibility (for API users):* django.core.mail largely insulates callers from the underlying Python email package. There are a few places where this leaks (e.g., attachments <https://docs.djangoproject.com/en/5.0/topics/email/#django.core.mail.EmailMessage:~:text=send()%20is%20called.-,attachments,-%3A%20A%20list%20of> allows legacy email MIMEBase <https://docs.python.org/3/library/email.mime.html> objects), but in general the switch should be transparent. (And I have some ideas for supporting the other cases.) - *Backwards compatibility (for third-party libraries):* Some libraries may use internals I'd propose removing (e.g., SafeMIME and friends); we'd handle this through deprecation. - *Backwards compatibility (bug-level):* There's probably some code out there that unintentionally depends on legacy email bugs (or the specific ways Django works around them). I don't have any examples, but I also don't have a good solution for when they surface. Plus, while Python's modern email API is pretty mature at this point, there are still new bugs being reported against it. Email is complicated. - *Security:* As noted above, the modern API should be more secure than the legacy one. But we also have a nice collection of email security tests—which *mostly* don't depend on internal implementation. We'd keep these. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f410ad79-a034-4275-88a7-22e7626c06fdn%40googlegroups.com.
