[Mailman-Developers] Re: Some Confusion over Archivers and data types
On 5/30/25 07:33, Thomas Ward via Mailman-Developers wrote:
Aha, you were correct! It's returning an `email.header.Header`
datatype, but it's also doing this for `msg.get('Subject', '(No
Subject)')` which looks like aberrant behavior since the same for
`msg['To']` returns a string!
I may have to raise that as a bug with Python! (Either that or Python
on the mailman system is just old enough that that was fixed later...
Python 3.10)
Whether `msg.get('header_name', 'default')` returns a string or an
email.header.Header object depends on the raw value and how it is parsed
into a msg object. It does not depend on the header_name.
In either case `str(msg.get('header_name', 'default'))` will return the
appropriate string.
--
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
___
Mailman-Developers mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/mailman-developers.python.org/
Mailman FAQ: https://wiki.list.org/x/AgA3
Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Some Confusion over Archivers and data types
Hi Thomas
First off, I was wrong about the type of header values as returned by
any of the accessors. They're just str (or list of str in the case of
`.get_all`). I guess Header is a write-only feature (necessary if you
want to use non-ASCII in the address headers), because it isn't useful
for parsing most headers.
You *can* get a non-str, namely `None`. But you say you tried
`msg.get('subject', '')`, which should return `''` in case of an
absent subject. So unless you've figured that out in the meantime,
you still have to deal with it. I would suggest printing or logging
`type(msg)` and `type(msg['subject']`.
Thomas Ward via Mailman-Developers writes:
> In later Python, there is a class of message called
> `email.message.EmailMessage` which has a few extra things than just
> `email.message.Message` (I think this is yielded by `email.parser`
> objects). By knowing that the class is essentially a subclass with
> some fixes of `email.message.Message` then the `iter_attachments()`
> approach I was using to iterate over attachments in the email
> message won't work because that is only part of
> `email.message.EmailMessage`.
If it isn't an `EmailMessage` (it probably isn't, everything in the
email package defaults to `Message` although there may be a better
policy than `Compat32`) you can probably do
```
def enhanceMessage(msg):
# check type here to avoid mayhem
msg.is_attachment = email.message.MIMEPart.is_attachment
msg._body_types = email.message.MIMEPart._body_types
msg.is_attachments = email.message.MIMEPart.is_attachments
```
(I didn't check the code terribly carefully, so if you get
AttributeErrors, just keep adding them according to the pattern
above.)
Or you could use something like
`msg = message_from_bytes(msg.as_bytes(), _class=EmailMessage)`,
although that's expensive.
> archiver function by Mailman helps. Because now I have to do the
> Old School(TM) way of iterating over `msg.walk()` and then
> determining if the message part has a content disposition and then
> confirm if it's an attachment or not and then have fun with it.
If for efficiency reasons you want to do it directly, the code in
/path/to/pypkgs/email/message.py, class MIMEPart is only about 30
lines including a couple of auxiliary methods and data. It should be
easy to adapt.
--
GNU Mailman consultant (installation, migration, customization)
Sirius Open Sourcehttps://www.siriusopensource.com/
Software systems consulting in Europe, North America, and Japan
___
Mailman-Developers mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/mailman-developers.python.org/
Mailman FAQ: https://wiki.list.org/x/AgA3
Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Some Confusion over Archivers and data types
Aha, you were correct! It's returning an `email.header.Header`
datatype, but it's also doing this for `msg.get('Subject', '(No
Subject)')` which looks like aberrant behavior since the same for
`msg['To']` returns a string!
I may have to raise that as a bug with Python! (Either that or Python
on the mailman system is just old enough that that was fixed later...
Python 3.10)
Either way, I'm starting to figure out what's busted here.
Thomas
On 2025-05-30 01:00, Stephen J. Turnbull wrote:
Thomas Ward via Mailman-Developers writes:
> So, I need some information (hint: type hinting in your examples,
> etc. would be wonderful):
Patches welcome ;-) However, normal type hints in the examples won't
help you here. We'd have to extract attribute references from inside
of function calls, stuff them into type-hinted variables, and use
those. Realistically, to find the type that matters to your problem
you either need to Use the Source, Luke, or RTFM.
> (1) What is the datatype of `msg` in the archive_message class?
Ask not what is the data type of `msg`. Ask instead what is the data
type of `msg['Subject']`. I expect that it's `email.header.Header`.
Try `TID_PATTERN.search(str(message_subject))[0]`.
___
Mailman-Developers mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/mailman-developers.python.org/
Mailman FAQ: https://wiki.list.org/x/AgA3
Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Some Confusion over Archivers and data types
Well, I noticed a second problem with my code.
On 2025-05-30 01:00, Stephen J. Turnbull wrote:
Thomas Ward via Mailman-Developers writes:
> (1) What is the datatype of `msg` in the archive_message class?
Ask not what is the data type of `msg`. Ask instead what is the data
type of `msg['Subject']`. I expect that it's `email.header.Header`.
Try `TID_PATTERN.search(str(message_subject))[0]`.
... and knowing the type that `msg` is out of the box is important to
that. In later Python, there is a class of message called
`email.message.EmailMessage` which has a few extra things than just
`email.message.Message` (I think this is yielded by `email.parser`
objects). By knowing that the class is essentially a subclass with some
fixes of `email.message.Message` then the `iter_attachments()` approach
I was using to iterate over attachments in the email message won't work
because that is only part of `email.message.EmailMessage`.
Which is where knowing the core datatype of `msg` passed into the
archiver function by Mailman helps. Because now I have to do the Old
School(TM) way of iterating over `msg.walk()` and then determining if
the message part has a content disposition and then confirm if it's an
attachment or not and then have fun with it.
This said, `msg.get('Subject', '')` also seemed to fail (but I usually
use that instead of referring to `msg['HEADER']` directly where possible)
I'll be testing this today, but knowing what the core datatype of `msg`
is will actually help with knowing which functions I need to call
against the `msg`. ;)
Thomas
___
Mailman-Developers mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/mailman-developers.python.org/
Mailman FAQ: https://wiki.list.org/x/AgA3
Security Policy: https://wiki.list.org/x/QIA9
[Mailman-Developers] Re: Some Confusion over Archivers and data types
On 5/29/25 09:49, Thomas Ward via Mailman-Developers wrote:
When we get to the TID_PATTERN.search(message_subject)[0] line though,
we get an error about it expecting a string or bytes like object.
So, I need some information (hint: type hinting in your examples, etc.
would be wonderful):
(1) What is the datatype of `msg` in the archive_message class? Is it
an email.message.EmailMessage or email.message.Message or some Mailman
datatype representation of a message?
It is a mailman.email.message.Message object which is a subclass of
email.message.Message that overrides the as_string() and as_bytes()
methods to work around a couple of bugs and adds a couple of methods.
See
https://gitlab.com/mailman/mailman/-/blob/master/src/mailman/email/message.py?ref_type=heads
(2) If the msg datatype is not of email.message.Message or
email.message.EmailMessage, how should we go about getting data and
headers *out* of the message for the process?
All the things you would use on an email.message.Message object should
work on a mailman.email.message.Message object.
I have no idea why
message_subject = msg.get('Subject', '(No Subject)')
would not set message_subject to a string.
Perhaps you can log `msg.as_string()` or other things to inspect the msg
object.
--
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
___
Mailman-Developers mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/mailman-developers.python.org/
Mailman FAQ: https://wiki.list.org/x/AgA3
Security Policy: https://wiki.list.org/x/QIA9
