Executive summary:

- There is a BOM in the X-Ham-Report header field.
- There is reason to believe that it, and not just any non-ASCII,
  triggered this rejection.
- Disabling the X-Ham-Report field (and possibly an X-Spam-Report
  field) seems to be the best option.

Christian via Mailman-Users writes in an earlier message:

 > Diagnostic-Code: smtp; 550 Headers contain illegal byte order mark (BOM)

and now:

 > Hello Mark Sapiro. On Thu, 3 Feb 2022 14:31:11 -0800, you wrote:
 > 
 > >>> X-Ham-Report: Spam detection software, running on the system
 > >>> "crift.digimouse.eu", has NOT identified this incoming email as
 > >>> spam.  The original message has been attached to this so you can view
 > >>> it or label similar future email.  If you have any questions, see
 > >>> root\@localhost for details. Content preview:  systemerweiterungen,
 > >>> benutzer, dein account, startobjekte: ist da noch was drin? Jean-Luc
 > >>> Aeby CH-4052 Basel > Am 03.02.2022 um 09:05 schrieb Max Röthlisberger

In the line above there is a SMALL LATIN LETTER O WITH UMLAUT (U+00F6)
which gets no complaint.

 > >>> Mus <mus...@gmx.net>: > > Guten Morgen zusammen > > Mein MacBook Pro,

In this line, immediately before "Guten Morgen" there is a ZERO-WIDTH
NO-BREAK SPACE (ZWNBSP, U+FEFF) aka "byte order mark" or BOM.  I'm
satisfied that the error above really is complaining about the ZWNBSP,
and not random non-ASCII.  I conclude that the spam milter used a
proper content transfer encoding for the X-Ham-Report header field.

ZWNBSP is now deprecated in favor of WORD JOINER (WJ, U+2060), but
conforming implementations should support both with identical
semantics, except as the first character where ZWNBSP has BOM
semantics and WJ is just a PITA.

 > >>> OS 10.11.6 sucht zu Hause nach einem Neustart 4 - 5 > mal im Heimnetz
 > >>> den ? [...]  Content analysis details:   (-0.0 points, 4.0 required)

 > > This is the only header in the message that looks suspicious. I 
 > > suspect the `?` characters are actually non-ascii characters in an 
 > > unencoded header and that's the problem. I think whatever is adding 

I suspect it's not unencoded, since it's very specific about the BOM,
and the BOM is not the first non-ASCII character in that field.  I
don't think this is a non-ASCII problem, I believe it's BOM-specific.

It appears to be the first character in body of the message quoted,
and ends up in the middle of the body of the message rejected.  I
guess the original source is a broken MUA that delegates editing the
body to an editor that prepends a BOM to all Unicode files (probably
including UTF-8, which is severely deprecated).  Then it copies that
file including BOM into the message after the CRLFCRLF that separates
the header from the body.

This really doesn't hurt anybody because of the way mail is parsed.
IMO the real culprit here is the excessively strict MTAs that are
apparently decoding that header field and examining it for merely
deprecated features of Unicode, and rejecting on that basis.  But
you're not going to get that fixed at other people's sites.

 > > this header (SpamExperts ?) is the root of the problem. If this can 
 > > be configured to not add that X-Ham-Report: header, that may solve 
 > > the issue.

 > I’ll contact the provider whether it is possible to switch off the
 > spam detection software for our lists.

You probably don't want to do that, though.  Even if you trust your
posters, there's no reason to suppose one couldn't get hacked.

 > > Or, you could patch 
 > > https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/Handlers/Cleanse.py#L62
 > >  
 > > and add
 > > ```
 > >         del msg['X-Ham-Report']
 > > ```
 > > to have Mailman remove it. That may help.

I recommend this instead.  I guess that in the case of spam there
might also be an X-Spam-Report header field.  Depending on under what
circumstances you block Spam, you may want to disable that as well.

Steve

------------------------------------------------------
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
    https://mail.python.org/archives/list/mailman-users@python.org/

Reply via email to