Re: [exim] spool format error (on some list messages)

2018-06-06 Thread Jeremy Harris via Exim-users
On 06/06/2018 09:00 PM, Heiko Schlittermann via Exim-users wrote:
> @Jeremy: Maybe we should announce that sa_exim will have
> some end-of-life in the near future?

I'm happy to add to the docs chapter that discusses the spool file
formats that they are specifically regarded as not being a stable
interface and liable to change from time to time.

Along with the obvious implication that we may break any program
(such as sa_exim) that tries to use them as such.


The supported interface to SA is described in
http://exim.org/exim-html-current/doc/html/spec_html/ch-content_scanning_at_acl_time.html


-- 
Cheers,
  Jeremy

-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/


Re: [exim] spool format error (on some list messages)

2018-06-06 Thread Heiko Schlittermann via Exim-users
exim-users--- via Exim-users  (Do 31 Mai 2018 21:52:51 
CEST):
..
> 
> >> 1fOL7J-0001BL-DC-H
> > …
> >> 031  X-Spam-Relay-Country: US US **
> >> 090  Subject: [tip:perf/urgent] perf tools: Fix perf.data format 
> >> description of
> >>  NRCPUS header
> >> 065  X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +)
> >> 066  X-SA-Exim-Scanned: Yes (on s-Mich Richter 
> > [ HERE THE BLANK LINE WAS EATEN, so Exim doesn't recognize
> >   this as the end of the header section of the message.
> >> 042  Acked-by: Andi Kleen 
> >> 044  Cc: Adrian Hunter 
> >> 036  Cc: David Ahern 
> >> 034  Cc: He Kuang 
> >> 053  Cc: Hendrik Brueckner 
> >> 038  Cc: Jin Yao 
> > …
> 
> You are right, the X-SA-Exim-Scanned header is truncated (after "on", I
> missed that before) it is set by sa-exim (code snipplet from sa-exim.c
> with line numbers):

Ah, that *I* didn't see, that there's a fragment of the header to be
added. Hm. The 's-' is part of the primary hostname?

> --cut
>   31 /* Exim includes */
>   32 #include "local_scan.h"
>   33 extern FILE   *smtp_out;   /* Exim's incoming SMTP
> output file */
>   34 extern int body_linecount; /* Line count in body */
>   35 extern uschar *primary_hostname;
> ...
> 1277 header_add(' ', "X-SA-Exim-Scanned: Yes (on %s)\n",
> primary_hostname);
> --cut

Ok, the spool wire format is off, you said. I'm not sure about the
mechanigs of sa_exim, that is, I do not have any clue *which* file it
sees and modifies. And/or if we built some optimisations which assume
that the spooled files (spooled in $spooldir/scan) are not altered.

For better theories about what's going on we need to know which files
sa_exim accesses.

If this is important and worth to be solved,, it would need some further
investigation.  

@Jeremy: Maybe we should announce that sa_exim will have
some end-of-life in the near future?

> All corrupted messages at least lack "primary-hostname" and the newline,
> some have other parts of the message in there. Any simple way to use a
> saved message to produce some more debugging information?

You can try to use something like

swaks --data ./saved-message -f … -t … --pipe 'exim -bh 1.1.1.1'

I'm not sure, if exim stops processing right before or right after the
local_scan() call.

As you do not want to test the ACL, exim -N could be your friend.

> achieve the sa-exim functionality (on the fly spamassassin scanning and
> greylisting depending on spamassassin scores)? Spamassassin integration
> via exiscan and greylisting as described in
> https://github.com/Exim/exim/wiki/SimpleGreylisting or greylistd? Any
> best practice on this topic? What I liked on sa-exim is, that there is
> no initial greylisting for unknown senders/hosts when they send mails
> with reasonable low spamassasin scores.

I do greylisting based on the announced content size. But your approach
might be good too.

I wrote some Perl function(s) to support greylisting in Exim, these
functions work reliable for years already. Tell me, if you're
interested, I"d update the docs and the scripts a bit and publish it.

(To be true, it is published already, but the docs are outdated.)

Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
-- 
 SCHLITTERMANN.de  internet & unix support -
 Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} -
 gnupg encrypted messages are welcome --- key ID: F69376CE -
 ! key id 7CBF764A and 972EAC9F are revoked since 2015-01  -


signature.asc
Description: PGP signature
-- 
## List details at https://lists.exim.org/mailman/listinfo/exim-users
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/