Nick Bright wrote:
On 7/23/2018 11:49 PM, Bill Cole wrote:
The goal is to get a copy of the message that is identical to what SA
saw when it arrived. For IMAP users, this is easiest to get with a
'missed spam' mailbox into which users can move messages for learning.
If you must rely on forwarded submissions, make sure users are
forwarding messages as attachments, and have the target deliver into a
mailbox that is processed to extract the 'message/rfc822' MIME
object(s) in those submissions and learn those, not the submission
mail itself.
Any specific utilities you could suggest?
I don't know about any standard utilities... but when I looked at this
some time ago I ended up writing my own.
As I posted upthread I posted the core code we use a while back, but
checking that archive link now it seems the canonical list archive is
not in the best shape and that link isn't very readable. I'll repaste
it here.
Locally, this is just a fragment in the mail delivery agent for our junk
mail reporting account, which in turn is derived from the more
generalized one used for all of our accounts. There's no reason you
couldn't call this from procmail or whatever your preferred delivery
agent is. Note you'll probably need to fill in a couple of loose ends
to make this work.
I think I've made further refinements for a couple of local edge cases
(eg, The Boss using (Al?)pine to [b]ounce (~"redirect") spam over once
in a while) but I'm pretty sure this is otherwise current.
-kgd
====
## waaaay up near the top of the script:
use MIME::Parser;
use MIME::Base64;
# Something(TM) needs to write the incoming message to a file at
# $tmp_file, or rearrange matters to tinker with the message directly
# in RAM.
# mail_deliver() is a local sub that writes the message to the
# appropriate account's mail folder as per the arguments. We use
# Maildir mailboxes, so:
# mail_deliver([maildir base path], [mode], [UID], [GID], [folder],
# [temporary working file])
# We keep the original "report" message in the reporting account's
# Inbox, and any RCF822 attachments get filed in the "reported" folder
# (thus the two calls to mail_deliver).
## begin new MIME::Parser deconstruction
my $parser = new MIME::Parser;
# apparently not in MIME::Parser 5.420, but it's in 5.427
# $parser->tmp_dir("$tmpdir/$time.$$.working");
# so we do this instead (eww)
umask 0077;
$parser->output_under($tmpdir);
# set some useful options
$parser->extract_nested_messages(0);
$parser->decode_bodies(0);
# sigh. be nice if the filenames could be arbitrarily *set* for
# the temp working copies... since not, we have to manually fiddle
# the data back and forth.
my $time = time;
my $partfile = "$tmpdir/$time.$$.part";
my $entity = $parser->parse_open($tmp_file);
foreach my $subent ($entity->parts()) {
my $type = $subent->mime_type;
if ($type =~ m|message/rfc822|i) {
# hack pthui. some idiot MUAs (or antispam plugins, more
likely) have started to
# base-64 the attached message, in its entirety. Which results
in the extracted
# message being just a blob of base64, unreadable by the mail
server or client.
# This is a major nuisance.
my $b64 = (
$subent->{mail_inet_head}->{mail_hdr_hash}->{'Content-Transfer-Encoding'} ?
${$subent->{mail_inet_head}->{mail_hdr_hash}->{'Content-Transfer-Encoding'}[0]}
=~ /: base64/
: 0);
if (my $fh = $subent->open("r")) {
open(FH, ">$partfile");
while (defined(my $line = $fh->getline)) {
if ($b64) {
print FH decode_base64($line);
} else {
print FH $line;
}
}
close(FH);
$fh->close;
mail_deliver("$settings{HD}/Maildir", 0660,
$settings{QT}{UID}, $settings{QT}{UID}, ".reported", $partfile);
}
}
}
# don't need to do this, mail_deliver does it for us...
# unlink $tmp;
mail_deliver("$settings{HD}/Maildir", '0600', $settings{QT}{UID},
$settings{QT}{UID}, 'INBOX', $tmp_file);
# clean up after the MIME::Parser object
$parser->filer->purge;
## done new MIME::Parser deconstruction
