A concept of 'ongoing conversation' or 'replied to' is implemented as
a 'pen pals' feature in amavisd, when it is used in place of spamd
to call SpamAssassin. The idea is to automatically contribute some negative
spam score points to ongoing conversations - based on envelope sender
and recipients, as well as Message-ID and References or In-Reply-To
mail header fields.
Amavisd-new release notes:
new in amavisd-new-2.4.2, June 27, 2006
- new feature: "pen pals soft-whitelisting" lowers spam score of received
replies (or followup correspondence) to a message previously sent by a
local user to this address;
Pre-requisites:
* both the outgoing and the incoming mail must pass through amavisd
(although outgoing mail may have checks disabled or made more permissive
if desired);
* SQL logging must be enabled (@storage_sql_dsn) and records should
be kept for at least several days (some statistics (2006-11 update):
90% of replied mail (or followups) is sent within 2 weeks since
previous correspondence, 40% within 24 hours, 20% within 3 hours,
10% within 30 minutes, 5% within 12 minutes);
* @mynetworks and @local_domains_maps must reflect reality, allowing amavisd
to distinguish between outgoing, incoming and internal-to-internal mail;
* the information about client IP address must be available to amavisd,
i.e. Postfix XFORWARD protocol extension must be enabled, or AM.PDP+milter;
* configuration variable $penpals_bonus_score must be set to a positive
value (such as 1.0, increase to perhaps 5 or 8 after seeing that it works),
zero disables the feature and is a default;
* $sql_clause{'sel_penpals'} must contain a SELECT clause (which by
default it does, unless overridden by an old assignment to %sql_clause
in amavisd.conf);
* sender/recipient address pair must exactly match recipient/sender pair of
previous correspondence (except for allowed case-changes in domain part),
which means that care must be taken when canonical and/or virtual mapping
is performed by MTA (such as mapping between internal and external address
forms) - if external address forms of local addresses are to be seen by
a content filter then canonical mapping (int->ext) must be done *before*
filtering and virtual mapping (ext->int) *after*; alternatively, if
internal address forms are to be seen by a content filter, then canonical
mapping should be done after filtering, and virtual mapping before;
see README.postfix, section "TO DO 'VIRTUAL ALIAS' MAPPING AND OTHER
POSTFIX CLEANUP PROCESSING BEFORE OR AFTER CONTENT FILTERING?"
(P.S. later renamed to 'Advanced Postfix and amavisd-new configuration');
How it works:
* SQL logging stores records about all mail messages processed by amavisd,
their sender, recipients, delivery status, mail contents type (no changes
there, this feature was introduced with amavisd-new-2.3.0); for the
purpose of pen pals scheme only records with local-domain senders matter;
* when a message is received, an SQL lookup against an SQL logging database
is performed, looking for previous messages sent in reverse direction,
i.e. from a local user (which is now a recipient of the current mail)
to the address that is now the sender of the message being processed;
A SELECT clause in $sql_clause{'sel_penpals'} is used, which by default
only considers records of previous messages that were actually
delivered (not rejected, discarded or bounced), and were not infected.
SQL lookup returns a timestamp of the most recent such message (if any),
the difference (in seconds) between the current time and the timestamp
is an 'age' as used in the following formula;
* an exponential decay formula calculates score points to be deducted
from the SA score:
weight = 1 / 2^(age/penpals_halflife)
score_boost = -penpals_bonus_score * weight
i.e. penpals_bonus_score is multiplied by 1, 1/2, 1/4, 1/8, 1/16, ...
at age 0, 1*halflife, 2*halflife, 3*halflife, 4*halflife ...
weight is a continuous function of age (actually, in steps of one second);
[...] (more in release notes)
new in: amavisd-new-2.5.0, April 23, 2007
- formerly penpals could only match replies to previous outgoing mail
where envelope sender and recipient addresses are exactly reversed.
Now, in addition to this, penpals can also match replies which reference
previous outgoing mail by its 'Message-ID' (taking into account the
'References' or 'In-Reply-To' header fields), even if the envelope
sender address of the reply is null or does not match a recipient address
of a previous outgoing mail. This covers for incoming replies to mailing
list postings, incoming message disposition notifications (MDN, RFC 3798)
and incoming replies from alias or role addresses. A query on a
message-id is fast compared to matching on recipient id, and if it
succeeds, the later one is skipped.
Mark
