Re: Does spamc unwrap spam reports?

[RW]

On Tue, 3 Jan 2017 10:39:52 +0100
Lukas Erlacher wrote:

> On 12/28/2016 03:12 PM, RW wrote:
> >
> > It's done in spamd. Don't attempt to remove X-Spam-* headers
> > yourself or it wont attempt to remove the mime encapsulation.
> >  
> 
> I'd like to convince myself of that... I ran `sudo -u debian-spamd
> spamc -c < spamspam.eml` on a mail that has spamlevel 14.4 and is
> encapsulated in a spam report. It gave 2.7/7.0... which I suppose is
> ok because it's an assessment of the spamminess of the whole mail.
> But that doesn't convince me...
> 
> How do I convince myself that it'll actually use the text of the 
> original spam mail to update the bayesian db?
 
You could try what I just did.

Edit a spam report and put made up words at the beginning of each
section, train it as spam and then put the made-up words through 
spamassassin -D bayes.


printf "\n\n Lhjkl  Ohjkl  Ihjkl \n" | spamassassin -D bayes
...
dbg: bayes: token 'ihjkl' => 0.986543689320388

Ihjkl was in the correct mime section at training - in the body of the
embedded spam.

What I don't get though is why isn't there a case-sensitive token
"Ihjkl"?

Re: Detecting Valid Message Replies

[RW]

On Tue, 3 Jan 2017 17:30:39 +0100
Reindl Harald wrote:

> Am 03.01.2017 um 17:20 schrieb RW:
> > In practice all you need to do is use a different domain or
> > sub-domain in the  Message-ID, one that's never used in email
> > addresses. Unless you are a large service provider it's unlikely to
> > be abused, and you can turn-it off in seconds if it becomes a
> > problem.  
> 
> in practice you have no control about the message-id at all since
> it's generated by the MUA and *not* the server 

It depends on the circumstances. Lots of admins have control over
hostnames or can deploy pre-configured email software. There's
also webmail where you do have control.

Re: Detecting Valid Message Replies

[mattg]

On 1/3/2017 12:17 PM, Dave Funk wrote:
> That would make the wrong decision in the following scenario:

Well it's not my decision.  I was simply pointing out, as you are, that
merely keeping track of outgoing messages IDs, and blacklisting based on
unknown message IDs, is prone to problems.  You have provided another
example to this effect.  I'm sure there are others.

Re: Detecting Valid Message Replies

[Dave Funk]

On Tue, 3 Jan 2017, ma...@assembly.state.ny.us wrote:


On 1/3/2017 8:12 AM, Christoffer G. Thomsen wrote:

blacklist or increase score for mails that reply to unknown
message IDs.


Remember that someone out in the world might do a "Reply all" to a
message which was also Cc'd to one of your users.  This would show up as
an unknown message ID.  Of course,to remedy this, you could also keep
track of incoming message IDs.


That would make the wrong decision in the following scenario:

  A sends message to B
  B replies to A and also adds C to the "CC" list
  (as B thinks that C should be involved in the conversation)

In this case C would receive a "reply" to a message that she's never seen 
before, but is a legitimate communication.


This scenario may seem contrived but I've seen it happen around me with 
some regularity (both as a recipient & creator).


And then there's the case where somebody forwards to you a reply that they 
got so you get a message "Re: blah de blah (fwd)"


--
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: Detecting Valid Message Replies

[Ian Zimmerman]

On 2017-01-03 13:47, Antony Stone wrote:

> Given the increasing usage of Google-based business email services
> (and others, similar), wouldn't that tend to prevent you being able to
> manipulate the Message-ID header, because you are no longer in charge
> of the outbound server used by senders on your domain?

Most MUAs insert a Message-ID header by themselves, and the MTA doesn't
touch it.  That is definitely how it works here, with mutt and exim.  In
fact my Message-IDs are generated by a script I wrote to override the
mutt built-in ones.

Even many gmail patrons use an IMAP capable MUA and use gmail as just a
SMTP submission server.  It doesn't work perfectly due to quirks in the
gmail IMAP implementation, but it works.

-- 
Please *no* private Cc: on mailing lists and newsgroups
Personal signed mail: please _encrypt_ and sign
Don't clear-text sign: http://cr.yp.to/smtp/8bitmime.html

Re: Detecting Valid Message Replies

[RW]

On Tue, 3 Jan 2017 19:11:35 +0800
Rob Gunther wrote:

> The other day I was thinking it would be cool if you could detect
> legitimate replies to mail I send.
> 
> Everyone gets spam with subjects like this:
> 
> Re: some type of trash subject
> 
> Obviously those are not replies to a message you sent.
> 
> That got me thinking of ways to authenticate a legitimate reply to a
> message I sent.
> 
> I came up with a concept called 'Authenticated Reply Detection'.
> 
> It uses the Message-ID header to encode some information when a
> message goes out - no database or saving of Message-ID is required.
> 
> If/When a reply comes back the details from Message-ID come back in
> the 'In-Reply-To' header.  If validated the message is guaranteed to
> be a reply to a message I sent, and spamassassin could adjust scoring
> accordingly.


In practice all you need to do is use a different domain or sub-domain
in the  Message-ID, one that's never used in email addresses. Unless
you are a large service provider it's unlikely to be abused, and you
can turn-it off in seconds if it becomes a problem.

Re: Detecting Valid Message Replies

[mattg]

On 1/3/2017 8:12 AM, Christoffer G. Thomsen wrote:
> blacklist or increase score for mails that reply to unknown 
> message IDs.

Remember that someone out in the world might do a "Reply all" to a
message which was also Cc'd to one of your users.  This would show up as
an unknown message ID.  Of course,to remedy this, you could also keep
track of incoming message IDs.

R: Detecting Valid Message Replies

[Nicola Piazzi]

Please take a look at this plugin :
http://saplugin.16mb.com/


Nicola Piazzi
CED - Sistemi
COMET s.p.a.
Via Michelino, 105 - 40127 Bologna – Italia
Tel.  +39 051.6079.293
Cell. +39 328.21.73.470
Web: www.gruppocomet.it



-Messaggio originale-
Da: Christoffer G. Thomsen [mailto:li...@cgt.name] 
Inviato: martedì 3 gennaio 2017 14:12
A: users@spamassassin.apache.org
Oggetto: Re: Detecting Valid Message Replies

On 2017-01-03 12:11, Rob Gunther wrote:
> Has anyone ever seen anything like that before for SA?  Does it exist 
> already or any type of plugin etc?

It's not SpamAssassin, but Rspamd (https://rspamd.com/) has a plugin called 
'replies' that stores message IDs of outgoing mails in Redis. If an incoming 
mail is a reply to a known outgoing message ID, the plugin will whitelist the 
incoming mail. It could probably be adapted to be used in the opposite way too, 
to blacklist or increase score for mails that reply to unknown message IDs. 
Perhaps a similar approach could be used in a SA plugin. Of course this does 
require a database, but at least you don't have to modify the MTA's message IDs 
(or become the MTA).

Re: Detecting Valid Message Replies

[Christoffer G. Thomsen]

On 2017-01-03 12:11, Rob Gunther wrote:
> Has anyone ever seen anything like that before for SA?  Does it exist
> already or any type of plugin etc?

It's not SpamAssassin, but Rspamd (https://rspamd.com/) has a plugin
called 'replies' that stores message IDs of outgoing mails in Redis. If
an incoming mail is a reply to a known outgoing message ID, the plugin
will whitelist the incoming mail. It could probably be adapted to be
used in the opposite way too, to blacklist or increase score for mails
that reply to unknown message IDs. Perhaps a similar approach could be
used in a SA plugin. Of course this does require a database, but at
least you don't have to modify the MTA's message IDs (or become the MTA).

Re: Detecting Valid Message Replies

[Antony Stone]

On Tuesday 03 January 2017 at 12:11:35, Rob Gunther wrote:

> The other day I was thinking it would be cool if you could detect
> legitimate replies to mail I send.

> I came up with a concept called 'Authenticated Reply Detection'.
> 
> It uses the Message-ID header to encode some information when a message
> goes out - no database or saving of Message-ID is required.

That seems like a neat idea, however it depends on you being in charge of the 
original outbound mail server as well as the (possibly different) one 
processing inbound mail and checking there for spam.

Given the increasing usage of Google-based business email services (and 
others, similar), wouldn't that tend to prevent you being able to manipulate 
the Message-ID header, because you are no longer in charge of the outbound 
server used by senders on your domain?

> If/When a reply comes back the details from Message-ID come back in the
> 'In-Reply-To' header.  If validated the message is guaranteed to be a reply
> to a message I sent, and spamassassin could adjust scoring accordingly.

I like the concept; sounds like it could work well for "traditional" self-
hosted email systems.

Regards,


Antony.

-- 
A user interface is like a joke.
If you have to explain it, it didn't work.

   Please reply to the list;
 please *don't* CC me.

Re: Detecting Valid Message Replies

[Kevin A. McGrail]

On January 3, 2017 6:11:35 AM EST, Rob Gunther  wrote:
>The other day I was thinking it would be cool if you could detect
>legitimate replies to mail I send.
>
>Everyone gets spam with subjects like this:
>
>Re: some type of trash subject
>
>Obviously those are not replies to a message you sent.
>
>That got me thinking of ways to authenticate a legitimate reply to a
>message I sent.
>
>I came up with a concept called 'Authenticated Reply Detection'.
>
>It uses the Message-ID header to encode some information when a message
>goes out - no database or saving of Message-ID is required.
>
>If/When a reply comes back the details from Message-ID come back in the
>'In-Reply-To' header.  If validated the message is guaranteed to be a
>reply
>to a message I sent, and spamassassin could adjust scoring accordingly.
>
>Has anyone ever seen anything like that before for SA?  Does it exist
>already or any type of plugin etc?
>
>
>Regards,
>
>Rob

I have definitely seen solutions to do this.  Unsure if they were in mimedefang 
or SpamAssassin.   Your encoded header idea I don't remember.  Good idea so you 
don't need a redid etc.

I predict it would be a good indicator of hamminess.  The lack of the header or 
a wrong header might not be spam though.

And if someone's account is compromised and they reply to messages  I wouldn't 
white list based on this but that is true of any white list feature.
Regards,
KAM

Detecting Valid Message Replies

[Rob Gunther]

The other day I was thinking it would be cool if you could detect
legitimate replies to mail I send.

Everyone gets spam with subjects like this:

Re: some type of trash subject

Obviously those are not replies to a message you sent.

That got me thinking of ways to authenticate a legitimate reply to a
message I sent.

I came up with a concept called 'Authenticated Reply Detection'.

It uses the Message-ID header to encode some information when a message
goes out - no database or saving of Message-ID is required.

If/When a reply comes back the details from Message-ID come back in the
'In-Reply-To' header.  If validated the message is guaranteed to be a reply
to a message I sent, and spamassassin could adjust scoring accordingly.

Has anyone ever seen anything like that before for SA?  Does it exist
already or any type of plugin etc?


Regards,

Rob

Re: Does spamc unwrap spam reports?

[Lukas Erlacher]

On 12/28/2016 03:12 PM, RW wrote:


It's done in spamd. Don't attempt to remove X-Spam-* headers yourself
or it wont attempt to remove the mime encapsulation.



I'd like to convince myself of that... I ran `sudo -u debian-spamd spamc 
-c < spamspam.eml` on a mail that has spamlevel 14.4 and is encapsulated 
in a spam report. It gave 2.7/7.0... which I suppose is ok because it's 
an assessment of the spamminess of the whole mail. But that doesn't 
convince me...


How do I convince myself that it'll actually use the text of the 
original spam mail to update the bayesian db?


Best,
Luke



smime.p7s
Description: S/MIME Cryptographic Signature