No X-Spam- headers appearing

[Philip Colmer]

I've just installed SA 3.3.2 on an Ubuntu server to be used with Mailman,
using apt-get install spamassassin.

I've mostly followed the instructions in
http://www.jamesh.id.au/articles/mailman-spamassassin/ - I say mostly
because (a) it looks like James was using a different distro so the
configuration file is in a different place and (b) it looks like he was
using an older version because the configuration options have changed.

Anyhow, SA seems to be working nicely with Mailman, in that /var/log/syslog
is showing me things like:

Sep 26 12:57:43 ip-10-141-164-156 spamd[7659]: spamd: connection from
localhost [127.0.0.1] at port 58125
Sep 26 12:57:43 ip-10-141-164-156 spamd[7659]: spamd: using default config
for linaro-mm-sig: /var/lib/spamassassin/linaro-mm-sig.prefs/user_prefs
Sep 26 12:57:43 ip-10-141-164-156 spamd[7659]: spamd: checking message 
3535d648a078fd18d0cc6f13ea347...@rkmryshu.net for linaro-mm-sig:999
Sep 26 12:57:48 ip-10-141-164-156 spamd[7659]: spamd: identified spam
(10.7/5.0) for linaro-mm-sig:999 in 4.5 seconds, 11234 bytes.
Sep 26 12:57:48 ip-10-141-164-156 spamd[7659]: spamd: result: Y 10 -
DOS_OE_TO_MX,MIME_BASE64_BLANKS,NO_DNS_FOR_FROM,RCVD_IN_BRBL_LASTEXT,RCVD_IN_PBL,RCVD_IN_XBL,RDNS_NONE,WEIRD_QUOTING
scantime=4.5,size=11234,user=linaro-mm-sig,uid=999,required_score=5.0,rhost=localhost,raddr=127.0.0.1,rport=58125,mid=
3535d648a078fd18d0cc6f13ea347...@rkmryshu.net,autolearn=no

However, the emails are NOT getting spam headers inserted, whether they are
ham or spam.

According to
http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Conf.html:

Note that X-Spam-Checker-Version is not removable because the version
information is needed by mail administrators and developers to debug
problems. Without at least one header, it might not even be possible to
determine that SpamAssassin is running.

so I would, at the very least, expect X-Spam-Checker-Version to appear in
all emails. Furthermore, the documentation says:

Here are some examples (these are the defaults, note that Checker-Version
can not be changed or removed):

  add_header spam Flag _YESNOCAPS_
  add_header all Status _YESNO_, score=_SCORE_ required=_REQD_
tests=_TESTS_ autolearn=_AUTOLEARN_ version=_VERSION_
  add_header all Level _STARS(*)_
  add_header all Checker-Version SpamAssassin _VERSION_ (_SUBVERSION_) on
_HOSTNAME_

Even though the documentation says these are the defaults, I've added them
anyway to /etc/spamassassin/local.cf and restarted spamd, but the headers
still aren't being inserted.

One blog I found (
http://blog.dmitryleskov.com/small-hacks/forcing-spamassassin-to-add-the-x-spam-status-header-to-ham-for-debugging/)
suggests that these headers are actually added by amavisd ... which I don't
have installed. However, the blog posting does say:

In other words, in configurations where SpamAssasin is controlled by
amavisd-new, the X-Spam- headers are actually added by the latter, and it
is amavisd-new that decides whether to add them.

so, since SA is *not* being controlled by amavisd-new on my system, I don't
think this applies anyway.

The same blog posting suggests that the X-Spam headers will only appear on
messages that have some score and not pure ham messages, but I've checked a
message that got a score of 10.7 and there are no headers in it.

What am I misunderstanding or what have I overlooked?

Thanks.

Philip

Re: Why so much scanning?

[Matus UHLAR - fantomas]

On Tue, 2013-09-24 at 21:12 -0700, google_t...@curranfamilynet.net wrote:

I just installed 3.3.2 on Slackware 14.  I am seeing what looks like
duplicated effort in /var/log/mailog.  Here is the log for a single
email. You can see the same message ID is processed first by root, then
twice as user bobo.  Is this normal?  I had been using SA 3.2.5 on
the decommissioned server and i never saw this:


On 26.09.13 01:17, Karsten Bräckelmann wrote:

You will have to review the entire mail processing chain on that
machine. Here's what I can tell from your logs:


The first scan appears to be by a milter -- most likely some (changed)
default configuration of Sendmail on Slackware. It is obvious this first
scan is not intended by you (Bayes DB locking fails, and missing custom
required_score conf).

Disable the unwanted milter in your Sendmail config.


Oh, no. Milter is the best way to refuse clear spam at SMTP time.
just configure it so it can use the correct user names.
Maybe using -e or -x options for spamass-milter?
--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Spam = (S)tupid (P)eople's (A)dvertising (M)ethod

Re: No X-Spam- headers appearing

[Karsten Bräckelmann]

On Thu, 2013-09-26 at 14:11 +0100, Philip Colmer wrote:
 I've just installed SA 3.3.2 on an Ubuntu server to be used with
 Mailman, using apt-get install spamassassin.
 
 I've mostly followed the instructions in
 http://www.jamesh.id.au/articles/mailman-spamassassin/  [...]

 However, the emails are NOT getting spam headers inserted, whether
 they are ham or spam. 

That's due to the Mailman filter in above reference.

The Mailman filter uses the SYMBOLS spamd method, rather than PROCESS,
which makes spamd return the status, score and a list of rules hit only.
Unlike with PROCESS, the message itself is not returned, thus no X-Spam
headers either.

A closer look at the python code suggests, the filter even hardly cares
about the rules hit -- it just cares about the score to decide whether
to pass, moderate or discard the message. That decision is passed back
to the Mailman filter chain.


 What am I misunderstanding or what have I overlooked?

Generally, the client (a mailman filter in your case) passes the message
to spamd, which after processing returns the modified message. As you
pointed out correctly, this modification by default adds some X-Spam
headers, at the very least an X-Spam-Version header.

The docs you cited apply to SpamAssassin. The method the Mailman filter
uses is spamd specific, though, a daemon implementation using SA at its
core.


There's also a general misunderstanding here. While the client passes
the message (a copy of the message, rather), it is up to the client what
it does with the returned data -- including outright ignoring the result
with the X-Spam headers added, and proceeding with (a copy of) the
original message.

In order to have the added X-Spam headers show up later, the client has
to discard the original copy, and pass along the modified version as
received from the daemon.


If you want the X-Spam headers, you will need a different Mailman
filter / documentation to follow.


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: No X-Spam- headers appearing

[Philip Colmer]

Thanks, Karsten, for your explanation. That makes sense and I'll have to
see whether the lack of headers is going to cause problems going forwards
or if looking in syslog will suffice.

Regards

Philip



On 26 September 2013 16:33, Karsten Bräckelmann guent...@rudersport.dewrote:

 On Thu, 2013-09-26 at 14:11 +0100, Philip Colmer wrote:
  I've just installed SA 3.3.2 on an Ubuntu server to be used with
  Mailman, using apt-get install spamassassin.
 
  I've mostly followed the instructions in
  http://www.jamesh.id.au/articles/mailman-spamassassin/  [...]

  However, the emails are NOT getting spam headers inserted, whether
  they are ham or spam.

 That's due to the Mailman filter in above reference.

 The Mailman filter uses the SYMBOLS spamd method, rather than PROCESS,
 which makes spamd return the status, score and a list of rules hit only.
 Unlike with PROCESS, the message itself is not returned, thus no X-Spam
 headers either.

 A closer look at the python code suggests, the filter even hardly cares
 about the rules hit -- it just cares about the score to decide whether
 to pass, moderate or discard the message. That decision is passed back
 to the Mailman filter chain.


  What am I misunderstanding or what have I overlooked?

 Generally, the client (a mailman filter in your case) passes the message
 to spamd, which after processing returns the modified message. As you
 pointed out correctly, this modification by default adds some X-Spam
 headers, at the very least an X-Spam-Version header.

 The docs you cited apply to SpamAssassin. The method the Mailman filter
 uses is spamd specific, though, a daemon implementation using SA at its
 core.


 There's also a general misunderstanding here. While the client passes
 the message (a copy of the message, rather), it is up to the client what
 it does with the returned data -- including outright ignoring the result
 with the X-Spam headers added, and proceeding with (a copy of) the
 original message.

 In order to have the added X-Spam headers show up later, the client has
 to discard the original copy, and pass along the modified version as
 received from the daemon.


 If you want the X-Spam headers, you will need a different Mailman
 filter / documentation to follow.


 --
 char *t=\10pse\0r\0dtu\0.@ghno
 \x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
 main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8?
 c=1:
 (c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0;
 }}}

False positive?

[岩崎洋佑]

Hello,

Some e-mails sent from my account are recognized as spam mails.
Could anyone tell me the cause?

Below is the header information of one of those spam mails.


Return-Path: x...@example.co.jp
X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on
example.co.jp
X-Spam-Level: 
X-Spam-Status: Yes, score=17.0 required=13.0
tests=AISHOU,CONTENT_TYPE_PRESENT,

DAIHYOU,DEETO,DIRECTUNKNOWN,DIRECTVECTANTDYN,DYN_AISHOU,DYN_DAIHYOU,DYN_DEETO,

DYN_FUAN,DYN_ONEGAI,DYN_RENRAKU,DYN_SUPPORT,FUAN,ISO2022JP_BODY,MIMEQENC,

NO_RECEIVED,OBSCURED_EMAIL,ONEGAI,ONLY1HOPDIRECT,QENCPTR1,QENCPTR2,RDNS_NONE,
RENRAKU,SUPPORT,THREAD_INDEX,X_CHINESE_RELAY autolearn=spam 
version=3.2.4
X-Spam-Report:
* -0.1 CONTENT_TYPE_PRESENT exists:Content-Type
*  1.0 ONLY1HOPDIRECT ONLY1HOPDIRECT
*  1.5 DIRECTVECTANTDYN directly received spam from vectant.ne.jp
*  0.3 DIRECTUNKNOWN directly received spam from suspicious dynamic IP
*  0.3 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
*  0.1 OBSCURED_EMAIL BODY: Message seems to contain rot13ed address
*  0.1 X_CHINESE_RELAY RBL: Received via a relay in China
*  [202.215.74.215 listed in cn.rbl.cluecentral.net]
*  0.2 RENRAKU RAW: renraku
*  0.1 DEETO RAW: deeto
*  0.2 AISHOU RAW: aishou
*  0.2 DAIHYOU RAW: daihyou
* -0.1 ISO2022JP_BODY RAW: ISO-2022-JP message
*  0.2 FUAN RAW: fuan
*  0.1 SUPPORT RAW: sapo-to
*  0.2 ONEGAI RAW: onegai
*  0.2 MIMEQENC FULL: Quoted-Printable mime definition
*  0.2 QENCPTR2 FULL: Quoted-Printable mime pattern
*  0.2 QENCPTR1 FULL: Quoted-Printable mime pattern
*  0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS
* -0.0 NO_RECEIVED Informational: message has no Received headers
*  2.0 DYN_ONEGAI DYN_ONEGAI
*  2.0 DYN_DAIHYOU DYN_DAIHYOU
*  2.0 DYN_SUPPORT DYN_SUPPORT
*  1.0 DYN_DEETO DYN_DEETO
*  2.0 DYN_RENRAKU DYN_RENRAKU
*  1.0 DYN_AISHOU DYN_AISHOU
*  2.0 DYN_FUAN DYN_FUAN
X-Original-To: x...@example.co.jp
Delivered-To: x...@example.co.jp
From: =?iso-2022-jp?B?GyRCOWI6ZRsoQiAbJEJBRzBsTzobKEI=?=
x...@example.co.jp
To: =?iso-2022-jp?B?GyRCODshIUQ+OSgbKEI=?= x...@example.co.jp
Subject: ***SPAM*** =?iso-2022-jp?B?UkU6IBskQjg9PnUkTkpzOXAbKEI=?=
Thread-Topic: =?iso-2022-jp?B?GyRCOD0+dSROSnM5cBsoQg==?=
Thread-Index:
Ac6ukfV5fR3KhI+0TAqZtmZa+kTFyQAJu+MV78AAAV0bAAG7KlBgAAEKaQAABipeMAAMvYcAAOpnXYA=
Date: Wed, 25 Sep 2013 04:27:44 +
Message-ID: xxx
References: xxx
In-Reply-To: 003701ceb5fb$240fd840$6c2f88c0$@example.co.jp
Accept-Language: ja-JP, en-US
Content-Language: ja-JP
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-originating-ip: [202.215.74.215]
x-forefront-prvs: 098076C36C
x-forefront-antispam-report:
SFV:NSPM;SFS:(189002)(199002)(13464003)(51704005)(377454003)(76796001)(76576001)(76786001)(66066001)(74502001)(74662001)(74876001)(19300405004)(19273905006)(81342001)(80022001)(56816003)(83072001)(56776001)(77096001)(59766001)(81542001)(19580395003)(77982001)(54356001)(76482001)(74366001)(54316002)(83322001)(19580405001)(53806001)(63696002)(74482001)(31966008)(69226001)(47446002)(74316001)(79102001)(33646001)(4396001)(47736001)(47976001)(49866001)(50986001)(80976001)(81686001)(46102001)(65816001)(51856001)(15202345003)(74706001)(15975445006)(81816001)(24736002)(562404015)(579004)(559001)(569005);DIR:OUT;SFP:;SCL:1;SRVR:HKNPR03MB162;H:HKNPR03MB164.apcprd03.prod.outlook.com;CLIP:202.215.74.215;FPR:;RD:InfoNoRecords;MX:3;A:1;LANG:ja;
Content-Type: multipart/mixed;
boundary=_002_aa0baeb366ae4afcba76ab64e6268ce2HKNPR03MB164apcprd03pro_
MIME-Version: 1.0
X-OriginatorOrg: example1.co.jp
X-Spam-Prev-Subject: =?iso-2022-jp?B?UkU6IBskQjg9PnUkTkpzOXAbKEI=?=



Regards.

Iwasaki

-- 

-NEWS---
■□　　中小企業向けIT活用支援サービス　　　□■
■□■□ 「IT侍」をリリースいたしました □■□■　

中小企業のIT環境から日本を元気に！！

IT侍はお電話一本でIT専門技術者が御社に駆けつけ
オフィスのITに関するトラブルを迅速に解決します。

◇IT侍の詳細はこちら⇒　http://itsamurai.biz/
-
★
株式会社セラク
   ITインフラ事業部
   岩崎　洋佑(Yosuke Iwasaki)
   〒160-0023
   東京都新宿区西新宿7-5-25　西新宿木村屋ビル6F
   TEL：03-3227-7528　FAX：03-3227-2131
　 E-mail：iwas...@seraku.co.jp
   U R L ：http://www.seraku.co.jp
★

Re: False positive?

[Karsten Bräckelmann]

On Fri, 2013-09-27 at 10:47 +0900, 岩崎洋佑 wrote:
 Some e-mails sent from my account are recognized as spam mails.
 Could anyone tell me the cause?
 
 Below is the header information of one of those spam mails.

 Return-Path: x...@example.co.jp
 X-Spam-Flag: YES
 X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on example.co.jp

Is that domain munging consistent? Does the Return-Path domain equal the
domain in the X-Spam-Checker-Version header? In other words, is it
*your* domain's outgoing SMTP classifying the mail as spam?

Or is that a badly munged, external recipient domain and server?


 X-Spam-Level: 
 X-Spam-Status: Yes, score=17.0 required=13.0 
 tests=AISHOU,CONTENT_TYPE_PRESENT,
   
 DAIHYOU,DEETO,DIRECTUNKNOWN,DIRECTVECTANTDYN,DYN_AISHOU,DYN_DAIHYOU,DYN_DEETO,
   
 DYN_FUAN,DYN_ONEGAI,DYN_RENRAKU,DYN_SUPPORT,FUAN,ISO2022JP_BODY,MIMEQENC,
   
 NO_RECEIVED,OBSCURED_EMAIL,ONEGAI,ONLY1HOPDIRECT,QENCPTR1,QENCPTR2,RDNS_NONE,
   RENRAKU,SUPPORT,THREAD_INDEX,X_CHINESE_RELAY autolearn=spam 
 version=3.2.4
 X-Spam-Report:
   * -0.1 CONTENT_TYPE_PRESENT exists:Content-Type
   *  1.0 ONLY1HOPDIRECT ONLY1HOPDIRECT
   *  1.5 DIRECTVECTANTDYN directly received spam from vectant.ne.jp

Lots of low-ish scoring custom rules snipped.

   *  2.0 DYN_ONEGAI DYN_ONEGAI
   *  2.0 DYN_DAIHYOU DYN_DAIHYOU
   *  2.0 DYN_SUPPORT DYN_SUPPORT
   *  1.0 DYN_DEETO DYN_DEETO
   *  2.0 DYN_RENRAKU DYN_RENRAKU
   *  1.0 DYN_AISHOU DYN_AISHOU
   *  2.0 DYN_FUAN DYN_FUAN

These rules are almost exclusively custom, third-party rules defined by
whoever runs the SA instance. Thus, the system administrator of that
machine / domain is the one you need to contact.

Stock SA rules did not classify your mail spam. The custom rules did.


 X-Original-To: x...@example.co.jp
 Delivered-To: x...@example.co.jp

Either bad domain munging, or internal mail.

 X-OriginatorOrg: example1.co.jp
  ^
Well, probably bad domain munging...


-- 
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}