date:20091002


On fre 02 okt 2009 04:47:56 CEST, Steven W. Orr wrote

I have all my SA tables up and running using InnoDB and using the
above table definitions. I just have one question:

Will the cronjob that was described here earlier

#!/bin/sh
howfar='where lastupdate  date_sub(now(), interval 3 month)'
mysql -h localhost -u sa -pssaa spamassassin EOF
delete from awl $howfar ;
delete from bayes_seen $howfar ;
EOF

also clean up the bayes_token table, or is there another cron job I



should use for that?

And, why is bayes_token.atime int(11) instead of
timestamp NOT NULL default CURRENT_TIMESTAMP on update
?

Is this a part of the design or is it more efficient?


ups i missed to post my cron and expire optimize part :=)

save as maint_bayes.sql

# http://www200.pair.com/mecham/spam/debian-spamassassin-sql.html

USE spamassassin
DELETE FROM awl WHERE lastupdate = DATE_SUB(SYSDATE(), INTERVAL 6 MONTH);
DELETE FROM awl WHERE count = 1 AND lastupdate = DATE_SUB(SYSDATE(),  
INTERVAL 60 DAY);


# remove local posted awl scores
DELETE FROM awl WHERE ip = 'none';

# delete where totscore is lower then -300
# DELETE FROM awl WHERE totscore = -300;

# delete where count  300
# DELETE FROM awl WHERE count  300;

# delete here msgid generated by spamassassin that have not being seen  
last 3 month
DELETE FROM bayes_seen WHERE lastupdate = DATE_SUB(SYSDATE(),  
INTERVAL 30 DAY);


# index optimize on innodb
ALTER TABLE awl ENGINE=INNODB;
ALTER TABLE bayes_seen ENGINE=INNODB;
ALTER TABLE bayes_token ENGINE=INNODB;

# ixhash
# ALTER TABLE ixhash ENGINE=INNODB;

save as maint_amavisd.sql

USE amavisd
# index optimize on innodb
ALTER TABLE maddr ENGINE=INNODB;
ALTER TABLE msgrcpt ENGINE=INNODB;
ALTER TABLE msgs ENGINE=INNODB;
ALTER TABLE quarantine ENGINE=INNODB;

and finaly from cron hourly:

#!/bin/sh
cd /path/to/maintain-sql-dir/  mysql -u user -ppassword -B   
maint_amavisd.sql

cd /path/to/maintain-sql-dir/  mysql -u user -ppassword -B  maint_bayes.sql


works fine on my 3.2.5 install, without any tears

--
xpoint

Problems with whitelist_from_rcvd

2009-10-02 Thread Igor Bogomazov

Hi,

When I add the string like:
whitelist_from s...@domain.mail
it works OK.

But:
whitelist_from_rcvd s...@domain.mail prefix.domain.mail
doesn't work.

I've checked rDNS of the prefix.domain.mail with 'host' utility - it's
all right.

And the appropriate mail header seems to be correct:
Received: from prefix.domain.mail (unknown [12.12.12.12])

What's the matter?


signature.asc
Description: PGP signature

Re: Problems with whitelist_from_rcvd


On fre 02 okt 2009 10:34:55 CEST, Igor Bogomazov wrote

And the appropriate mail header seems to be correct:
Received: from prefix.domain.mail (unknown [12.12.12.12])
What's the matter?


unknown reverse dns is postfix answer for not found reverse dns, so  
host was in the test you did wrong


host 12.12.12.12 gives unknown
dig unknown gives 12.12.12.12 ?

prefix.domain.mail is the helo header

--
xpoint

Re: Problems with whitelist_from_rcvd

2009-10-02 Thread Jeff Mincy

   From: Igor Bogomazov b...@hl.ru
   Date: Fri, 2 Oct 2009 12:34:55 +0400

   When I add the string like:
   whitelist_from s...@domain.mail
   it works OK.

   But:
   whitelist_from_rcvd s...@domain.mail prefix.domain.mail
   doesn't work.

   I've checked rDNS of the prefix.domain.mail with 'host' utility - it's
   all right.

   And the appropriate mail header seems to be correct:
   Received: from prefix.domain.mail (unknown [12.12.12.12])

   What's the matter?

It is hard to say for sure without seeing actual received headers.

You need to use the last external relay used by the email.

From man Mail::SpamAssassin::Conf. 

   whitelist_from_rcvd ...

   This string is matched against the reverse DNS lookup used during
   the handover from the internet to your internal network's mail
   exchangers.  It can either be the full hostname, or the domain
   component of that hostname.  ...

The easiest way to figure out which one to use is to add a Relay
header using:
   add_header all Relay trusted=_RELAYSTRUSTED_, untrusted=_RELAYSUNTRUSTED_

Then get the RDNS from the first untrusted=[ip=... rdns=RDNS ...] relay.
If the RDNS is blank then the whitelist_from_rcvd won't work.

Your internal_networks and trusted_networks needs to be setup correctly.

-jeff

Re: Problems with whitelist_from_rcvd


On Fri, 2 Oct 2009, Igor Bogomazov wrote:


whitelist_from_rcvd s...@domain.mail prefix.domain.mail
doesn't work.

I've checked rDNS of the prefix.domain.mail with 'host' utility - it's 
all right.


You don't check rDNS using host, you check it using dig -x 
host.ip.addr.here



And the appropriate mail header seems to be correct:
Received: from prefix.domain.mail (unknown [12.12.12.12])

What's the matter?


The (unknown [12.12.12.12]) part shows that rDNS for that IP address is 
_not_ configured, or that DNS on your MTA cannot resolve it for some 
reason. The text after from is the string used in the client's HELO, 
which can be anything at all and cannot be trusted for authentication.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  You cannot bring about prosperity by discouraging thrift. You
  cannot help small men by tearing down big men. You cannot
  strengthen the weak by weakening the strong. You cannot lift the
  wage-earner by pulling down the wage-payer. You cannot help the
  poor man by destroying the rich. You cannot keep out of trouble by
  spending more than your income. You cannot further the brotherhood
  of man by inciting class hatred. You cannot establish security on
  borrowed money. You cannot build character and courage by taking
  away men's initiative and independence. You cannot help men
  permanently by doing for them what they could and should do for
  themselves.   -- William J. H. Boetcker
---
 Approximately 9081780 firearms legally purchased in the U.S. this year

Re: I am getting all external domain emails subject tagged as SpamSpam


On Thu, 1 Oct 2009, empiric wrote:


Oct  1 13:22:39 mail postfix/smtp[17579]: E0EAD19B349:
to=u...@example.com, relay=mail.example.com[10.65.200.72]:25, delay=7.1,
delays=0.09/0/0.01/7, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as
3DD1212B701)


None of that really logs useful information to troubleshoot this problem. 
You should try to see what the Subject: header is at each step of 
processing, including how it's coming into your MTA from outside.


Can you set up a sniffer on port 25 and send in a message from the 
Internet and see what the Subject: header says in the packet capture?


What programs is Amavis calling to process the message prior to SA?

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  You cannot bring about prosperity by discouraging thrift. You
  cannot help small men by tearing down big men. You cannot
  strengthen the weak by weakening the strong. You cannot lift the
  wage-earner by pulling down the wage-payer. You cannot help the
  poor man by destroying the rich. You cannot keep out of trouble by
  spending more than your income. You cannot further the brotherhood
  of man by inciting class hatred. You cannot establish security on
  borrowed money. You cannot build character and courage by taking
  away men's initiative and independence. You cannot help men
  permanently by doing for them what they could and should do for
  themselves.   -- William J. H. Boetcker
---
 Approximately 9081780 firearms legally purchased in the U.S. this year

Questions about SA

2009-10-02 Thread Jose Luis Marin Perez


I have some questions: 

 - How to calculate the amount of memory and CPU used by each process Spamd? 
 - Approximately 85% of spam are in Spanish, this can be a problem for 
SpamAssassin? 
 - Which tool can I use to get statistics of SpamAssassin, I am currently using 
the script sa-stats.pl.

Thanks

Jose Luis 
_
Connect to the next generation of MSN Messenger 
http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-ussource=wlmailtagline

Re: Problems with whitelist_from_rcvd

2009-10-02 Thread Kris Deugau


John Hardin wrote:
You don't check rDNS using host, you check it using dig -x 
host.ip.addr.here


Actually, unless your DNS configuration is doing something bizarre, they 
should give back the same basic info - dig is just a lot more verbose:


[kdeu...@turboprop ~]$ host 209.91.179.62
62.179.91.209.in-addr.arpa domain name pointer deepnet.cx.
[kdeu...@turboprop ~]$ dig -x 209.91.179.62

;  DiG 9.2.4  -x 209.91.179.62
;; global options:  printcmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: NOERROR, id: 62009
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;62.179.91.209.in-addr.arpa.IN  PTR

;; ANSWER SECTION:
62.179.91.209.in-addr.arpa. 892 IN  PTR deepnet.cx.

;; AUTHORITY SECTION:
179.91.209.in-addr.arpa. 890IN  NS  ns3.vianet.ca.
179.91.209.in-addr.arpa. 890IN  NS  ns4.vianet.ca.
179.91.209.in-addr.arpa. 890IN  NS  ns1.vianet.ca.
179.91.209.in-addr.arpa. 890IN  NS  ns2.vianet.ca.

;; ADDITIONAL SECTION:
ns1.vianet.ca.  22  IN  A   209.91.128.30
ns2.vianet.ca.  22  IN  A   204.187.89.10
ns3.vianet.ca.  22  IN  A   209.91.174.60
ns4.vianet.ca.  22  IN  A   204.187.88.5

;; Query time: 2 msec
;; SERVER: 209.91.179.154#53(209.91.179.154)
;; WHEN: Fri Oct  2 10:33:50 2009
;; MSG SIZE  rcvd: 213

[kdeu...@turboprop ~]$ dig -x 209.91.179.62 +short
deepnet.cx.
[kdeu...@turboprop ~]$

(The AUTHORITY and ADDITIONAL sections may not be returned depending 
on your DNS cache;  BIND returns the above, DJB's dnscache doesn't 
return either.)


-kgd, wearing his ISP DNS admin hat

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread RW

On Thu, 1 Oct 2009 18:54:40 -0600
LuKreme krem...@kreme.com wrote:

 On Oct 1, 2009, at 18:36, Karsten Bräckelmann
 guent...@rudersport.de wrote:
 
  Same for RCVD_IN_DNSWL. If it positively matches, it either it is
  correct, or wrong. A false positive is a match, that is wrong. No  
  matter
  the score you assign the test.
 
 Lke others havecsaid, you can make the words mean whatever you want.  
 However, if you want to be understood you need to speak the Lingua  
 Franca. If you choose to use a term differently than everyone else
 you WILL be misunderstood and corrected.

Except that so far the lunatics haven't taken-over the asylum and you
are in a 3 to 2 minority, so please don't claim to be speaking for
everyone. 

A false match on a test is a false-positive. It doesn't reverse for a
ham test, simply because you're more used to thinking about spam tests. 

Do you apply the same usage to anything else? For example, do you
reverse the meaning of off and on for air-conditioning to make it
consistent with heating, so on always mean make hotter?

Re: Problems with whitelist_from_rcvd

2009-10-02 Thread Bill Landry

John Hardin wrote:
 On Fri, 2 Oct 2009, Igor Bogomazov wrote:
 
 whitelist_from_rcvd s...@domain.mail prefix.domain.mail
 doesn't work.

 I've checked rDNS of the prefix.domain.mail with 'host' utility - it's
 all right.
 
 You don't check rDNS using host, you check it using dig -x
 host.ip.addr.here

Why not, they come up with the same thing?:

host 207.210.83.140
140.83.210.207.in-addr.arpa domain name pointer ga.impsec.org.

dig -x 207.210.83.140 +short
ga.impsec.org.

Bill

Re: Problems with whitelist_from_rcvd


On Fri, 2 Oct 2009, Kris Deugau wrote:


John Hardin wrote:

 You don't check rDNS using host, you check it using dig -x
 host.ip.addr.here


Actually, unless your DNS configuration is doing something bizarre, they 
should give back the same basic info - dig is just a lot more verbose:


-kgd, wearing his ISP DNS admin hat


...I stand corrected. Thanks.

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Gun Control laws cannot reduce violent crime, because gun control
  laws assume a violent criminal will obey the law.
---
 Approximately 9085920 firearms legally purchased in the U.S. this year

Re: Questions about SA


On Fri, 2 Oct 2009, Jose Luis Marin Perez wrote:


- Approximately 85% of spam are in Spanish, this can be a problem for
  SpamAssassin?


Possibly. Most of the default rules and most third-party rules are for 
English. This would tend to reduce your hit rate, but a properly-trained 
Bayes would help correct that.


I don't know if anybody is generating third-party rules for 
spanish-language spam...



- Which tool can I use to get statistics of SpamAssassin, I am currently
  using the script sa-stats.pl.


sa-stats.pl is a good tool to get your local rule performance.

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Gun Control laws cannot reduce violent crime, because gun control
  laws assume a violent criminal will obey the law.
---
 Approximately 9085920 firearms legally purchased in the U.S. this year

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread Charles Gregory


On Fri, 2 Oct 2009, RW wrote:

However, if you want to be understood you need to speak the Lingua
Franca. If you choose to use a term differently than everyone else
you WILL be misunderstood and corrected.


If everyone calls an apple an orange, then yeah, it's an orange.


A false match on a test is a false-positive. It doesn't reverse for a
ham test, simply because you're more used to thinking about spam tests.


The distinction is whether the 'false positive' refers to the overall 
scoring of the message (FP=ham flagged as spam) or an individual test 
(FP=test triggered incorrectly). I consider *both* usages correct in this 
group. And as I vaguely recall, the OP did use sufficient context for even 
a lame-brain like myself to realize he meant the latter.


The FP on the named rule had the potential to cause an FN.


Do you apply the same usage to anything else? For example, do you
reverse the meaning of off and on for air-conditioning to make it
consistent with heating, so on always mean make hotter?


Do you TURN UP or TURN DOWN your air-conditioning?
Depends on whether someone has a simple numerical control
or is adjusting a thermostat. Plus colloquial usage, of course. :)
But yeah, you hit pretty close with your analogy. Just chose
the wrong words. :)

- Charles

required_score keeps reverting to 5

2009-10-02 Thread Jefferson Davis



I have recently updated to 3.2.4 - for some reason my required_score keeps 
reverting to 5, basically ignoring or everriding the settings in local.cf.

The ruleset 10_default_prefs.cf has these settings, and this is where it 
appears to come from.  While I have commented out the offending line(s) in this 
file, my concern is that the next rule update will overwrite my changes and 
every time it updates I'll have to play whack-a-mole, re-editing the file.

I'm hoping I'm just doing this the wrong way and someone can enlighten me as to 
the correct method.

Thanks. 

-- 
Jefferson K Davis
Technology and Information Systems Manager
Standard School District
1200 North Chester Ave
Bakersfield, CA 93308
661.392.2110


This message was sent using IMP, the Internet Messaging Program.

Re: .cn Oddity

2009-10-02 Thread MySQL Student

Hi All,

Regarding the .cn oddity, I added these to my rules, and of about 79k
messages today so far, I have the following:

uri LOC_URI_CN  m;^https?://[^/?]+\.cn\b;
uri T_CN_8_URL  /[\/.]+\w{8}\.cn(?:$|\/|\?)/i

LOC_URI_CN: 2926
T_CN_8_URL: 1634

HTH,
Alex

Re: required_score keeps reverting to 5

2009-10-02 Thread Charles Gregory


On Fri, 2 Oct 2009, Jefferson Davis wrote:

I have recently updated to 3.2.4 - for some reason my required_score keeps
reverting to 5, basically ignoring or everriding the settings in local.cf.


Some Linux (presumed) disties have non-standard configuration 
directories - but when you manually upgrade, the path to it gets set

back to the package default. Check for existence of:
   /etc/spamassassin/
   /etc/mail/spamassassin/
...etc.

- C

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread Marc Perkel




Charles Gregory wrote:

On Fri, 2 Oct 2009, RW wrote:

However, if you want to be understood you need to speak the Lingua
Franca. If you choose to use a term differently than everyone else
you WILL be misunderstood and corrected.


If everyone calls an apple an orange, then yeah, it's an orange.


A false match on a test is a false-positive. It doesn't reverse for a
ham test, simply because you're more used to thinking about spam tests.


The distinction is whether the 'false positive' refers to the overall 
scoring of the message (FP=ham flagged as spam) or an individual test 
(FP=test triggered incorrectly). I consider *both* usages correct in 
this group. And as I vaguely recall, the OP did use sufficient context 
for even a lame-brain like myself to realize he meant the latter.


The FP on the named rule had the potential to cause an FN.


Do you apply the same usage to anything else? For example, do you
reverse the meaning of off and on for air-conditioning to make it
consistent with heating, so on always mean make hotter?


Do you TURN UP or TURN DOWN your air-conditioning?
Depends on whether someone has a simple numerical control
or is adjusting a thermostat. Plus colloquial usage, of course. :)
But yeah, you hit pretty close with your analogy. Just chose
the wrong words. :)

- Charles



Q. Do I make a left at the next intersection?
A. Right!

Daily statistics into email

 
Some just mentioned sa-stats.pl statistics, and I then wrote a script for me to 
post daily stats for me into email.

This is not nuclear science, but I still share it.

It is HTML formatted because I use Outlook Express to read mail, but it is easy 
to fix

The file is named so that it runs just before 00-logrotate in Debian Linux.

Non HTML version:

--- /etc/cron.daily/00a-sa-stats ---
#!/bin/sh

/usr/local/bin/sa-stats.pl



HTML version for GUI users

--- /etc/cron.daily/00a-sa-stats ---
#!/bin/sh

FILE=/tmp/stats.mail

echo To: ja...@wellington$FILE
echo Subject: SpamAssassin statistics$FILE
echo Content-Type: text/html; charset=\us-ascii\$FILE
echo$FILE
/usr/local/bin/sa-stats.pl | /usr/bin/txt2html $FILE

/usr/sbin/sendmail ja...@wellington  $FILE
rm $FILE



The HTML version requires txt2html program which is installable on Debian apt, 
and propably others too. If not, maybe cpan..

Sendmail command is available with sendmail and postfix emailers, dunno about 
others.

southwest airlines sends out their own phishing email

not to be outdone by hackers and thieves, phishing for PPI, southwest 
airlines is sending out their own DKIM signed, SPF PASSED, from their 
own servers, their very own phishing email. (didn't one of the major 
banks do something like this 3 years ago?)


all servers in the links are http (not https), and are on 
*.luv.southwest.com ip's.

http://luv.southwest.com/servlet/cc6?(and some number that i erased)
looks like ip is owned by 'Responsys'?

host luv.southwest.com
luv.southwest.com has address 12.130.131.30
luv.southwest.com mail is handled by 20 imh2.rsys4.net.
luv.southwest.com mail is handled by 10 imh.rsys4.net.
mirror# whois 12.130.131.30
ATT WorldNet Services ATT (NET-12-0-0-0-1)
 12.0.0.0 - 12.255.255.255
CERFnet ATTENS-SJC1-2 (NET-12-130-128-0-1)
 12.130.128.0 - 12.130.191.255
CI - Responsys SID-10369 ATTWH-12-130-131-0-24-0809094253 
(NET-12-130-131-0-1)

 12.130.131.0 - 12.130.131.255

I looked up numbers on their web site.

I called southwest.  they say the hold time is between 45 mins and 1 
hour and 6 mins.  (i wonder why).

I called responsys.  phone doesn't even ring (800-624-5356)

I won't post full body, because of all the web bugs in it it could lead 
to the account of the person who brought this to my attention, but for 
people I know, Imight share it.


content of the email is a typical phishing email:
does anyone know if TSA really wants the airlines to collect this 
information?

*

*Action Required: TSA Changes Require You To Update Your Account*

*

Dear Future victim of identify fraud[sic],

Southwest Airlines has been working in cooperation with the TSA to 
introduce Secure Flight, a federally mandated program designed to help 
enhance the security of domestic and international commercial air travel 
through the use of improved watch list* matching.




Southwest Airlines is therefore required to collect additional Secure 
Flight Passenger Data, which includes:


   * Your full name, exactly as it appears on the current (non-expired)
 government-issued photo ID that you will be traveling with
   * Date of birth
   * Gender
   * The TSA-issued Redress Number** (if applicable)


here are headers.
yep, dkim passed on my end (before I munged the headers)

From - Fri Oct  2 13:27:11 2009
X-Mozilla-Status: 0001
X-Mozilla-Status2: 
Received: from mx1.secnap.com.ionspam.net ([204.89.241.253]) by 
secnap3.secnap.com over TLS secured channel with Microsoft 
SMTPSVC(6.0.3790.3959);

Fri, 2 Oct 2009 13:27:05 -0400
Received: from localhost (mx1.secnap.com.ionspam.net [204.89.241.253])
   by mx1.secnap.com.ionspam.net (Postfix) with ESMTP id 936342B7C91
   for spamt...@secnap.net; Fri,  2 Oct 2009 13:27:05 -0400 (EDT)
Received: from omp.luv.southwest.com (omp.luv.southwest.com 
[12.130.137.222])

   by mx1.secnap.com.ionspam.net (Postfix) with ESMTP id BA8CE2B7C7B
   for spamt...@secnap.net; Fri,  2 Oct 2009 13:27:03 -0400 (EDT)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=southwest; 
d=luv.southwest.com;
h=MIME-Version:Content-Type:Content-Transfer-Encoding:Date:From:Reply-To:Subject:List-Unsubscribe:To:Message-Id; 
i=rapidrewa...@luv.southwest.com;

bh=K9LTM4P8WM/e8CFLBk2b3E5eKKA=;
b=CovqQo71dauGXRfa0/e/1yqWPkjJhNrrGITrt34DKCk2SfX8zTrbtcDFdmNabtnIAPvTbF982oUe
  
VhYLXdl5uN7qDddhsDZ4Y2l7qa/4li0RXSWQIMPt8zCPCTL/2a1zMH7MsAOtGaucHkxhiHQMZwT9

  +rfozAHcpB98YHsdDLE=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=southwest; 
d=luv.southwest.com;

b=c4Y0HLpkWe1F5sC9DHPIDTgks95ippZeicmDIahk5M9ci+xT7iQUnzHqUncH6+Agtjf13Gwh8bKz
  
h65VN0uzG/HChchBerQpH/3JrhkCzlkyyHJfnONEPc8njpeGDg/5BYqbASDCnzKHxs8WvCIlMcI9

  EqpTLSW7ZdrNYvrx3mE=;
Received: by omp.luv.southwest.com (PowerMTA(TM) v3.5r10) id 
hoorue0morc3 for scheid...@secnap.net; Fri, 2 Oct 2009 10:27:02 -0700 
(envelope-from rapidrewa...@luv.southwest.com)

MIME-Version: 1.0
Content-Type: text/html;
   charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Date: Fri, 2 Oct 2009 10:27:01 -0700
From: Southwest Airlines Rapid Rewards rapidrewa...@luv.southwest.com
Reply-To: Southwest Airlines Rapid Rewards re...@luv.southwest.com
Subject: Important Notice: TSA Secure Flight
List-Unsubscribe: 
http://luv.southwest.com?lPHpkDCABDVTElJoLpKLssFlLJgHiDgLmEa

Return-Path: rapidrewa...@luv.southwest.com
X-OriginalArrivalTime: 02 Oct 2009 17:27:05.0688 (UTC) 
FILETIME=[8FDDF580:01CA4385]

--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
 *| *SECNAP Network Security Corporation

   * Certified SNORT Integrator
   * 2008-9 Hot Company Award Winner, World Executive Alliance
   * Five-Star Partner Program 2009, VARBusiness
   * Best Anti-Spam Product 2008, Network Products Guide
   * King of Spam Filters, SC Magazine 2008

_
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com

Re: Problems with whitelist_from_rcvd


On Fri, 2 Oct 2009, Bill Landry wrote:


John Hardin wrote:

On Fri, 2 Oct 2009, Igor Bogomazov wrote:


I've checked rDNS of the prefix.domain.mail with 'host' utility - it's
all right.


You don't check rDNS using host, you check it using dig -x
host.ip.addr.here


Why not, they come up with the same thing?:


I apologize; I don't use host and I looked at the man page for an explicit 
reverse option and didn't see one, and jumped to the wrong conclusion.


Igor, can you show us how you used host and what it output?

--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Gun Control laws cannot reduce violent crime, because gun control
  laws assume a violent criminal will obey the law.
---
 Approximately 9088680 firearms legally purchased in the U.S. this year

Re: Daily statistics into email

2009-10-02 Thread Warren Togami


http://ruleqa.spamassassin.org/
If you are capable of processing your mail nightly in cron, why don't 
you join the nightly mass check?  You can help to test the rules and 
make the sa-update channel better.  We especially need non-English ham 
in the nightly masscheck.


http://wiki.apache.org/spamassassin/NightlyMassCheck
Here's HOWTO.  The documentation is a bit confusing.  I'm working on a 
much simpler version of this.


What distro do you use?

Warren Togami
wtog...@redhat.com

Re: Daily statistics into email

On Fri, 2009-10-02 at 20:45 +0300, Jari Fredriksson wrote:

 Sendmail command is available with sendmail and postfix emailers,
 dunno about others.

You don't need to use sendmail: if the cron job writes anything to
stdout (or stderr) this is automatically mailed to root. 

If you'd rather that mail sent to root comes to you instead, just add a
redirection line to /etc/aliases. Don't forget to regenerate the aliases
database by running 'aliases' or your redirection won't take effect.


Martin

Re: Daily statistics into email

 On Fri, 2009-10-02 at 20:45 +0300, Jari Fredriksson wrote:
 
 Sendmail command is available with sendmail and postfix
 emailers, dunno about others.
 
 You don't need to use sendmail: if the cron job writes
 anything to stdout (or stderr) this is automatically
 mailed to root. 
 
 If you'd rather that mail sent to root comes to you
 instead, just add a redirection line to /etc/aliases.
 Don't forget to regenerate the aliases database by
 running 'aliases' or your redirection won't take effect. 
 
 

That HTML version needs to add a header for Content-Type. That is not possible 
by just echoing somehing, as those go automatically to the body.

The non-html version uses cron's default behaviour, but the html version must 
use sendmail.

Re: Daily statistics into email

 http://ruleqa.spamassassin.org/
 If you are capable of processing your mail nightly in
 cron, why don't you join the nightly mass check?  You can
 help to test the rules and make the sa-update channel
 better.  We especially need non-English ham in the
 nightly masscheck. 
 
 http://wiki.apache.org/spamassassin/NightlyMassCheck
 Here's HOWTO.  The documentation is a bit confusing.  I'm
 working on a much simpler version of this.
 
 What distro do you use?
 
 Warren Togami
 wtog...@redhat.com

I'm using Debian Lenny. This really interests me as a SpamFighter! Going into 
the links now.

Re: Daily statistics into email

 http://ruleqa.spamassassin.org/
 If you are capable of processing your mail nightly in
 cron, why don't you join the nightly mass check?  You can
 help to test the rules and make the sa-update channel
 better.  We especially need non-English ham in the
 nightly masscheck.
 
 http://wiki.apache.org/spamassassin/NightlyMassCheck
 Here's HOWTO.  The documentation is a bit confusing.  I'm
 working on a much simpler version of this.
 
 What distro do you use?
 
 Warren Togami
 wtog...@redhat.com
 
 I'm using Debian Lenny. This really interests me as a
 SpamFighter! Going into the links now. 

A second thought.. My hardware does not allow extra work. I currently have two 
hosts with SpamAssassin in my network, both of them use only one child. There 
is a 3rd host for Amavisd-new, and it also has only one thread for 
virusscanners.

I get about 1-15000 spam per month, and this setup barely manages it 
without great delays. I don't think I can run those tests with these machines.

I dream about a multicore server, but currently I'm laid off from my day work 
and wondering where to get bread and beer from.

I will join later when that is possible, but not now.

Re: southwest airlines sends out their own phishing email

2009-10-02 Thread Steven W. Orr

On 10/02/09 13:52, quoth Michael Scheidell:
 not to be outdone by hackers and thieves, phishing for PPI, southwest 
 airlines is sending out their own DKIM signed, SPF PASSED, from their own
 servers, their very own phishing email. (didn't one of the major banks do
 something like this 3 years ago?)

I have no idea what the story is here but from what you say here, it's not
clear whether responsys is a legitimate marketing company that was hired by
southwest.

For example:


southwest.com.  900 IN  A   208.94.153.100

but the MX for southwest is

southwest.com.  900 IN  MX  10 mail-1.southwest.com.
southwest.com.  900 IN  MX  10 mail-2.southwest.com.

Then look at luv.southwest.com which has

luv.southwest.com.  90  IN  A   12.130.131.30
but also has a reverse dns
30.131.130.12.in-addr.arpa. 3600 IN PTR luv.southwest.com.

Then the MX for luv says:
luv.southwest.com.  90  IN  MX  20 imh2.rsys4.net.
luv.southwest.com.  90  IN  MX  10 imh.rsys4.net.

which also happens to be ns1.responsys.net

Assuming responsys *is* legit, they could do a better job of reputation
management.

 
 all servers in the links are http (not https), and are on 
 *.luv.southwest.com ip's. http://luv.southwest.com/servlet/cc6?(and some
 number that i erased) looks like ip is owned by 'Responsys'?
 
 host luv.southwest.com luv.southwest.com has address 12.130.131.30 
 luv.southwest.com mail is handled by 20 imh2.rsys4.net. luv.southwest.com
 mail is handled by 10 imh.rsys4.net. mirror# whois 12.130.131.30 ATT
 WorldNet Services ATT (NET-12-0-0-0-1) 12.0.0.0 - 12.255.255.255 CERFnet
 ATTENS-SJC1-2 (NET-12-130-128-0-1) 12.130.128.0 - 12.130.191.255 CI -
 Responsys SID-10369 ATTWH-12-130-131-0-24-0809094253 (NET-12-130-131-0-1) 
 12.130.131.0 - 12.130.131.255
 
 I looked up numbers on their web site.
 
 I called southwest.  they say the hold time is between 45 mins and 1 hour
 and 6 mins.  (i wonder why). I called responsys.  phone doesn't even ring
 (800-624-5356)
 
 I won't post full body, because of all the web bugs in it it could lead to
 the account of the person who brought this to my attention, but for people
 I know, Imight share it.
 
 content of the email is a typical phishing email: does anyone know if TSA
 really wants the airlines to collect this information? *
 
 *Action Required: TSA Changes Require You To Update Your Account*
 
 *
 
 Dear Future victim of identify fraud[sic],
 
 Southwest Airlines has been working in cooperation with the TSA to 
 introduce Secure Flight, a federally mandated program designed to help 
 enhance the security of domestic and international commercial air travel 
 through the use of improved watch list* matching.
 
 
 
 Southwest Airlines is therefore required to collect additional Secure 
 Flight Passenger Data, which includes:
 
 * Your full name, exactly as it appears on the current (non-expired) 
 government-issued photo ID that you will be traveling with * Date of birth 
 * Gender * The TSA-issued Redress Number** (if applicable)
 
 
 here are headers. yep, dkim passed on my end (before I munged the headers)
 
 From - Fri Oct  2 13:27:11 2009 X-Mozilla-Status: 0001 X-Mozilla-Status2:
  Received: from mx1.secnap.com.ionspam.net ([204.89.241.253]) by 
 secnap3.secnap.com over TLS secured channel with Microsoft 
 SMTPSVC(6.0.3790.3959); Fri, 2 Oct 2009 13:27:05 -0400 Received: from
 localhost (mx1.secnap.com.ionspam.net [204.89.241.253]) by
 mx1.secnap.com.ionspam.net (Postfix) with ESMTP id 936342B7C91 for
 spamt...@secnap.net; Fri,  2 Oct 2009 13:27:05 -0400 (EDT) Received: from
 omp.luv.southwest.com (omp.luv.southwest.com [12.130.137.222]) by
 mx1.secnap.com.ionspam.net (Postfix) with ESMTP id BA8CE2B7C7B for
 spamt...@secnap.net; Fri,  2 Oct 2009 13:27:03 -0400 (EDT) 
 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=southwest; 
 d=luv.southwest.com; 
 h=MIME-Version:Content-Type:Content-Transfer-Encoding:Date:From:Reply-To:Subject:List-Unsubscribe:To:Message-Id;
  i=rapidrewa...@luv.southwest.com; bh=K9LTM4P8WM/e8CFLBk2b3E5eKKA=; 
 b=CovqQo71dauGXRfa0/e/1yqWPkjJhNrrGITrt34DKCk2SfX8zTrbtcDFdmNabtnIAPvTbF982oUe
 
 
 
 VhYLXdl5uN7qDddhsDZ4Y2l7qa/4li0RXSWQIMPt8zCPCTL/2a1zMH7MsAOtGaucHkxhiHQMZwT9
 
 
 +rfozAHcpB98YHsdDLE= DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns;
 s=southwest; d=luv.southwest.com; 
 b=c4Y0HLpkWe1F5sC9DHPIDTgks95ippZeicmDIahk5M9ci+xT7iQUnzHqUncH6+Agtjf13Gwh8bKz
 
 
 
 h65VN0uzG/HChchBerQpH/3JrhkCzlkyyHJfnONEPc8njpeGDg/5BYqbASDCnzKHxs8WvCIlMcI9
 
 
 EqpTLSW7ZdrNYvrx3mE=; Received: by omp.luv.southwest.com (PowerMTA(TM)
 v3.5r10) id hoorue0morc3 for scheid...@secnap.net; Fri, 2 Oct 2009
 10:27:02 -0700 (envelope-from rapidrewa...@luv.southwest.com) 
 MIME-Version: 1.0 Content-Type: text/html; charset=UTF-8 
 Content-Transfer-Encoding: quoted-printable Date: Fri, 2 Oct 2009 10:27:01
 -0700 From: Southwest Airlines Rapid Rewards

Re: southwest airlines sends out their own phishing email




Steven W. Orr wrote:

On 10/02/09 13:52, quoth Michael Scheidell:
  
not to be outdone by hackers and thieves, phishing for PPI, southwest 
airlines is sending out their own DKIM signed, SPF PASSED, from their own

servers, their very own phishing email. (didn't one of the major banks do
something like this 3 years ago?)



I have no idea what the story is here but from what you say here, it's not
clear whether responsys is a legitimate marketing company that was hired by
southwest.

For example:
Then look at luv.southwest.com which has

  

but, southwest would need to subdeligate luv.southwest.com.

it REALLY looks like someone at southwest had this done.

its stupid.. it encourages users to disclose private data over an 
insecure channel, and whoever authorized this (if its southwest) needs a 
LONG vacation.


oh, and I checked our managed email servers?  HUNDREDS AND HUNDREDS of 
these emails are coming in to all our clients.
many to email addresses that no longer exist, but 99% to current, legit 
emails.


other more interesting thing:  the frequent flyer number?  its real, and 
it belongs to the recipients.


so, is this a phishing email I need to block? or legit email I need to 
whitelist?


southwest's phone has a 1 hour hold time.

imagine that.



  

--
Michael Scheidell, CTO
Phone: 561-999-5000, x 1259
 *| *SECNAP Network Security Corporation

   * Certified SNORT Integrator
   * 2008-9 Hot Company Award Winner, World Executive Alliance
   * Five-Star Partner Program 2009, VARBusiness
   * Best Anti-Spam Product 2008, Network Products Guide
   * King of Spam Filters, SC Magazine 2008


_
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com

_

Re: Daily statistics into email

On Fri, 2009-10-02 at 21:33 +0300, Jari Fredriksson wrote:
  On Fri, 2009-10-02 at 20:45 +0300, Jari Fredriksson wrote:
  
  Sendmail command is available with sendmail and postfix
  emailers, dunno about others.
  
  You don't need to use sendmail: if the cron job writes
  anything to stdout (or stderr) this is automatically
  mailed to root. 
  
  If you'd rather that mail sent to root comes to you
  instead, just add a redirection line to /etc/aliases.
  Don't forget to regenerate the aliases database by
  running 'aliases' or your redirection won't take effect. 
  
  
 
 That HTML version needs to add a header for Content-Type. That is not 
 possible by just echoing somehing, as those go automatically to the body.
 
 The non-html version uses cron's default behaviour, but the html version must 
 use sendmail.
 
As crond must also use sendmail to ship any text that is left for it to
deal with and you've already inserted the MIME header, it seems to me
that the HTML processing would happen anyway, regardless of whether the
call to sendmail was implicit or explicit. Hence I assumed that you'd
just sendmail to avoid using the aliases system. What did I miss?


Martin

Re: southwest airlines sends out their own phishing email

On Fri, 2009-10-02 at 13:52 -0400, Michael Scheidell wrote:
 not to be outdone by hackers and thieves, phishing for PPI, southwest 
 airlines is sending out their own DKIM signed, SPF PASSED, from their 
 own servers, their very own phishing email. (didn't one of the major 
 banks do something like this 3 years ago?)
 
I reckon its a scam.

Here's why:

$ host luv.southwest.com
luv.southwest.com has address 12.130.131.30
luv.southwest.com mail is handled by 10 imh.rsys4.net.
luv.southwest.com mail is handled by 20 imh2.rsys4.net.

BUT
===
$ host southwest.com
southwest.com has address 208.94.153.100
southwest.com has address 208.94.152.100
southwest.com mail is handled by 10 mail-1.southwest.com.
southwest.com mail is handled by 10 mail-2.southwest.com.

$ host www.southwest.com
www.southwest.com has address 208.94.152.100
www.southwest.com has address 208.94.153.100

$ dig southwest.com ANY

;  DiG 9.5.1-P3-RedHat-9.5.1-3.P3.fc10  southwest.com ANY
;; global options:  printcmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: NOERROR, id: 11302
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 2, ADDITIONAL: 4

;; QUESTION SECTION:
;southwest.com. IN  ANY

;; ANSWER SECTION:
southwest.com.  805 IN  MX  10 mail-2.southwest.com.
southwest.com.  805 IN  MX  10 mail-1.southwest.com.
southwest.com.  805 IN  A   208.94.152.100
southwest.com.  805 IN  A   208.94.153.100
southwest.com.  602930  IN  NS  ns-2.southwest.com.
southwest.com.  602930  IN  NS  ns-1.southwest.com.

;; AUTHORITY SECTION:
southwest.com.  602930  IN  NS  ns-2.southwest.com.
southwest.com.  602930  IN  NS  ns-1.southwest.com.

;; ADDITIONAL SECTION:
mail-1.southwest.com.   805 IN  A   12.5.136.140
mail-2.southwest.com.   805 IN  A   63.169.44.140
ns-1.southwest.com. 172704  IN  A   12.5.136.190
ns-2.southwest.com. 172704  IN  A   63.169.44.190

;; Query time: 34 msec
;; SERVER: 192.168.7.2#53(192.168.7.2)
;; WHEN: Fri Oct  2 19:26:30 2009
;; MSG SIZE  rcvd: 239

AND
===
$ whois 208.94.153.100
[Querying whois.arin.net]
[whois.arin.net]

OrgName:Southwest Airlines Co. 
OrgID:  SOUTHW
Address:2702 Love Field Dr.
City:   Dallas
StateProv:  TX
PostalCode: 75235
Country:US

NetRange:   208.94.152.0 - 208.94.155.255 
CIDR:   208.94.152.0/22 
OriginAS:   AS16759,  AS29816
NetName:SOUTHWEST-ECOM-1
NetHandle:  NET-208-94-152-0-1
Parent: NET-208-0-0-0-0
NetType:Direct Assignment
NameServer: NS-1.SOUTHWEST.COM
NameServer: NS-2.SOUTHWEST.COM
Comment:
RegDate:2008-09-03
Updated:2009-06-24

RAbuseHandle: CEB25-ARIN
RAbuseName:   Butler, Chad Eric
RAbusePhone:  +1-214-792-7196
RAbuseEmail:  chad.but...@wnco.com 

RNOCHandle: CEB25-ARIN
RNOCName:   Butler, Chad Eric
RNOCPhone:  +1-214-792-7196
RNOCEmail:  chad.but...@wnco.com 

RTechHandle: CEB25-ARIN
RTechName:   Butler, Chad Eric
RTechPhone:  +1-214-792-7196
RTechEmail:  chad.but...@wnco.com 

OrgTechHandle: CEB25-ARIN
OrgTechName:   Butler, Chad Eric
OrgTechPhone:  +1-214-792-7196
OrgTechEmail:  chad.but...@wnco.com

Which is is, ahem, somewhat different from your whois reply.

Responsys appears to be ResponSys.com of San Bruno, CA, who are a
leading global provider of on-demand marketing solutions that empower
companies to market more effectively through email, direct mail, and
mobile channels. Since 1998, Responsys’ hosted solution has served as a
proven alternative to expensive, complex, on-premise marketing
software. - IOW they're at best a UCE source.

The MXs found by host luv.southwest.com (imh2.rsys4.net and
imh.rsys4.net) are controlled by Responsys: a whois query on rsys4.net
points straight back to them. Also the luv.southwest.com IP
(12.130.131.30) is smack in the IP range that ATT say belongs to
Responsys, (12.130.131.0 - 12.130.131.255).


Martin

Re: southwest airlines sends out their own phishing email

2009-10-02 Thread Art Greenberg

My employer's travel department just sent out a memo asking for the same 
information. No reference to Southwest Airlines in the memo.


Coincidence?

--
Art Greenberg
a...@eclipse.net

Re: southwest airlines sends out their own phishing email


On fre 02 okt 2009 21:42:22 CEST, Michael Scheidell wrote

southwest's phone has a 1 hour hold time.


nope, in time waiting do this spamassassin 21 -D -t msg | grep  
domain | less


what domains is listed ?, some trd party domains that does not use  
known nameserver ?, eg why would a airliner use another nameserver  
then a phisher ?


is some of the url listed on rbl ?

any freemail in ?

maybe stupid questions, but if you ask your self you will get the answer

--
xpoint

Re: southwest airlines sends out their own phishing email

On Fri, 2009-10-02 at 15:42 -0400, Michael Scheidell wrote:

 it REALLY looks like someone at southwest had this done.
 
 its stupid.. it encourages users to disclose private data over an
 insecure channel, and whoever authorized this (if its southwest) needs
 a LONG vacation.

Should somebody ask TSA if this is legitimate use of their name?


Martin

Re: southwest airlines sends out their own phishing email


Benny Pedersen wrote:

On fre 02 okt 2009 21:42:22 CEST, Michael Scheidell wrote

southwest's phone has a 1 hour hold time.


nope, in time waiting do this spamassassin 21 -D -t msg | grep 
domain | less


what domains is listed ?, some trd party domains that does not use 
known nameserver ?, eg why would a airliner use another nameserver 
then a phisher ?



luv.southwest.com


is some of the url listed on rbl ?


no


any freemail in ?


no


maybe stupid questions, but if you ask your self you will get the answer

still doesn't answer, dkim signed, spf passes, all domains end in 
.southwest.com




_
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.secnap.com/products/spammertrap/

_

if this is legit, SW needs to protect their servers Re: southwest airlines sends out their own phishing email

from other that have see this email from other airlines:
(and, sw needs to protect my PPI by using SSL servers, not plain text
servers that belong to a marketing company)

Is the TSA “trying to scare me into providing personal information”?

June 2, 2009

Secure Flight. Just the mention of those two words is enough to confuse,
frustrate or frighten the average air traveler. As in, “The
Transportation Security Administration’s new Secure Flight program will
require you to … (insert name of ridiculous new policy here).”

The question now isn’t what is Secure Flight. It’s, “what isn’t it?

Frank Perch got the following email from AirTran the other day, for example.

Recently, the Transportation Security Administration announced
changes to their watch list matching process called Secure Flight. The
mission of Secure Flight is to enhance the security of domestic and
international air travel through the use of improved watch list
matching. Another benefit will be greatly reduced incidents of
passengers being misidentified with names on the TSA’s watch lists.

http://www.elliott.org/blog/is-the-tsa-trying-to-scare-me-into-providing-personal-information/

He thought it was a scam.

The email does not exactly say, but strongly implies, that if I goof
up — if my name on the reservation does not exactly match the format on
my ID — that my ticket will not be valid.

My first reaction to this email was actually that it must be a
phishing email of some kind. Some crook is trying to scare me into
providing personal information. Yet the email seemed to pass many of the
usual phishing tests. I couldn’t find any spoofed hyperlinks for instance.

I was still suspicious though because none of the other airlines I
deal with was contacting me about this alleged requirement, which the
email says is effective TODAY, and also usually when there is something
important like that one would expect a bit of advance notice.

As it turns out, the email is legit, and so is the requirement. But
Perch’s note underscores the fact that there’s so much misinformation
about the new TSA policy, it’s amazing that air travel hasn’t ground to
a halt.

_
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com

Re: southwest airlines sends out their own phishing email


On fre 02 okt 2009 22:03:23 CEST, Michael Scheidell wrote
still doesn't answer, dkim signed, spf passes, all domains end in  
.southwest.com


then some using a smtp auth or hacked computer inside, or dkim-sign
any mails ?

send to abuse at theredomain dot tld, yes its a grey area where one
like me from outside cant do much other then tell them

--
xpoint

Re: Daily statistics into email

 On Fri, 2009-10-02 at 21:33 +0300, Jari Fredriksson wrote:
 On Fri, 2009-10-02 at 20:45 +0300, Jari Fredriksson
 wrote: 
 
 Sendmail command is available with sendmail and postfix
 emailers, dunno about others.
 
 You don't need to use sendmail: if the cron job writes
 anything to stdout (or stderr) this is automatically
 mailed to root.
 
 If you'd rather that mail sent to root comes to you
 instead, just add a redirection line to /etc/aliases.
 Don't forget to regenerate the aliases database by
 running 'aliases' or your redirection won't take effect.
 
 
 
 That HTML version needs to add a header for
 Content-Type. That is not possible by just echoing
 somehing, as those go automatically to the body.  
 
 The non-html version uses cron's default behaviour, but
 the html version must use sendmail. 
 
 As crond must also use sendmail to ship any text that is
 left for it to deal with and you've already inserted the
 MIME header, it seems to me that the HTML processing
 would happen anyway, regardless of whether the call to
 sendmail was implicit or explicit. Hence I assumed that
 you'd just sendmail to avoid using the aliases system.
 What did I miss?  
 

As I have understood it, crond grabs the stdout of the process it runs, and 
puts that as the payload of the email it generates. It writes the mail 
headers for that mail itself.

An email contains two parts, the headers and the body. They are separated by a 
blank line.

First blank line in the stream is the separator. It is not possible to inject 
headers in the stdout of the cron job, as cron injects the blank line between 
the body and the headers it generates.

If you do not belive, just make a file like this:

--- text.sh --

echo Content-Type: text/html; charset=\us-ascii\
echo 
echo Hello | txt2html

---

And then command


batch text.sh

There is a blank line between Content-Type and Hello, but the Content-Type line 
WILL get to the body, and the html gets injected after it as raw html code, not 
as html (because the actual content type will be text not html).

Cron does NOT allow manipulating the headers. ALL the output is assumed to be 
payload.

If you know better, please post a sample.

Re: Daily statistics into email

On Fri, 2009-10-02 at 23:28 +0300, Jari Fredriksson wrote:
 There is a blank line between Content-Type and Hello, but the
 Content-Type line WILL get to the body, and the html gets injected
 after it as raw html code, not as html (because the actual content
 type will be text not html).
 
 Cron does NOT allow manipulating the headers. ALL the output is
 assumed to be payload.
 
Fair comment: I asked the question and got your answer. Thanks.

However, did you know that sendmail isn't completely dumb? 

I have a nightly backup job that stops Postgres, does the backup and
restarts it. The Postgres stop and start are done using 'service', which
returns a Stopping Postgres [FAILED] message if Postgres fails to
stop. FAILED is red, so the line contains non-ASCII characters. Until I
fixed this failure sendmail was converting the entire message body into
base64 encoding due to the X-term sequences being used to mark the start
and stop of the red foreground.

I can't see any reference to this behavior in the sendmail manpage.


Martin

Re: Daily statistics into email

 On Fri, 2009-10-02 at 23:28 +0300, Jari Fredriksson wrote:
 There is a blank line between Content-Type and Hello,
 but the Content-Type line WILL get to the body, and the
 html gets injected after it as raw html code, not as
 html (because the actual content type will be text not
 html). 
 
 Cron does NOT allow manipulating the headers. ALL the
 output is assumed to be payload.
 
 Fair comment: I asked the question and got your answer.
 Thanks. 
 
 However, did you know that sendmail isn't completely dumb?
 
 I have a nightly backup job that stops Postgres, does the
 backup and restarts it. The Postgres stop and start are
 done using 'service', which returns a Stopping Postgres
 [FAILED] message if Postgres fails to stop. FAILED is
 red, so the line contains non-ASCII characters. Until I
 fixed this failure sendmail was converting the entire
 message body into base64 encoding due to the X-term
 sequences being used to mark the start and stop of the
 red foreground.  
 
 I can't see any reference to this behavior in the
 sendmail manpage. 

This is something that I have no knowledge.

Could you see the source format of the mail? I can't think anything except it 
being in HTML format, as there is no AFAIK no other formats for rich text in 
email.

Maybe sendmail (was it really sendmail?) can convert when it sees ANSI or 
something codes in the data..

Interesting.

RE RCVD_VIA_APNIC

2009-10-02 Thread hamann . w


 Warren Togami wrote:
 # 2005/07/29, http://www.apnic.net/db/ranges.html
 header   RCVD_VIA_APNIC Received =~ 
 /[^0-9.](?:5[89]|6[01]|12[456]|20[23]|21[0189]|22[012])(?:\.[012]?[0-9]{1,2}){3}(?:\]|\)|
  
 )/
 describe RCVD_VIA_APNIC Received through a relay in Asia/Pacific Network

 Adam Katz had this rule in one of his channels.  While it is wholly 
 unsafe to be used alone, it could be useful in masscheck statistics and 
 possibly if used in meta booleans in combination with other rules.

 http://www.apnic.net/publications/research-and-insights/ip-address-trends/apnic-resource-range
 Unfortunately, in testing the above rule on my own corpus I see it is 
 missing some obvious Asian addresses.  This page reveals that the regex 
 is out of date.  Does there exist a good automated way to convert many 
 CIDR ranges to a single regex?
 
 Warren Togami

Hi Warren,

I am using the geoIP database in a similar context, but rather than converting 
to regex,
I convert to a cdb file and do a lookup on that.
To integrate with spamassassin, a perl cdb module would be needed

More info about cdb is available at http://cr.yp.to/cdb.html

Regards
Wolfgang

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread mouss

RW wrote:
 On Fri, 02 Oct 2009 00:14:52 +0200
 mouss mo...@ml.netoyen.net wrote:
 
 RW wrote:
 
 The term  false-positive can apply to any test. A test for ham
 that matches a spam is a false-positive, it's a matter of context.
 spam too can be (re)defined. and actually any term. but it is assumed
 here that we talk about spam detection. so false negative means miss
 and false positive means false alarm. this is the common terminology
 inherited from intrusion detection.
 
 The term comes from statistics, not intrusion detection. I don't
 know much about the latter, perhaps people in that field are a little
 sloppy in their usage, more  likely all the tests are expressed as
 tests for intrusion, so the same kind of issue doesn't arise.
 
 The source of your confusion is that you are mixing-up the terminology
 of the overall classification and individual test results. Think of
 this way, in a fingerprint comparison the meanings of TP, TN, FP and FN
 are obvious and intrinsic to the test, it would be absurd to switch
 them around depending on whether it's evidence for the defence or
 prosecution.

let's take it more easily: Please explain to me what was an FP in this
thread.

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread mouss

Karsten Bräckelmann wrote:
 On Fri, 2009-10-02 at 00:08 +0200, mouss wrote:
 Karsten Bräckelmann wrote:
 False positive. Something, that matches (positive) the criterion for a
 certain test, but should not (false).
 
 I stand to what I said.
 

I'm not surprised:)

 you can certainly devise a system to detect alpha(foo) where alpha is a
 function mapping a Banach space to a Hilbert Space, and define what FP,
 FN, FX mean in the context you consider. you can also say let PI=69,
 ... . but conventions are here for a reason. they allow us to
 understand each others more easily. the fact that children of today can
 solve computation problems that great scientists of the old times
 couldn't handle is thanks to conventions (think of a/b * c/d =
 (a*c)/(b*d), which looks trivial today, but wasn't before).

 when talking about spam or intrusion detection, FN means missing and
 FP means false alarm. if we allow defining FN and FP differently, then
 we'll need to rewrite a lot of books, reports, articles, ...
 
 IFF you are talking about the black box that spam detection is, that is
 true.
 
 If you are talking about a rule like FORGED_MUA_OUTLOOK, it appears to
 be that simple. However, it is not. You are looking at a single test,
 which -- if positive -- either is correct or wrong.
 

I understand the rationale, but I find this too abstract for common
discussions.

 Same for RCVD_IN_DNSWL. If it positively matches, it either it is
 correct, or wrong. A false positive is a match, that is wrong. No matter
 the score you assign the test.
 

except that it depends what the test really means. dnswl doesn't mean
the listed hosts never send spam. I am happy that it lists debian list
servers, Orange, ... etc.

 
 This concept is NOT specific to spam detection, or even computer
 science. As a matter of fact, when I first really grasped the concept, a
 medical scientist explained it to me.
 

now that you say it, this is true. I too believ that medical science has
precedence in this area.

 Yes, a FP for a rule that identifies *ham* actually evaluated positive
 on a spam. It only appears to be spam centric on this list, cause it is
 mainly dedicated to identifying spam, not ham.
 
 You might want to ask wikipedia as well. And don't focus on the spam
 filtering *example*, which again exclusively talks about a rule
 identifying spam. Not ham.
 

my point was that in a spam oriented forum, the meaning of some words is
what most of us (yes, this is hard to define) think they mean. the
principle of least astonishment.


anyway, I'm sorry for bringing the discussion to this sand. so I will
stop here (of course, offlist is ok for any discussion, including
garbage without collection:)

Re: southwest airlines sends out their own phishing email


Benny Pedersen wrote:

On fre 02 okt 2009 22:03:23 CEST, Michael Scheidell wrote
still doesn't answer, dkim signed, spf passes, all domains end in 
.southwest.com


then some using a smtp auth or hacked computer inside, or dkim-sign
any mails ?


SUPPRIZE.. its legit folks.

SF phone lines, and web site have been swamped by people all day calling 
to see if this was legit!


http://www.blogsouthwest.com/blog/secure-flight-procedures

(however, its STILL AN INSECURE HTTP BASED FORM ON A PARTNER SITE, A 
PARTNER WHO IS A PERMISSION BASED EMAIL MARKETING COMPANY)


Bad, stupid, really stupid... go put your dunce cap on and sit in the 
corner.


I believe that this attempt violated the TSA's privacy policies as well 
(asking a third party to collect information over a non ssl encrypted, 
non authenticated web site?)



_
This email has been scanned and certified safe by SpammerTrap(r). 
For Information please see http://www.spammertrap.com

_

Re: [SA] RE RCVD_VIA_APNIC

2009-10-02 Thread Adam Katz

Warren Togami wrote:
 # 2005/07/29, http://www.apnic.net/db/ranges.html
 header   RCVD_VIA_APNIC Received =~ 
 /[^0-9.](?:5[89]|6[01]|12[456]|20[23]|21[0189]|22[012])(?:\.[012]?[0-9]{1,2}){3}(?:\]|\)|
  
 )/
 describe RCVD_VIA_APNIC Received through a relay in Asia/Pacific Network

 Adam Katz had this rule in one of his channels. While it is wholly 
 unsafe to be used alone, it could be useful in masscheck statistics
 and possibly if used in meta booleans in combination with other
 rules.
 
 Unfortunately, in testing the above rule on my own corpus I see it
 is missing some obvious Asian addresses. This page reveals that the
 regex is out of date. Does there exist a good automated way to
 convert many CIDR ranges to a single regex?

Hm.  I didn't know that APNIC's space was updated that often.  I'll
adjust my rule.  Also, though I didn't say anything when you
approached me in IRC (we're on vastly different schedules), I did make
some changes to the rule so as to make it safer, including checking
against trusted networks and DNS whitelists and scoring it at 0.001.

__RCVD_VIA_APNIC will soon be updated to a monster constructed from a
hand-tweaked copy of the table at http://www.apnic.net/db/ranges.html
and fed into Regexp::Assemble (post-tweaked perl code is attached).

The attached apnic.cf.txt file (named so as to better appear in your
mail reader) is a sample of the pending latest revision in khop-bl.

As to its missing some obvious Asian addresses ... I believe that is
because many Asian addresses are outside the jurisdiction of APNIC,
for example, I believe Japan has three /8 networks (43, 126, and 133)
independent of APNIC ... and that's just by eying the XKCD map of the
IPv4 space!
# 2009/10/02 from http://www.apnic.net/db/ranges.html   meta bits added 20090930
header __RCVD_VIA_APNIC Received =~ 
/(?-xism:[^0-9.](?:2(?:0(?:2(?:\.1(?:2(?:3\.(?:0?(?:[4-9][0-9]|3[2-9])|[12][0-9]{2})\.[012]?[0-9]{1,2}|[^3]\.(?:012]?[0-9]{1,2}){2})|[^2]3\.(?:012]?[0-9]{1,2}){2})|(?:.[02]?[0-9]{1,2}){3})|3(?:.[012]?[0-9]{1,2}){3})|(?:1[0189]|2[012])(?:.[012]?[0-9]{1,2}){3})|1(?:(?:2[0123456]|8[023]|1\d|75)(?:.[012]?[0-9]{1,2}){3}|69\.2(?:1[0-9]|2[0-3]|0[89])(?:.[012]?[0-9]{1,2}){2})|(?:5[89]|6[01])(?:.[012]?[0-9]{1,2}){3})(?:\]\)\s]))/
meta RCVD_VIA_APNIC __RCVD_VIA_APNIC  !__KHOP_DNSWLD  !ALL_TRUSTED
describe RCVD_VIA_APNIC Received through a relay in Asia/Pacific Network
tflags   RCVD_VIA_APNIC noautolearn 
#score   RCVD_VIA_APNIC 0.4 0.2 0.7 0.5 # lowered for autolearn BLs
scoreRCVD_VIA_APNIC 0.001 # 20090930: not suitable for blanket publication

meta __KHOP_DNSWLD  RCVD_IN_DNSWL_LOW || RCVD_IN_DNSWL_MED || 
RCVD_IN_DNSWL_HI || RCVD_IN_JMF_W || RCVD_IN_BSP_TRUSTED || RCVD_IN_IADB_DOPTIN 
|| RCVD_IN_IADB_ML_DOPTIN || RCVD_IN_IADB_VOUCHED || RCVD_IN_SSC_TRUSTED_COI
#!/usr/bin/perl -w

use Regexp::Assemble;

my $ra = Regexp::Assemble-new;
my $start = '[^0-9.]';
my $end = '(?:\]\)\s])';
my $cidr8tail = '(?:.[012]?[0-9]{1,2}){3}' . $end;

$ra-add($start . '58' . $cidr8tail);
$ra-add($start . '59' . $cidr8tail);
$ra-add($start . '60' . $cidr8tail);
$ra-add($start . '61' . $cidr8tail);
$ra-add($start . '110' . $cidr8tail);
$ra-add($start . '111' . $cidr8tail);
$ra-add($start . '112' . $cidr8tail);
$ra-add($start . '113' . $cidr8tail);
$ra-add($start . '114' . $cidr8tail);
$ra-add($start . '115' . $cidr8tail);
$ra-add($start . '116' . $cidr8tail);
$ra-add($start . '117' . $cidr8tail);
$ra-add($start . '118' . $cidr8tail);
$ra-add($start . '119' . $cidr8tail);
$ra-add($start . '120' . $cidr8tail);
$ra-add($start . '121' . $cidr8tail);
$ra-add($start . '122' . $cidr8tail);
$ra-add($start . '123' . $cidr8tail);
$ra-add($start . '124' . $cidr8tail);
$ra-add($start . '125' . $cidr8tail);
$ra-add($start . '126' . $cidr8tail);
$ra-add($start . '169\.20[89](?:.[012]?[0-9]{1,2}){2}' . $end);
$ra-add($start . '169\.21[0-9](?:.[012]?[0-9]{1,2}){2}' . $end);
$ra-add($start . '169\.22[0-3](?:.[012]?[0-9]{1,2}){2}' . $end);
$ra-add($start . '175' . $cidr8tail);
$ra-add($start . '180' . $cidr8tail);
$ra-add($start . '182' . $cidr8tail);
$ra-add($start . '183' . $cidr8tail);
$ra-add($start . '202(?:.[02]?[0-9]{1,2}){3}' . $end);
$ra-add($start . '202\.12[^3]\.(?:012]?[0-9]{1,2}){2}' . $end);
$ra-add($start . '202\.1[^2]3\.(?:012]?[0-9]{1,2}){2}' . $end);
$ra-add($start . '202\.123\.[12][0-9]{2}\.[012]?[0-9]{1,2}' . $end);
$ra-add($start . '202\.123\.0?[4-9][0-9]\.[012]?[0-9]{1,2}' . $end);
$ra-add($start . '202\.123\.0?3[2-9]\.[012]?[0-9]{1,2}' . $end);
$ra-add($start . '203' . $cidr8tail);
$ra-add($start . '210' . $cidr8tail);
$ra-add($start . '211' . $cidr8tail);
$ra-add($start . '218' . $cidr8tail);
$ra-add($start . '219' . $cidr8tail);
$ra-add($start . '220' . $cidr8tail);
$ra-add($start . '221' . $cidr8tail);
$ra-add($start . '222' . $cidr8tail);

print header __RCVD_VIA_APNIC\tReceived =~ / . $ra-re . /\n;

SIGCHLD query

What causes a spamd 3.2.5 child process to be terminated by receiving a
SIGCHLD signal? 

I've looked at the spamc and spamd manpages but there's no mention of
them there. I can't remember seeing them discussed on this maillist
either.

My last month's logs show 7 of them and I can't work out what caused
them to be sent. However, Jose Luis Marin Perez' system is seeing a lot
of them - on the order of 10% of messages scanned are getting hit by
them, though his seem to be connected with very long running scans.

So, what do these signals mean and what should I do to my SA
configuration to get rid of them.


Martin

Re: Daily statistics into email

On Sat, 2009-10-03 at 00:03 +0300, Jari Fredriksson wrote:
 This is something that I have no knowledge.
 
It was a surprise to me too!

 Could you see the source format of the mail? I can't think anything
 except it being in HTML format, as there is no AFAIK no other formats
 for rich text in email.
 
I didn't keep any examples,  but from memory the headers set the content
to base64 and the body was a single base64 block. Normally the body is
not MIME - its simply a plain ASCII body. During the backup 
I accumulate the report in a temp file and right at the end I cat it to
stdout. 

 Maybe sendmail (was it really sendmail?) can convert when it sees ANSI
 or something codes in the data..
 
AFAIK its just postfix.sendmail. I suppose its possible that crond does
the encoding internally, but that's not mentioned in its manpage.


Martin

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread Karsten Bräckelmann

On Sat, 2009-10-03 at 00:25 +0200, mouss wrote:
 Karsten Bräckelmann wrote:

False positive. Something, that matches (positive) the criterion for a
certain test, but should not (false).
  
  I stand to what I said.
 
 I'm not surprised:)

;)


  IFF you are talking about the black box that spam detection is, that is
  true.
  
  If you are talking about a rule like FORGED_MUA_OUTLOOK, it appears to
  be that simple. However, it is not. You are looking at a single test,
  which -- if positive -- either is correct or wrong.
 
 I understand the rationale, but I find this too abstract for common
 discussions.

*shrug*  You're not obliged to participate in a thread, if it is
confusing to you. That's the wonders of open discussion and diverse
input. You might stumble upon something you didn't know before... ;)


  Same for RCVD_IN_DNSWL. If it positively matches, it either it is
  correct, or wrong. A false positive is a match, that is wrong. No matter
  the score you assign the test.
 
 except that it depends what the test really means. dnswl doesn't mean
 the listed hosts never send spam. I am happy that it lists debian list
 servers, Orange, ... etc.

Exactly, in the context of a single rule (as opposed to detecting
spam), it depends on what the rule really means. Or in short, its
score's sign...


  This concept is NOT specific to spam detection, or even computer
  science. As a matter of fact, when I first really grasped the concept, a
  medical scientist explained it to me.
 
 now that you say it, this is true. I too believ that medical science has
 precedence in this area.
 
  Yes, a FP for a rule that identifies *ham* actually evaluated positive
  on a spam. It only appears to be spam centric on this list, cause it is
  mainly dedicated to identifying spam, not ham.
  
  You might want to ask wikipedia as well. And don't focus on the spam
  filtering *example*, which again exclusively talks about a rule
  identifying spam. Not ham.
 
 my point was that in a spam oriented forum, the meaning of some words is
 what most of us (yes, this is hard to define) think they mean. the
 principle of least astonishment.

Of course, these terms mostly come up WRT to overall score of a message,
which applies to detecting spam.

However, on this very list, it also commonly is referred to single rules
FP'ing, *without* pushing the ham above the required_score threshold.


The only aspect new and obviously confusing to some regulars on this
list is the negative sign of the rule's score. Inverting the is spam
test logic also inverts the meaning of F[PN]. Whether one likes this or
not.

It's all about context.


And FWIW, it is wrong to base your definitions on what the majority
thinks is correct. The majority and what's believed to be common
knowledge too often is wrong. You can observe this in real life, too...
I prefer to educate the masses instead.


-- 
char *t=\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128  (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: DNSWL and JMF White false positives, what to do exactly?

2009-10-02 Thread RW

On Sat, 03 Oct 2009 00:12:37 +0200
mouss mo...@ml.netoyen.net wrote:

 RW wrote:
  On Fri, 02 Oct 2009 00:14:52 +0200
  mouss mo...@ml.netoyen.net wrote:
  

  The source of your confusion is that you are mixing-up the
  terminology of the overall classification and individual test
  results. Think of this way, in a fingerprint comparison the
  meanings of TP, TN, FP and FN are obvious and intrinsic to the
  test, it would be absurd to switch them around depending on whether
  it's evidence for the defence or prosecution.
 
 let's take it more easily: Please explain to me what was an FP in this
 thread.

A test intended for identifying ham was being hit on spam.

A hit on a rule is a positive result. When a rule hits something it's
intended to identify, it's a true positive. When a rule hits something
it's not intended to identify, it's a false positive, and so on.

The same terminology can be used for SpamAssassin's overall spam
classification, but that's a different matter. If you talk about a rule
hit being an FN, because it might contribute to a classification FN then
you are using the terminology like a cargo-cultist.

Re: Daily statistics into email

 On Sat, 2009-10-03 at 00:03 +0300, Jari Fredriksson wrote:
 This is something that I have no knowledge.
 
 It was a surprise to me too!
 
 Could you see the source format of the mail? I can't
 think anything except it being in HTML format, as there
 is no AFAIK no other formats for rich text in email.
 
 I didn't keep any examples,  but from memory the headers
 set the content to base64 and the body was a single
 base64 block. Normally the body is not MIME - its simply
 a plain ASCII body. During the backup 
 I accumulate the report in a temp file and right at the
 end I cat it to stdout.
 
 Maybe sendmail (was it really sendmail?) can convert
 when it sees ANSI or something codes in the data..
 
 AFAIK its just postfix.sendmail. I suppose its possible
 that crond does the encoding internally, but that's not
 mentioned in its manpage. 
 

But let us keep in mind that it is the client that renders the mail for us to 
see.

it must be some format the the client must understand. postfix.sendmail is not 
a client, and whatever it does must be understandable by the client.

Re: Daily statistics into email