Re: [AMaViS-user] How to identify SPAM and SPAMMY mail in SQL
Spam = S, spammy = s Please note that MySQL is case insensitive on many string comparsion functions with certain encodings. Therefore if using VARCHAR or CHAR s == S could be true for your system. Either change your columns to VARBINARY or BINARY - or use the BINARY() function when comparing values. If running a large DB you should definitively change column types, as using string functions in your WHERE statements is seldom a good idea (performance-wise). Best regards, Thomas Gelf Luis Daniel Lucio Quiroz wrote: Hi all, Having SQL schema, in what field and what values I can identify if an email was SPAM and SPAMMY. I'm pretty sure that SPAM is in msgs.content = S, but spammy? TIA LD -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] 503 5.5.1 Error: nested MAIL
Diego, Update to lenny and etch problems started ... I have major problems with Amavis and do not know if courier-imap and postfix are also involved. The current version I'm using are: amavisd-new 1:2.6.1. dfsg-1 postfix 2.5.5-1.1 courier-imap-ssl 4.4.0-2 Sometimes sending an email from my local account to another account in my local domain. Normally I connect to remotely using TLS with SAS. But I think I also had this error logged in my mailbox from my lan. p...@pepe.com: host localhost [127.0.0.1] said: 503 5.5.1 Failed, id = 28470-17-14, from MTA ([127.0.0.1]: 10025): 503 5.5.1 Error: nested MAIL command (in reply to end of DATA command) amavisd-new-2.6.2 release notes, December 15, 2008 BUG FIXES - when feeding a message by SMTP back to MTA and MTA rejects a recipient as invalid and a smtp connection cache is enabled, the SMTP protocol can get out of step, rejecting the next message in the same connection with a 503 5.5.1 Error: nested MAIL command; this only affects (hopefully) rare sites where recipient validation is performed after content filtering instead of before content filtering; reported by Richard Smits; Mark -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] Delivery Failures From Banned Emails to Sender
Carlos, When someone sends a message that is blocked / quarantined for banned contents or viruses, I would like to get notification sent to postmast...@$mydomain however I do not want notification sent to the sender that their message was not delivered based on whatever reason. Right now I am getting admin notifications but so are the senders. How can I stop notifications to the sender that their incoming message was not relayed to it's destination? The sender is probably getting a delivery status notification. Turn it off by setting: $final_spam_destiny = D_DISCARD; instead of D_BOUNCE. Second issue is when the sender receives a notification that their message was rejected, the message is sent from postmas...@mail.iamghost.com rather than just postmas...@iamghost.com. The domain name in the $hdrfrom_notify_sender defaults to $myhostname. Either change the $myhostname, or assign $hdrfrom_notify_sender directly: $hdrfrom_notify_sender = 'Postmaster postmas...@iamghost.com'; Mark -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] Q about partitioning tables and data types
Le samedi 1 août 2009 11:15:43, Nathan M a écrit : On Fri, Jul 31, 2009 at 7:59 PM, Luis Daniel Lucio Quirozluis.daniel.lu...@gmail.com wrote: A stupid question, what advantages do I get if I use partition_tag with amavisd+mysql? Faster database cleanups. On a high traffic mail server, cleanups can begin to take days instead of minutes. Depending on your mail traffic load, I imagine it's probably not worth the trouble (unless you want the most efficient setup no matter what), unless your handling 500k messages per day. - N OK, and I've already a database with some emails, if I change $sql_partition_tag = 1, will I mess the DB? -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] Q about partitioning tables and data types
On Mon, Aug 3, 2009 at 9:22 AM, Luis Daniel Lucio
Quirozluis.daniel.lu...@gmail.com wrote:
OK, and I've already a database with some emails, if I change
$sql_partition_tag = 1, will I mess the DB?
$sql_partition_tag = 1; would do little good. If you have the db
setup with partition_tag, it's probably defaulting to 0 right now.
Changing everything to 1 wouldn't help your db optimization.
I think the typical setting is to put in there the iso8601 week value.
Like this:
$sql_partition_tag = sub { my($msginfo)=...@_; iso8601_week($msginfo-rx_time)
};
If your running a recent version of amavisd it'll handle the partition
tag without any trouble. However, if your using any applications
which access those databases for user quarantines, etc, make sure they
are compatible with the partition_tag. There's prior discussion and
info in the mysql readme about this topic as well.
- N
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] policy bank options ignored?
Tim,
I have the following for my originating senders:
$interface_policy{'10026'} = 'ORIGINATING';
$policy_bank{'ORIGINATING'} = {
originating = 1,
virus_admin_maps = [virus_adm...@$mydomain],
spam_admin_maps = [spam_adm...@$mydomain],
warnbadhsender = 1,
warnspamsender = 1,
spam_tag_level_maps = [2.1],
spam_tag2_level_maps = [7],
spam_kill_level_maps = [10],
spam_dsn_cutoff_level_maps = [3000],
final_spam_destiny = D_BOUNCE,
smtpd_discard_ehlo_keywords = ['8BITMIME'],
bypass_banned_checks_maps = [1],
terminate_dsn_on_notify_success = 0
};
My mail log says DSN suppressed, but I thought this was configured to
send one.
The DSN for spam is suppressed when spam score exceeds spam_dsn_cutoff_level.
Note that since 2.4.3 there are two such levels:
2.4.3 release_notes:
- introduce new variable @spam_dsn_cutoff_level_bysender_maps (also member of
policy banks), complementing an existing by-recipient list of lookup tables
@spam_dsn_cutoff_level_maps. The new variable serves to make it possible
to trim down spam bounces to domains sending their own bounces with non-null
return path (envelope sender address) and without DSN NOTIFY=NEVER option,
but also to frequently abused domains, or to those sending marginal spam.
When spam level exceeds either the @spam_dsn_cutoff_level_bysender_maps
or the @spam_dsn_cutoff_level_maps level, (non)delivery status notification
is suppressed even with $final_spam_destiny set to D_BOUNCE;
Their defaults are:
@spam_dsn_cutoff_level_maps = (\$sa_dsn_cutoff_level);
@spam_dsn_cutoff_level_bysender_maps = (\$sa_dsn_cutoff_level);
If your intention is to disable spam cutoff level within a
policy bank, set both to undef:
spam_dsn_cutoff_level_maps = undef,
spam_dsn_cutoff_level_bysender_maps = undef,
or set the:
sa_dsn_cutoff_level = undef,
which achieves the same.
warnspamsender = 1,
Don't do this, the 'warnspamsender' is a relict from old versions
and is only there for compatibility with old config files. Sending
of a DSN (= bounce) is nowadays controlled through final_spam_destiny
and the spam_dsn_cutoff_level.
In testing, I get spam notifications to my spam_admin address (this is
not configured elsewhere), but the spam is not bounced to the sender.
because of a missing spam_dsn_cutoff_level_bysender_maps=undef
Also, the notification email says Content analysis details: (998.8
points, 5.0 required) but 5.0 is not the setting anywhere, and
clearly not in this policy.
The 5.0 comes from a SpamAssassin config file ('required_hits')
and only appears in the SpamAssassin report, it has no other
effect. Just ignore it, or for cosmetic reasons set required_hits
in local.cf to your kill level (or tag2 level if you prefer).
What is the difference between warnspamsender and
final_spam_destiny = D_BOUNCE supposed to be
anyway?
It's conceptually the same thing, the warnspamsender is mapped to
final_spam_destiny for backwards compatibility. Don't bother with
warnspamsender and don't change its default value, just pretend
it never existed.
It seems the originating = 1 must override certain things.
I have cut it out and am getting behavior closer to what I want.
Don't do that. If you know mail hitting a policy bank is coming
from local networks or authenticated users, the 'originating'
flag is supposed to be set - it allows amavisd to distinguish
between internal-to-internal and inbound mail, among other
things it controls DKIM signing.
Mark
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] Q about partitioning tables and data types
Luis, OK, and I've already a database with some emails, if I change $sql_partition_tag = 1, will I mess the DB? Should be alright. Note that each partition will collect its own subset of e-mail addresses in a table maddr, which means some addresses will be duplicates, typically all local user addresses and frequent senders. Unless you intend to make use of partitions, it does not pay off to use different values, it just wastes some db storage space. Mark -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
[AMaViS-user] POLL: Database Hardware vs Daily Messages
I am curious what amavisd-new users out there have for DB setups for maintaining the storage component of amavis data (maddr, msgrcpt, msgs, quarantine tables). I'll start. Incoming Email: 10,000,000 messages/day Current Hardware: Single Core 2000Mhz Athlon 64 CPU 4GB RAM 2 x Seagate ES 500GB 7200rpm in RAID1 setup This hardware is not cutting it any longer.. Proposed new hardware hope to have online in 2 weeks. Quad Core 2.3Ghz Opteron CPU 8GB RAM 4 x WD Velociraptor 10krpm 300GB in RAID10 setup - N -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] POLL: Database Hardware vs Daily Messages
On Mon, 3 Aug 2009, Nathan M wrote: I am curious what amavisd-new users out there have for DB setups for maintaining the storage component of amavis data (maddr, msgrcpt, msgs, quarantine tables). I'll start. Incoming Email: 10,000,000 messages/day Current Hardware: Single Core 2000Mhz Athlon 64 CPU 4GB RAM 2 x Seagate ES 500GB 7200rpm in RAID1 setup This hardware is not cutting it any longer.. Proposed new hardware hope to have online in 2 weeks. Quad Core 2.3Ghz Opteron CPU 8GB RAM 4 x WD Velociraptor 10krpm 300GB in RAID10 setup Add 2x 146GB disks in RAID1 for OS and swap. In anything with intensive I/O it really helps. Go SAS for both OS disks and data if you can. I've found SAS seriously out performs SATAII for multiple mixed simultaneous read/write operations. SATAII, although MTBF is much better than it used to be still has problems with multiple simultaneous I/O requests and will become a serious bottleneck. We use 3 routing nodes running amavisd-new writing to a single database on a host which houses multiple databases. We use mailzu to view the quarantine and release messages plus an inhouse app to manage the sql tables for policies. The database host is a Dell PE1950, 4G RAM, with LSI MegaRAID (which is our bottleneck on I/O, PERC controllers are slow) and 2 RAID1 giving seperate spindles for OS and data. We don't have near the mail you handle (only around 20k/day) but the DB host does around 30 queries/second average. Regards Jim - N -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/ -- James Bourne | Email:jbou...@hardrock.org UNIX Systems Administration | WWW: http://www.hardrock.org Custom UNIX Programming | Linux: The choice of a GNU generation -- All you need's an occasional kick in the philosophy. Frank Herbert Need an inexpensive domain alternative? http://fastforwarddomains.com -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] Q about partitioning tables and data types
Le lundi 3 août 2009 12:13:06, Nathan M a écrit :
On Mon, Aug 3, 2009 at 9:22 AM, Luis Daniel Lucio
Quirozluis.daniel.lu...@gmail.com wrote:
OK, and I've already a database with some emails, if I change
$sql_partition_tag = 1, will I mess the DB?
$sql_partition_tag = 1; would do little good. If you have the db
setup with partition_tag, it's probably defaulting to 0 right now.
Changing everything to 1 wouldn't help your db optimization.
I think the typical setting is to put in there the iso8601 week value.
Like this:
$sql_partition_tag = sub { my($msginfo)=...@_; iso8601_week($msginfo-rx_time)
};
If your running a recent version of amavisd it'll handle the partition
tag without any trouble. However, if your using any applications
which access those databases for user quarantines, etc, make sure they
are compatible with the partition_tag. There's prior discussion and
info in the mysql readme about this topic as well.
- N
---
--- Let Crystal Reports handle the reporting - Free Crystal Reports 2008
30-Day trial. Simplify your report design, integration and deployment - and
focus on what you do best, core application coding. Discover what's new
with Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
Thanx, I've read that. I'm modifying mailzu-ng, as far as I've read code, it
does not use partition_tag field.
I'm concerned more about 2nd level support, in where we do sql statements
directly rathern than Web frontend. Such as dropping all all messages after 5
years (huge db jejje) or something like this.l
LD
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] policy bank options ignored?
Thanks for your reply :)
On Mon, 3 Aug 2009 19:11:45 +0200
Mark Martinec mark.martinec+ama...@ijs.si wrote:
The DSN for spam is suppressed when spam score exceeds spam_dsn_cutoff_level.
Yes, that's why I set it so high, I was using the GTUBE email to test
which scores around 1000.
Note that since 2.4.3 there are two such levels:
2.4.3 release_notes:
- introduce new variable @spam_dsn_cutoff_level_bysender_maps (also member of
policy banks), complementing an existing by-recipient list of lookup tables
@spam_dsn_cutoff_level_maps. The new variable serves to make it possible
to trim down spam bounces to domains sending their own bounces with non-null
return path (envelope sender address) and without DSN NOTIFY=NEVER option,
but also to frequently abused domains, or to those sending marginal spam.
When spam level exceeds either the @spam_dsn_cutoff_level_bysender_maps
or the @spam_dsn_cutoff_level_maps level, (non)delivery status notification
is suppressed even with $final_spam_destiny set to D_BOUNCE;
Their defaults are:
@spam_dsn_cutoff_level_maps = (\$sa_dsn_cutoff_level);
@spam_dsn_cutoff_level_bysender_maps = (\$sa_dsn_cutoff_level);
If your intention is to disable spam cutoff level within a
policy bank, set both to undef:
spam_dsn_cutoff_level_maps = undef,
spam_dsn_cutoff_level_bysender_maps = undef,
or set the:
sa_dsn_cutoff_level = undef,
which achieves the same.
Interesting, although I did manage to get the result I was looking for
once I restarted Amavis *and* postfix. Not sure why...
warnspamsender = 1,
Don't do this, the 'warnspamsender' is a relict from old versions
and is only there for compatibility with old config files. Sending
of a DSN (= bounce) is nowadays controlled through final_spam_destiny
and the spam_dsn_cutoff_level.
Ah. OK. I have removed it.
In testing, I get spam notifications to my spam_admin address (this is
not configured elsewhere), but the spam is not bounced to the sender.
because of a missing spam_dsn_cutoff_level_bysender_maps=undef
I'll add this.
Also, the notification email says Content analysis details: (998.8
points, 5.0 required) but 5.0 is not the setting anywhere, and
clearly not in this policy.
The 5.0 comes from a SpamAssassin config file ('required_hits')
and only appears in the SpamAssassin report, it has no other
effect. Just ignore it, or for cosmetic reasons set required_hits
in local.cf to your kill level (or tag2 level if you prefer).
Ah, OK. Thanks!
What is the difference between warnspamsender and
final_spam_destiny = D_BOUNCE supposed to be
anyway?
It's conceptually the same thing, the warnspamsender is mapped to
final_spam_destiny for backwards compatibility. Don't bother with
warnspamsender and don't change its default value, just pretend
it never existed.
OK, cool. Good to know.
It seems the originating = 1 must override certain things.
I have cut it out and am getting behavior closer to what I want.
Don't do that. If you know mail hitting a policy bank is coming
from local networks or authenticated users, the 'originating'
flag is supposed to be set - it allows amavisd to distinguish
between internal-to-internal and inbound mail, among other
things it controls DKIM signing.
Again, thanks for that info!
--TimH
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/
Re: [AMaViS-user] POLL: Database Hardware vs Daily Messages
On Mon, Aug 03, 2009 at 10:24:48AM -0700, Nathan M wrote: I am curious what amavisd-new users out there have for DB setups for maintaining the storage component of amavis data (maddr, msgrcpt, msgs, quarantine tables). I'll start. Incoming Email: 10,000,000 messages/day Current Hardware: Single Core 2000Mhz Athlon 64 CPU 4GB RAM 2 x Seagate ES 500GB 7200rpm in RAID1 setup This hardware is not cutting it any longer.. Proposed new hardware hope to have online in 2 weeks. Quad Core 2.3Ghz Opteron CPU 8GB RAM 4 x WD Velociraptor 10krpm 300GB in RAID10 setup So what is your budged? These look like a bit cheap solutions. Think about getting a battery backed RAID box with cache, it will enable much more than internal disks. -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ AMaViS-user mailing list AMaViS-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
