Re: [vchkpw] Removng Duplicate Email
Hi Remo:
Yes, I use maildrop and the standard Spamassassin filters in
/usr/share/spamassassin. My local.cf file in /etc/mail/spamassassin is:
# These values can be overridden by editing ~/.spamassassin/user_prefs.cf
# (see spamassassin(1) for details)
# These should be safe assumptions and allow for simple visual sifting
# without risking lost emails.
ok_locales all
skip_rbl_checks 1
required_hits 5.0
rewrite_header Subject ***SPAM***
report_safe 0
use_pyzor 1
use_razor2 1
use_dcc 1
use_auto_whitelist 1
use_bayes 1
use_bayes_rules 1
bayes_auto_learn 1
bayes_file_mode 0666
add_header all Status _YESNO_ Autolearn=_AUTOLEARN_ score=_SCORE_
required=_REQD_ tests=_TESTS_ bayes=_BAYES_ Checker-Version SpamAssassin
_VERSION_ (_SUBVERSION_) on _HOSTNAME_
add_header all Level _STARS(*)_
add_header spam Flag _YESNOCAPS_
I also run DCC, Pyzor and Razor2 (as you can see). The dups are
intermittent and happen when the server is under heavy load. I read an
article here:
http://www.cyber-sentry.com/index.php?id=108 as to why this can happen. The
filtering software appears to be working fine. My qmail install is a
standard toaster install from here:
http://wiki.qmailtoaster.com/index.php/CentOS_4.3_QmailToaster_Install. All
I really need to do is implement this eliminate-dups script if it's
required. Maybe play with the timeouts of Pyzor, Razor2 etc?
Cheers,
Dave
- Original Message -
From: Remo Mattei [EMAIL PROTECTED]
To: vchkpw@inter7.com
Sent: Monday, December 04, 2006 12:36 AM
Subject: Re: [vchkpw] Removng Duplicate Email
so u get dub when mail comes in? do u use maildrop? if you also use
other filter that may cause the problem.
Remo
Dave Cook wrote:
Hi Remo:
That was just a left-over from testing. I removed the user-default and
I still get dups. That's where I started this morning when the dup
thing started.
Getting back to the eliminate-dups script.
1) I have the following in .qmail-delivery: (in
/home/vpopmail/domains/mydomain.com/)
| /usr/local/sbin/eliminate-dups mydomain.com duphash
[EMAIL PROTECTED]
2) I have the following in .qmail-default: (in
/home/vpopmail/domains/mydomain.com/)
| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox
3) The eliminate-dups script is here:
#! /usr/bin/perl
#--
# Copyright 2006 Russell Nelson [EMAIL PROTECTED]
# This program is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#--
#
#
# Modified for use with vpopmail
# by Chris Hardie [EMAIL PROTECTED]
# originally by Russell Nelson, http://www.qmail.org/eliminate-dups
#
#
# Now, if delivery to the mbox is deferred, eliminate-dups will NOT be
# run a second time for the same message.
#
# Set up ~vpopmail/domains/domain.com/.qmail-default as follows:
#
# | /home/vpopmail/bin/vdelivermail '' [EMAIL PROTECTED]
#
# Then create a ~vpopmail/domains/domain.com/.qmail-delivery file as
follows:
#
# |bin/eliminate-dups domain.com duphash
# [EMAIL PROTECTED]
#
# Now, if delivery to the mbox is deferred, eliminate-dups will NOT be
# run a second time for the same message.
my $basedir = /home/vpopmail/domains;
my $domainname = shift;
my $hname = shift;
my $hashname = $basedir/$domainname/$hname;
use Digest::MD5;
$md5 = new Digest::MD5;
$loose = 1; # loose matching if set.
while() {
last if /^$/;
next if $ignore_continue /^\s/;
$ignore_continue = 0;
if (/^received:/i) {
$ignore_continue = 1;
next;
}
if (!$loose) {
$headers .= $_;
next;
}
if ($keep_continue /^\s/) {
$headers .= $_;
next;
}
$keep_continue = 0;
if (m/^(from|message-id|date):/i) {
$headers .= $_;
$keep_continue = 1;
next;
}
next;
}
$md5-add($headers);
$md5-addfile(STDIN);
$hash = $md5-hexdigest;
print $headers Our hash:$hash\n;
if (open(HASH, $hashname.newer)) {
flock(HASH, 2);
while(HASH) { chomp; exit 99 if $_ eq $hash; }
}
open(HASH, $hashname.older) || die $0: Cannot open $hashname.older;
while(HASH) { chomp; exit 99 if $_ eq $hash; }
# roll the files once a week.
if (-M $hashname.older 7) {
rename($hashname.newer, $hashname.older) || die $0: Unable
to move newer to older;
}
# add the hash to the received messages list.
open(HASH, $hashname.newer) || die $0: Cannot append to
$hashname.newer;
print HASH $hash\n;
close(HASH);
print Original message;
exit 0;
a) Putting | /home/vpopmail/bin/vdelivermail '' [EMAIL PROTECTED] in
the .qmail-default file stops email delivery and doesn't work.
b) This file is owned by vpopmail.vchkpw and is executable and is in
my /usr/local/sbin directory.
Question: Does vpopmail read .qmail files in the user's Maildirs? (i.e.
/home/vpopmail/domains/mydomain.com
Re: [vchkpw] Removng Duplicate Email
Dang, I was going to use that one. Maybe I'll switch. Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Monday, December 04, 2006 11:35 AM Subject: Re: [vchkpw] Removng Duplicate Email I would use bill shupp toasterI just build a centos 4.4 with that it's the best online... Just my 2 cents Remo Dave Cook wrote: Hi Remo: Yes, I use maildrop and the standard Spamassassin filters in /usr/share/spamassassin. My local.cf file in /etc/mail/spamassassin is: # These values can be overridden by editing ~/.spamassassin/user_prefs.cf # (see spamassassin(1) for details) # These should be safe assumptions and allow for simple visual sifting # without risking lost emails. ok_locales all skip_rbl_checks 1 required_hits 5.0 rewrite_header Subject ***SPAM*** report_safe 0 use_pyzor 1 use_razor2 1 use_dcc 1 use_auto_whitelist 1 use_bayes 1 use_bayes_rules 1 bayes_auto_learn 1 bayes_file_mode 0666 add_header all Status _YESNO_ Autolearn=_AUTOLEARN_ score=_SCORE_ required=_REQD_ tests=_TESTS_ bayes=_BAYES_ Checker-Version SpamAssassin _VERSION_ (_SUBVERSION_) on _HOSTNAME_ add_header all Level _STARS(*)_ add_header spam Flag _YESNOCAPS_ I also run DCC, Pyzor and Razor2 (as you can see). The dups are intermittent and happen when the server is under heavy load. I read an article here: http://www.cyber-sentry.com/index.php?id=108 as to why this can happen. The filtering software appears to be working fine. My qmail install is a standard toaster install from here: http://wiki.qmailtoaster.com/index.php/CentOS_4.3_QmailToaster_Install. All I really need to do is implement this eliminate-dups script if it's required. Maybe play with the timeouts of Pyzor, Razor2 etc? Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Monday, December 04, 2006 12:36 AM Subject: Re: [vchkpw] Removng Duplicate Email !DSPAM:45744843321701987214747! -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.409 / Virus Database: 268.15.6/567 - Release Date: 12/4/2006
Re: [vchkpw] Removng Duplicate Email
do you have a .qmail-user file? what are the setting on that? Remo Dave Cook wrote: Hi: I'm trying to use Chris Hardie's modified Russell Nelson script: http://www.mail-archive.com/vchkpw@inter7.com/msg08313.html to remove duplicate email my customers are getting. I followed everything to the letter but can't seem to get it to work. I tried the .default-delivery file setup in the /home/vpopmail/domains/mydomain.com/ directory and modified the .qmail-default file for that domain as per the instructions but nothing seems to be happening. I'm new to vpopmail so if someone has got this working could you shed some light on it for me? Where should these files reside to have this work properly? Cheers, Dave Cook !DSPAM:457358e1262601480917376!
Re: [vchkpw] Removng Duplicate Email
Hi Remo: Thanks for the response. Yes, I have that file. For myself as an example: .qmail-dcook and it contains: | /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Sunday, December 03, 2006 7:35 PM Subject: Re: [vchkpw] Removng Duplicate Email do you have a .qmail-user file? what are the setting on that? Remo Dave Cook wrote: Hi: I'm trying to use Chris Hardie's modified Russell Nelson script: http://www.mail-archive.com/vchkpw@inter7.com/msg08313.html to remove duplicate email my customers are getting. I followed everything to the letter but can't seem to get it to work. I tried the .default-delivery file setup in the /home/vpopmail/domains/mydomain.com/ directory and modified the .qmail-default file for that domain as per the instructions but nothing seems to be happening. I'm new to vpopmail so if someone has got this working could you shed some light on it for me? Where should these files reside to have this work properly? Cheers, Dave Cook !DSPAM:457358e1262601480917376! -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.409 / Virus Database: 268.15.6/565 - Release Date: 12/2/2006
Re: [vchkpw] Removng Duplicate Email
so is this the .qmail-default or the user? Remo Dave Cook wrote: Hi Remo: Thanks for the response. Yes, I have that file. For myself as an example: .qmail-dcook and it contains: | /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Sunday, December 03, 2006 7:35 PM Subject: Re: [vchkpw] Removng Duplicate Email !DSPAM:4573737c283213607818837!
Re: [vchkpw] Removng Duplicate Email
Hi Remo: It's the user default file in /home/vpopmail/domains/mydomain.com/ I also have a .qmail-default file in that same directory with the same contents. Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Sunday, December 03, 2006 8:51 PM Subject: Re: [vchkpw] Removng Duplicate Email so is this the .qmail-default or the user? Remo Dave Cook wrote: Hi Remo: Thanks for the response. Yes, I have that file. For myself as an example: .qmail-dcook and it contains: | /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Sunday, December 03, 2006 7:35 PM Subject: Re: [vchkpw] Removng Duplicate Email !DSPAM:4573737c283213607818837! -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.409 / Virus Database: 268.15.6/565 - Release Date: 12/2/2006
Re: [vchkpw] Removng Duplicate Email
why do u have a user and a default with the same info? try to remove the user's one and see if u get mail and if it gets dubs.. Just a suggestion.. My 2 cents Dave Cook wrote: Hi Remo: It's the user default file in /home/vpopmail/domains/mydomain.com/ I also have a .qmail-default file in that same directory with the same contents. Cheers, Dave - Original Message - From: Remo Mattei [EMAIL PROTECTED] To: vchkpw@inter7.com Sent: Sunday, December 03, 2006 8:51 PM Subject: Re: [vchkpw] Removng Duplicate Email !DSPAM:45739d0b199767320219403!
Re: [vchkpw] Removng Duplicate Email
Hi Remo:
That was just a left-over from testing. I removed the user-default and I
still get dups. That's where I started this morning when the dup thing
started.
Getting back to the eliminate-dups script.
1) I have the following in .qmail-delivery: (in
/home/vpopmail/domains/mydomain.com/)
| /usr/local/sbin/eliminate-dups mydomain.com duphash
[EMAIL PROTECTED]
2) I have the following in .qmail-default: (in
/home/vpopmail/domains/mydomain.com/)
| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox
3) The eliminate-dups script is here:
#! /usr/bin/perl
#--
# Copyright 2006 Russell Nelson [EMAIL PROTECTED]
# This program is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#--
#
#
# Modified for use with vpopmail
# by Chris Hardie [EMAIL PROTECTED]
# originally by Russell Nelson, http://www.qmail.org/eliminate-dups
#
#
# Now, if delivery to the mbox is deferred, eliminate-dups will NOT be
# run a second time for the same message.
#
# Set up ~vpopmail/domains/domain.com/.qmail-default as follows:
#
# | /home/vpopmail/bin/vdelivermail '' [EMAIL PROTECTED]
#
# Then create a ~vpopmail/domains/domain.com/.qmail-delivery file as
follows:
#
# |bin/eliminate-dups domain.com duphash
# [EMAIL PROTECTED]
#
# Now, if delivery to the mbox is deferred, eliminate-dups will NOT be
# run a second time for the same message.
my $basedir = /home/vpopmail/domains;
my $domainname = shift;
my $hname = shift;
my $hashname = $basedir/$domainname/$hname;
use Digest::MD5;
$md5 = new Digest::MD5;
$loose = 1; # loose matching if set.
while() {
last if /^$/;
next if $ignore_continue /^\s/;
$ignore_continue = 0;
if (/^received:/i) {
$ignore_continue = 1;
next;
}
if (!$loose) {
$headers .= $_;
next;
}
if ($keep_continue /^\s/) {
$headers .= $_;
next;
}
$keep_continue = 0;
if (m/^(from|message-id|date):/i) {
$headers .= $_;
$keep_continue = 1;
next;
}
next;
}
$md5-add($headers);
$md5-addfile(STDIN);
$hash = $md5-hexdigest;
print $headers Our hash:$hash\n;
if (open(HASH, $hashname.newer)) {
flock(HASH, 2);
while(HASH) { chomp; exit 99 if $_ eq $hash; }
}
open(HASH, $hashname.older) || die $0: Cannot open $hashname.older;
while(HASH) { chomp; exit 99 if $_ eq $hash; }
# roll the files once a week.
if (-M $hashname.older 7) {
rename($hashname.newer, $hashname.older) || die $0: Unable to
move newer to older;
}
# add the hash to the received messages list.
open(HASH, $hashname.newer) || die $0: Cannot append to
$hashname.newer;
print HASH $hash\n;
close(HASH);
print Original message;
exit 0;
a) Putting | /home/vpopmail/bin/vdelivermail '' [EMAIL PROTECTED] in
the .qmail-default file stops email delivery and doesn't work.
b) This file is owned by vpopmail.vchkpw and is executable and is in my
/usr/local/sbin directory.
Question: Does vpopmail read .qmail files in the user's Maildirs? (i.e.
/home/vpopmail/domains/mydomain.com/dcook)
I need some direction on where these files should reside, if it actually
works and how I can test it. Any help greatly appreciated.
Cheers,
Dave
- Original Message -
From: Remo Mattei [EMAIL PROTECTED]
To: vchkpw@inter7.com
Sent: Sunday, December 03, 2006 11:21 PM
Subject: Re: [vchkpw] Removng Duplicate Email
why do u have a user and a default with the same info? try to remove the
user's one and see if u get mail and if it gets dubs..
Just a suggestion..
My 2 cents
Dave Cook wrote:
Hi Remo:
It's the user default file in /home/vpopmail/domains/mydomain.com/ I
also have a .qmail-default file in that same directory with the same
contents.
Cheers,
Dave
- Original Message - From: Remo Mattei [EMAIL PROTECTED]
To: vchkpw@inter7.com
Sent: Sunday, December 03, 2006 8:51 PM
Subject: Re: [vchkpw] Removng Duplicate Email
!DSPAM:45739d0b199767320219403!
--
No virus found in this incoming message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.15.6/565 - Release Date: 12/2/2006
Re: [vchkpw] Removng Duplicate Email
so u get dub when mail comes in? do u use maildrop? if you also use
other filter that may cause the problem.
Remo
Dave Cook wrote:
Hi Remo:
That was just a left-over from testing. I removed the user-default and
I still get dups. That's where I started this morning when the dup
thing started.
Getting back to the eliminate-dups script.
1) I have the following in .qmail-delivery: (in
/home/vpopmail/domains/mydomain.com/)
| /usr/local/sbin/eliminate-dups mydomain.com duphash
[EMAIL PROTECTED]
2) I have the following in .qmail-default: (in
/home/vpopmail/domains/mydomain.com/)
| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox
3) The eliminate-dups script is here:
#! /usr/bin/perl
#--
# Copyright 2006 Russell Nelson [EMAIL PROTECTED]
# This program is free software; you can redistribute it and/or
# modify it under the same terms as Perl itself.
#--
#
#
# Modified for use with vpopmail
# by Chris Hardie [EMAIL PROTECTED]
# originally by Russell Nelson, http://www.qmail.org/eliminate-dups
#
#
# Now, if delivery to the mbox is deferred, eliminate-dups will NOT be
# run a second time for the same message.
#
# Set up ~vpopmail/domains/domain.com/.qmail-default as follows:
#
# | /home/vpopmail/bin/vdelivermail '' [EMAIL PROTECTED]
#
# Then create a ~vpopmail/domains/domain.com/.qmail-delivery file as
follows:
#
# |bin/eliminate-dups domain.com duphash
# [EMAIL PROTECTED]
#
# Now, if delivery to the mbox is deferred, eliminate-dups will NOT be
# run a second time for the same message.
my $basedir = /home/vpopmail/domains;
my $domainname = shift;
my $hname = shift;
my $hashname = $basedir/$domainname/$hname;
use Digest::MD5;
$md5 = new Digest::MD5;
$loose = 1; # loose matching if set.
while() {
last if /^$/;
next if $ignore_continue /^\s/;
$ignore_continue = 0;
if (/^received:/i) {
$ignore_continue = 1;
next;
}
if (!$loose) {
$headers .= $_;
next;
}
if ($keep_continue /^\s/) {
$headers .= $_;
next;
}
$keep_continue = 0;
if (m/^(from|message-id|date):/i) {
$headers .= $_;
$keep_continue = 1;
next;
}
next;
}
$md5-add($headers);
$md5-addfile(STDIN);
$hash = $md5-hexdigest;
print $headers Our hash:$hash\n;
if (open(HASH, $hashname.newer)) {
flock(HASH, 2);
while(HASH) { chomp; exit 99 if $_ eq $hash; }
}
open(HASH, $hashname.older) || die $0: Cannot open $hashname.older;
while(HASH) { chomp; exit 99 if $_ eq $hash; }
# roll the files once a week.
if (-M $hashname.older 7) {
rename($hashname.newer, $hashname.older) || die $0: Unable
to move newer to older;
}
# add the hash to the received messages list.
open(HASH, $hashname.newer) || die $0: Cannot append to
$hashname.newer;
print HASH $hash\n;
close(HASH);
print Original message;
exit 0;
a) Putting | /home/vpopmail/bin/vdelivermail '' [EMAIL PROTECTED] in
the .qmail-default file stops email delivery and doesn't work.
b) This file is owned by vpopmail.vchkpw and is executable and is in
my /usr/local/sbin directory.
Question: Does vpopmail read .qmail files in the user's Maildirs? (i.e.
/home/vpopmail/domains/mydomain.com/dcook)
I need some direction on where these files should reside, if it actually
works and how I can test it. Any help greatly appreciated.
Cheers,
Dave
- Original Message - From: Remo Mattei [EMAIL PROTECTED]
To: vchkpw@inter7.com
Sent: Sunday, December 03, 2006 11:21 PM
Subject: Re: [vchkpw] Removng Duplicate Email
!DSPAM:4573b2d0307555258618536!
