Sam, you're welcome.
Do you think we should go after the "Not a directory" and "File exist" errors, or is that something you can eventually duplicate locally and circumvent? ------------------------ Erald Troja Sam Clippinger wrote: > I received the logs; thanks. > > I see the problem -- in the two logs you sent, the two different remote > servers are identifying their senders using the following statements: > MAIL FROM:<-@> SIZE=555 > MAIL FROM:<@> SIZE=474 > Presumably they're trying to indicate that the sender address is empty > because the message is system-generated (probably a bounce message). > However, the correct way give an empty address is like this: > MAIL FROM:<> > According to my reading of the RFCs, using the at symbol (@) without a > domain name is not valid. Trust spambots to come up with new ways to > break the rules. > > spamdyke's parser is being confused by the extra (illegal) characters > between the angle brackets, so it's ignoring them and using the "SIZE" > parameter as the sender's address. This shouldn't be very hard to fix; > I'll get right on it. > > Thanks for reporting this! > > -- Sam Clippinger > > Erald Troja wrote: >> Sam, >> >> i'm ready when you are. >> >> How do I contact you privately? >> >> I got 2 such occurrences. >> >> Thanks. >> >> >> ------------------------ >> Erald Troja >> >> >> Sam Clippinger wrote: >> >>> I've tried a bunch of different ideas but I'm not having any success >>> trying to make the graylist filter produce "size_XXXX" files. spamdyke >>> should ignore the "size" parameter when the sender address is given. >>> >>> If this is happening as frequently as your logs show, could you enable >>> full logging (with "full-log-dir") and capture one of these message >>> deliveries? (You can send the log file to me privately if you don't >>> want the data on the list.) I'd love to find a way to reproduce this >>> problem and fix it. >>> >>> -- Sam Clippinger >>> >>> Erald Troja wrote: >>> >>>> Sam, >>>> >>>> ever since that incident, the only ERRORs >>>> that we're getting are the "File exists" with >>>> some sporadic "Is a directory" ERRORs >>>> >>>> We've so far been unable to duplicate the "Not a directory" ERRORs >>>> yet we are still able to find files starting with 'size' keyword inside >>>> the graylist directory. >>>> >>>> Today we found one more such file namely 'size_1003' onto one of our the >>>> graylist directories. The entry in the maillog is as shown >>>> >>>> Sep 30 08:09:18 mail01 spamdyke[2584]: DENIED_GRAYLISTED from: size=1003 >>>> to: [EMAIL PROTECTED] origin_ip: 98.135.205.165 origin_rdns: >>>> h165.205.135.98.ip.windstream.net auth: (unknown) >>>> >>>> >>>> I don't have a way to find the headers, or know what was retried >>>> to be delivered as all we have in the log files are entries such as >>>> >>>> /var/log/maillog.1.bz2:Sep 30 04:20:51 mail01 spamdyke[23810]: >>>> DENIED_GRAYLISTED from: size=483 >>>> /var/log/maillog.1.bz2:Sep 30 04:27:53 mail01 spamdyke[18932]: >>>> DENIED_GRAYLISTED from: size=382 >>>> /var/log/maillog.1.bz2:Sep 30 04:32:53 mail01 spamdyke[27422]: >>>> DENIED_GRAYLISTED from: size=469 >>>> /var/log/maillog.1.bz2:Sep 30 04:33:33 mail01 spamdyke[28849]: >>>> DENIED_GRAYLISTED from: size=454 >>>> /var/log/maillog.1.bz2:Sep 30 04:54:09 mail01 spamdyke[3211]: >>>> DENIED_GRAYLISTED from: size=534 >>>> /var/log/maillog.1.bz2:Sep 30 05:06:50 mail01 spamdyke[25643]: >>>> DENIED_GRAYLISTED from: size=978 >>>> /var/log/maillog.1.bz2:Sep 30 07:57:23 mail01 spamdyke[10831]: >>>> DENIED_GRAYLISTED from: size=974 >>>> /var/log/maillog.1.bz2:Sep 30 08:08:29 mail01 spamdyke[1073]: >>>> DENIED_GRAYLISTED from: size=593 >>>> /var/log/maillog.1.bz2:Sep 30 08:09:18 mail01 spamdyke[2584]: >>>> DENIED_GRAYLISTED from: size=1003 >>>> /var/log/maillog.1.bz2:Sep 30 08:14:35 mail01 spamdyke[12471]: >>>> DENIED_GRAYLISTED from: size=511 >>>> /var/log/maillog.1.bz2:Sep 30 08:56:35 mail01 spamdyke[27126]: >>>> DENIED_GRAYLISTED from: size=517 >>>> /var/log/maillog.1.bz2:Sep 30 09:30:36 mail01 spamdyke[29039]: >>>> DENIED_GRAYLISTED from: size=479 >>>> >>>> We tried a recursive search for each IP which has a 'size=' from >>>> entry, and found none to be retried again, making it such impossible >>>> to find out full headers. >>>> >>>> Note, that from the above occurrences where the from address shows as >>>> 'size=' only the very above log entry had indeed a file called 'size_1003' >>>> >>>> I am note sure if they are related. >>>> >>>> >>>> >>>> >>>> >>>> ------------------------ >>>> Erald Troja >>>> >>>> >>>> Sam Clippinger wrote: >>>> >>>> >>>>> If you could search for the first entries showing "DENIED_GRAYLISTED" >>>>> for the recipient address that is having problems >>>>> ([EMAIL PROTECTED]), they should show what the sender's address >>>>> was. That address may have been parsed incorrectly, so knowing what >>>>> value spamdyke produced would be valuable. If you have the real >>>>> messages that were finally delivered after the graylist filter >>>>> passed/failed, it would be handy to compare the correct address to >>>>> spamdyke's interpretation. >>>>> >>>>> -- Sam Clippinger >>>>> >>>>> Erald Troja wrote: >>>>> >>>>> >>>>>> Sam, >>>>>> >>>>>> We keep for two weeks and we might still have the logs. >>>>>> >>>>>> What exactly would you like me to revert to you with? >>>>>> >>>>>> Thanks. >>>>>> ------------------------- >>>>>> Erald Troja >>>>>> [EMAIL PROTECTED] >>>>>> 646.528.6671 >>>>>> >>>>>> -----Original Message----- >>>>>> From: Sam Clippinger <[EMAIL PROTECTED]> >>>>>> >>>>>> Date: Sat, 27 Sep 2008 21:56:46 >>>>>> To: spamdyke users<[email protected]> >>>>>> Subject: Re: [spamdyke-users] Errors in my log files regarding >>>>>> directory/file creation >>>>>> >>>>>> >>>>>> How long do you save log files? If you've only been running spamdyke >>>>>> for a couple of weeks, could you search your logs to find the first >>>>>> entries for these addresses that are causing problems now? I'm >>>>>> particularly concerned about the "size_447" and "size_583" files -- they >>>>>> could represent a problem with spamdyke's address parser. I'd really >>>>>> like to figure out how the remote server sent a recipient address that >>>>>> was so badly parsed. >>>>>> >>>>>> -- Sam Clippinger >>>>>> >>>>>> Erald Troja wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Sam, >>>>>>> >>>>>>> We're using http://www.spamdyke.org/releases/spamdyke-4.0.4.tgz >>>>>>> >>>>>>> We never tried Spamdyke before 2 weeks, so 4.0.4 is the sole >>>>>>> version we've ever tried. >>>>>>> >>>>>>> Thanks. >>>>>>> ------------------------ >>>>>>> Erald Troja >>>>>>> >>>>>>> >>>>>>> Sam Clippinger wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> What version of spamdyke are you running right now? Were these files >>>>>>>> (that should be directories) created by an older version of spamdyke >>>>>>>> or >>>>>>>> by the version you are now using? >>>>>>>> >>>>>>>> -- Sam Clippinger >>>>>>>> >>>>>>>> Erald Troja wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> Sam, >>>>>>>>> >>>>>>>>> thanks for the reply. I did run with config-test option and I'm >>>>>>>>> seeing >>>>>>>>> quite a few errors. >>>>>>>>> >>>>>>>>> Here's some facts. >>>>>>>>> >>>>>>>>> 1)/usr/local/bin/spamdyke is set with 755 perms and it's owned by >>>>>>>>> root:root >>>>>>>>> >>>>>>>>> 2)/var/tmp/spamdyke.graylist.d/ is set with 755 and it's >>>>>>>>> vpopmai:vchkpw >>>>>>>>> ownership >>>>>>>>> >>>>>>>>> 3)any directory within /var/tmp/spamdyke.graylist.d/ is set with 700 >>>>>>>>> and >>>>>>>>> vpopmail:vchkpw >>>>>>>>> >>>>>>>>> 4)my calling line in qmail init script is >>>>>>>>> tcpserver -v $RRDNSKEY -R -c $TCP_SERVERS $IPLIMIT >>>>>>>>> $RELAYCHKARG -u $USER_VPOPMAIL -g $GROUP_VCHKPW 0 smtp $RBL $SPAMDYKE >>>>>>>>> qmail-smtpd vchkpw t >>>>>>>>> rue cmd5checkpw true 2>&1 | splogger smtpd & >>>>>>>>> >>>>>>>>> all in one line. >>>>>>>>> >>>>>>>>> As far as I can tell the permissions are set properly. >>>>>>>>> >>>>>>>>> Here's some more discoveries/facts >>>>>>>>> >>>>>>>>> Here's an entry onto the maillog files >>>>>>>>> >>>>>>>>> /var/log/maillog.1.bz2:Sep 25 16:11:02 mail01 spamdyke[18977]: ERROR: >>>>>>>>> cannot write to graylist file >>>>>>>>> /var/tmp/spamdyke.graylist.d/mydomainname.com/webmaster/barb.com/york: >>>>>>>>> >>>>>>>>> Not a directory >>>>>>>>> >>>>>>>>> /var/log/maillog.1.bz2:Sep 25 16:11:02 mail01 spamdyke[18977]: >>>>>>>>> ALLOWED >>>>>>>>> from: [EMAIL PROTECTED] to: [EMAIL PROTECTED] origin_ip: >>>>>>>>> 89.231.87.134 origin_rdns: host-89-231-87-134.opoczno.mm.pl auth: >>>>>>>>> (unknown) >>>>>>>>> >>>>>>>>> >>>>>>>>> Turns out >>>>>>>>> /var/tmp/spamdyke.graylist.d/mydomainname.com/webmaster/barb.com >>>>>>>>> >>>>>>>>> is indeed created as a file, when in turn it should have been created >>>>>>>>> as a directory. >>>>>>>>> >>>>>>>>> Also, i'm finding miscellaneous files such as size_447 or size_583 >>>>>>>>> inside the /var/tmp/spamdyke.graylist.d/mydomainname.com/webmaster >>>>>>>>> directory for one and others as well. >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the headers from the spam message in FULL. >>>>>>>>> ------------------------------------------------------ >>>>>>>>> Return-Path: <[EMAIL PROTECTED]> >>>>>>>>> Delivered-To: [EMAIL PROTECTED] >>>>>>>>> Received: (qmail 19015 invoked by uid 399); 25 Sep 2008 16:11:02 -0400 >>>>>>>>> X-Spam-Checker-Version: SpamAssassin 3.1.4 (2006-07-25) on localhost >>>>>>>>> X-Spam-Level: *** >>>>>>>>> X-Spam-Status: No, score=3.4 required=4.0 tests=HELO_DYNAMIC_IPADDR >>>>>>>>> autolearn=disabled version=3.1.4 >>>>>>>>> X-Virus-Scan: Scanned by clamdmail 0.15 (no viruses); >>>>>>>>> Thu, 25 Sep 2008 16:11:02 -0400 >>>>>>>>> Received: from unknown (HELO host-89-231-87-134.opoczno.mm.pl) >>>>>>>>> (89.231.87.134) >>>>>>>>> by mail01.myserver.com with SMTP; 25 Sep 2008 16:11:02 -0400 >>>>>>>>> Received-SPF: none (mail01.myserver.com: domain at barb.com does not >>>>>>>>> designate permitted sender hosts) >>>>>>>>> identity=mailfrom; client-ip=89.231.87.134; >>>>>>>>> envelope-from=<[EMAIL PROTECTED]>; >>>>>>>>> Message-ID: <[EMAIL PROTECTED]> >>>>>>>>> From: =?koi8-r?B?7snLz8zByiD+xcLP1MHSxdc=?= <[EMAIL PROTECTED]> >>>>>>>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> >>>>>>>>> Subject: =?koi8-r?B?98HbwSDcxsbFy9TJ187B0SDSxcvMwc3BLg==?= >>>>>>>>> Date: Thu, 25 Sep 2008 18:23:44 +0000 >>>>>>>>> MIME-Version: 1.0 >>>>>>>>> Content-Type: text/plain; >>>>>>>>> charset="koi8-r" >>>>>>>>> Content-Transfer-Encoding: 8bit >>>>>>>>> X-Priority: 3 >>>>>>>>> X-MSMail-Priority: Normal >>>>>>>>> X-Mailer: Microsoft Outlook Express 6.00.2720.3000 >>>>>>>>> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2727.1300 >>>>>>>>> ----------------------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> Can anyone point out where the permission issue might be? >>>>>>>>> >>>>>>>>> We're using ext3 file system with blocks=1k >>>>>>>>> >>>>>>>>> config-test shows many 'Not a directory' ERROR warnings. >>>>>>>>> >>>>>>>>> Please advise. >>>>>>>>> >>>>>>>>> >>>>>>>>> ------------------------ >>>>>>>>> Erald Troja >>>>>>>>> >>>>>>>>> >>>>>>>>> Sam Clippinger wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> Something is wrong with the permissions on your graylist folders. >>>>>>>>>> spamdyke is not able to see that files exist or it's not able to >>>>>>>>>> tell >>>>>>>>>> what type of files they are (i.e. directories or regular files). If >>>>>>>>>> the >>>>>>>>>> folder permissions look correct, it could be a filesystem problem -- >>>>>>>>>> I've had to do some special coding for spamdyke on XFS filesystems >>>>>>>>>> in >>>>>>>>>> the past. You may be able to get more information about what's >>>>>>>>>> happening with spamdyke's "config-test" option. >>>>>>>>>> >>>>>>>>>> When the graylist filter encounters errors like this, spamdyke just >>>>>>>>>> skips the graylist filter. The message is processed normally, just >>>>>>>>>> as >>>>>>>>>> if the graylist filter was not enabled. You might receive more spam >>>>>>>>>> as >>>>>>>>>> a result but you shouldn't lose any email. >>>>>>>>>> >>>>>>>>>> -- Sam Clippinger >>>>>>>>>> >>>>>>>>>> Erald Troja wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> Greetings folks, >>>>>>>>>>> >>>>>>>>>>> fairly new to Spamdyke and we're running on a minimal >>>>>>>>>>> configuration such as the one below >>>>>>>>>>> >>>>>>>>>>> log-level=info >>>>>>>>>>> graylist-level=always-create-dir >>>>>>>>>>> graylist-dir=/var/tmp/spamdyke.graylist.d >>>>>>>>>>> graylist-exception-ip-file=/etc/spamdyke/whitelist.conf >>>>>>>>>>> ##all will be graylisted for 15 minutes initial attempt >>>>>>>>>>> graylist-min-secs=900 >>>>>>>>>>> ##whoever passes graylisting can send for 24 hours >>>>>>>>>>> graylist-max-secs=86400 >>>>>>>>>>> reject-unresolvable-rdns=true >>>>>>>>>>> reject-empty-rdns=true >>>>>>>>>>> connection-timeout-secs=2400 >>>>>>>>>>> idle-timeout-secs=240 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> OS is centos 4.6 final and we're utilizing Hsphere qmail binaries >>>>>>>>>>> from >>>>>>>>>>> Psoft. >>>>>>>>>>> >>>>>>>>>>> We're utilizing 1k blocks on /var/tmp to reduce directory size. >>>>>>>>>>> >>>>>>>>>>> We've noticed error such as the ones below on our maillog which is >>>>>>>>>>> a concern. >>>>>>>>>>> >>>>>>>>>>> mail01 spamdyke[7232]: ERROR: unable to create directory >>>>>>>>>>> /var/tmp/spamdyke.graylist.d/domain.com/user/fromdomain.com: File >>>>>>>>>>> exists >>>>>>>>>>> >>>>>>>>>>> mail01 spamdyke[24535]: ERROR: cannot write to graylist file >>>>>>>>>>> /var/tmp/spamdyke.graylist.d/domain.com/user/fromdomain.com/windsor: >>>>>>>>>>> Not >>>>>>>>>>> a directory >>>>>>>>>>> >>>>>>>>>>> I've replaced original domains hosted with us with domain.com and >>>>>>>>>>> sending party domains with fromdomain.com >>>>>>>>>>> >>>>>>>>>>> There's plenty of disk space left on the /var/tmp partition. >>>>>>>>>>> >>>>>>>>>>> 1.Main question is, why might such be caused and how to avoid it? >>>>>>>>>>> >>>>>>>>>>> 2.Also what is defined in Spamdyke to happen to such email, is it >>>>>>>>>>> lost, >>>>>>>>>>> is it retried or? >>>>>>>>>>> >>>>>>>>>>> Thanks and blessings to all involved >>>>>>>>>>> with Spamdyke >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> spamdyke-users mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> spamdyke-users mailing list >>>>>>>>> [email protected] >>>>>>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> spamdyke-users mailing list >>>>>>>> [email protected] >>>>>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> _______________________________________________ >>>>>>> spamdyke-users mailing list >>>>>>> [email protected] >>>>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> spamdyke-users mailing list >>>>>> [email protected] >>>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>>> _______________________________________________ >>>>>> spamdyke-users mailing list >>>>>> [email protected] >>>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> spamdyke-users mailing list >>>>> [email protected] >>>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>>> >>>>> >>>>> >>>> _______________________________________________ >>>> spamdyke-users mailing list >>>> [email protected] >>>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>>> >>>> >>> _______________________________________________ >>> spamdyke-users mailing list >>> [email protected] >>> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >>> >>> >> _______________________________________________ >> spamdyke-users mailing list >> [email protected] >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > _______________________________________________ > spamdyke-users mailing list > [email protected] > http://www.spamdyke.org/mailman/listinfo/spamdyke-users > _______________________________________________ spamdyke-users mailing list [email protected] http://www.spamdyke.org/mailman/listinfo/spamdyke-users
