Re: [Declude.JunkMail] Beta 3.0.4.4 update 09/24/05A

[Matt]

John, The problem was fixed where the Queue Manager run could steal files from Declude.  I believe this was IMail's issue.  It definitely hasn't been an issue on my server for about 2 years, and we run our queue every 10 minutes on high volume. I have noted Q files stranded in my spool that were renamed.  This seems to happen mostly when the server has been overloaded for long periods of time, but I'm sure that it also has happened on normal volume.  That alone should be evidence of the need to handle errors more gracefully.  Maybe they need to be renamed within the proc folder as well even though there is no chance of them being stolen.  Maybe they are being renamed already (I haven't tried the beta).  Regardless, an active monitoring (as opposed to passive monitoring) function seems necessary.  That function could also recycle the service occasionally to further prevent the effects of leaks. Pete seems to have gotten the service part of Sniffer perfect on his first try, though he didn't have issues with managing a spool or the files themselves, and he doesn't need to restart the service if it fails.  He also has a single executable that runs as a service and as a command line application.  I'm wondering if it would be better if decludeproc.exe was folded into declude.exe.  IMail calls the handler application, and that can be relied upon, and a single executable might then be able to do the active monitoring of the service with graceful error recovery.  I suppose that decludeproc.exe could do this also, I don't know.  I could very well be oversimplifying as I am totally unaware of the issues that necessitated the change to a service in the first place, and of course I am not deeply familiar with the inner workings of their application at this level. Matt John Tolmachoff (Lists) wrote: Declude renamed the files in the spool from smd to ~md, not Imail. When Imail was receiving, the Q was T until the entire D file was received and created. The problem also came when the Queue Manger did a normal timed run every 30 minutes or so depending on configuration.   Yes, the file was as is and the only thing changed was the local domain.   John T eServices For You   -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Saturday, September 24, 2005 3:40 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Beta 3.0.4.4 update 09/24/05A   John, The idea regarding the restart is for decludeproc to handle the process itself.  Windows isn't always successful at restarting something, and sometimes things enter a bad state and you need something to actively monitor the process to make sure that it functions. The old Declude doesn't have any regular operational issues with IMail stealing files (only happens when you restart the queue manager out of order with IMail SMTP).  If you look in the spool on IMail, you will see files named something like _D0A58F27011E822C.~MD.  I think that this is IMail that is locking the files while the D* file is being received.  If I am correct, old Declude places a lock on the Q* file (or maybe it's the D* file), and this is what keeps the IMail Queue Manager from stealing them.  The trick here is that if Declude crashes, the file should be released eventually, and Declude won't reprocess it unless Declude crashed when the file was in overflow. When Declude takes either the D* or Q* file out of the spool, it has to take responsibility for handling the E-mail gracefully, and that includes recovery from errors.  It might have something in there already that handles this, but there is also the new issue of a crash in Declude stopping delivery until it is restarted, so decludeproc needs to be able to monitor and start the service.  Sniffer does something similar, but instead of restarting the service, it just continues to operate as a command line application. I did test one of the E-mails that you provided and it was successfully processed by Declude 2.0.6.16, so the error appears to be new to the 3.0 versions as long as I can trust that the formating of these files was maintained accurately as it was passed to the list and received by me. On a related note, it would be really, really helpful if we knew what types of things in Declude were modified when things go into a full release.  Knowing what areas might have been modified, such as maybe the message parsing routines, can help a great deal with troubleshooting.  I think that after more than a year under the new system we know well enough to expect that despite the best intentions, there are often issues, and I expect fully for this to continue.  E-mail is unbelievably important to many companies, even more so important than the telephone due to it's ability to transmit data.  If you interrupt that process it can cost people business, their jobs and it can even result in a lawsuit.  This isn't need-to-know information, it is absolutely necessary information that administrators have to know.  The community is fairly good as a whole at sharing this information, but we definitely don't detect every issue, or share every known issue.  I personally have a list that I am not bothering to share with Declude because they are busy dealing with much greater issues at the moment (supporting IMail 8.2+ users). Matt John Tolmachoff (Lists) wrote: Matt, I have the Decludeproc set to restart in the services manager, so it does get restarted. However, everytime it would restart, it would try running the file again and crash and so on. In the mean time, all the other messages stacked up.   I have no problem having the OS restart the service each time. I do like the idea the files are moved out of the spool so that Imail can not touch them.   What I do like is your idea of renaming, and that could be done in the work folder. Once decludeproc starts to process a message in the work folder, it appends the extension to say .proc and then when finished renames it back. Then program decludeproc to not start processing a message that ends in .proc. That way, if decludeproc does crash and restart, it will not try to process the messages that end in .proc. You could then write a script that looks for .proc files in the work folder older than say 30 minutes and send a notice, or move them and then send a notice.   John T eServices For You   -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of Matt Sent: Saturday, September 24, 2005 2:32 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Beta 3.0.4.4 update 09/24/05A   John, Good find on this one.  Curiously, I searched my Hold folder for examples of this and could find none.  I would expect to have seen some of these unless possibly the patterns are well tagged and scored too high.  We have pretty high volume over here. I'm speculating here, but this might suggest something that could cause Declude to crash.  In the old days where these files were mostly stored in the spool, Declude could crash gracefully.  Now with Declude moving everything out of the spool, if it crashes, it will then start up and crash again.  I believe that this is probably not a good way to construct the product if at all possible. In the older Decludes the crash recovery was automatic unless a message landed in overflow that could cause it to crash.  I would recommend that Declude keep files in the spool if at all possible.  This means instead of moving Q files to a proc folder, they just be locked and left in the spool folder.  That way the chances of a crash taking down the system will be much less. I consider this to be very, very important if Declude is going to start running as a service.  You can't create the possibility of a crash that can repeatedly take down the service if it is avoidable.  If this crash was caused by Declude not being able to properly parse a malformed message, there are likely many other ways to also crash it even if this one issue is resolved.  This might also be the reason why I am not seeing these in my Hold file...Declude 2.0.6.16 might be crashing when trying them, but it isn't a service so it chugs on. There are other possible solutions besides keeping things in the spool.  One might also be changing the name of the Q file as a first step, and not bothering to reprocess files that don't appear with their original name.  Naturally I'm not totally sure about how everything works, so I'm sure that there could be even other much better ways to resolve this. Declude as a service also could use a mechanism for restarting itself when called by IMail so that crashes are self-recoverable.  I would imagine that if decludeproc.exe is called and it isn't running as a service, it should be able to start itself. Matt John Tolmachoff (Lists) wrote: Through a time consuming process, I have apparently identified the trigger for the problem with the decludeproc stopping and messages backing up.   Attached are text files which are D files for messages that caused this. The 2 times in the last 4 days that the decludeproc has stopped. Each time, one of these files was in the work folder. After disabling the decludeproc, moving the entire contents of the work folder to a temp folder, then feeding one pair of files to the proc folder at a time, I was able to isolate it to these. Each one of the 3 is identical in the included characters and body formatting.   I would ask that any one else having this problem please check and see if you have any of these kind of files stuck in the work folder.   If you need any help in figuring this out, please let me know.   If we can all confirm this, Declude will be able to fix the problem that faster.   John T eServices For You                   Received: from mail.bddtb.com [218.187.44.83] by mail.eservicesforyou.net   (SMTPD-8.20) id A77A0614; Fri, 23 Sep 2005 15:53:46 -0700 Received: (qmail 31553 invoked by uid 540); 24 Sep 2005 00:55:52 +0900 Date: 24 Sep 2005 00:55:52 +0900 Message-ID: <[EMAIL PROTECTED]> From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject:  $B%-%d%s%Z!<%sCGB37hDj(B!   $B!z!z!z1|MM%i%s%A(B10000$B1_%-%d%s%Z!<%s!z!z!z(B   $B$49%I>D:$$$F$$$k1|MM%i%s%A%-%d%s%Z!<%s(B $B;R6!C#$N2F5Y$_$b=*$o$j$h$j$$$C$=$&3hF0E*$J?M:JC#!y(B $B$=$s$J?M:JC#$K(B100pt$B!\EEOCG'>Z!a(B900pt$B!a(B10000$B1_L5NA%]%$%s%H$r;[EMAIL PROTECTED](B http://www.blazeblow.net/?num=0303 $B2F$N4V;R6!C#$N2F5Y$_$N0Y<+J,$N;~4V$,;W$&$h$&$K<h$l$J$+$C$?(B $B1|MMC#$,5^>e>:EPO?Cf(B!!! ------------------------------------------------------------ $BL5NA%]%$%s%H$r>e<j$K;H$C$F$*9%$_$N1|MM$r$_$D$1$F(B! $B1|MM(B&$B$=$l0J30(B($BL$:'(B)[EMAIL PROTECTED];~EPO?<u$1IU$1Cf(B http://www.blazeblow.net/?num=0303                   $BG[?.5qH]$O$3$A$i$N%"%I%l%9$^$G$*4j$$$7$^$9!#(B $B!!!!!!(B       $B"-"-"-"-(B    $B!!!!!!([EMAIL PROTECTED] ----------------------------------------------------------- Please inform the following address if this mail is unnecessary. [EMAIL PROTECTED]                 Received: from mail.bvvnb.com [218.187.44.117] by mail.eservicesforyou.net   (SMTPD-8.20) id AFDA0394; Fri, 23 Sep 2005 17:37:46 -0700 Received: (qmail 31945 invoked by uid 509); 24 Sep 2005 02:08:29 +0900 Date: 24 Sep 2005 02:08:29 +0900 Message-ID: <[EMAIL PROTECTED]> From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject:  $B%-%d%s%Z!<%sCGB37hDj(B!   $B!z!z!z1|MM%i%s%A(B10000$B1_%-%d%s%Z!<%s!z!z!z(B   $B$49%I>D:$$$F$$$k1|MM%i%s%A%-%d%s%Z!<%s(B $B;R6!C#$N2F5Y$_$b=*$o$j$h$j$$$C$=$&3hF0E*$J?M:JC#!y(B $B$=$s$J?M:JC#$K(B100pt$B!\EEOCG'>Z!a(B900pt$B!a(B10000$B1_L5NA%]%$%s%H$r;[EMAIL PROTECTED](B http://www.blazeblow.net/?num=0303 $B2F$N4V;R6!C#$N2F5Y$_$N0Y<+J,$N;~4V$,;W$&$h$&$K<h$l$J$+$C$?(B $B1|MMC#$,5^>e>:EPO?Cf(B!!! ------------------------------------------------------------ $BL5NA%]%$%s%H$r>e<j$K;H$C$F$*9%$_$N1|MM$r$_$D$1$F(B! $B1|MM(B&$B$=$l0J30(B($BL$:'(B)[EMAIL PROTECTED];~EPO?<u$1IU$1Cf(B http://www.blazeblow.net/?num=0303                   $BG[?.5qH]$O$3$A$i$N%"%I%l%9$^$G$*4j$$$7$^$9!#(B $B!!!!!!(B       $B"-"-"-"-(B    $B!!!!!!([EMAIL PROTECTED] ----------------------------------------------------------- Please inform the following address if this mail is unnecessary. [EMAIL PROTECTED]                 Received: from mail.ikura-chan.com [218.187.44.145] by mail.eservicesforyou.net   (SMTPD-8.20) id A58C0294; Fri, 16 Sep 2005 16:30:20 -0700 Received: (qmail 27998 invoked by uid 509); 17 Sep 2005 00:07:47 +0900 Date: 17 Sep 2005 00:07:47 +0900 Message-ID: <[EMAIL PROTECTED]> From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject: $B1|MM%i%s%A(B!   $B!z!z!z(B10.000$B1_%-%c%s%Z!<%s<B;\Cf!z!z!z(B   $B1|MM%i%s%A$G$OB~:#MD:J!A;O$^$j<c:J!&[EMAIL PROTECTED]>$X(B10.000$B1_J,L5NA%"%]<h$j(B($B?)$YJ|Bj(B) $B%-%c%s%Z!<%s<B;\Cf!@(B(^O^)$B!?(B $B!!!!!!!!!!(Bhttp://www.blazeblow.net/?num=0308 $B#O#P#E#N$7$F4V$b$J$$?7;2<T$N%5%$%H$G$9$,O7J^%5%$%H$KDI$$$D$1DI$$1[$;$r%b%C%H!<[EMAIL PROTECTED];o$K9-9p5-:\(B($B8=7QB35-:\Cf(B) $B$N$*$+$2$G8=;[EMAIL PROTECTED]@-$N?t$,<c43B?$$EPO?>u67$G$9!#(B   $B2F$N$3$N;~4|K;$7$=$&$J1|MM(B!$B$G$b:G6a$O$G$O3F<+$GM=Dj$rF~$l$k;~Be(B!$B1|MM$K$b2F!y$OFCJL$J5(@a(By(^$B!<(B^)y$B3Z$7$_"v$r5a$a$F1|MMJ}$,(B $BB3!9EPO?Cf(B! $B!!!!!!!!!!(Bhttp://www.blazeblow.net/?num=0308   10.000$B1_J,L5NA$r>e<j$K;H$C$F1|MM(B($BEvA3L$:'[EMAIL PROTECTED](B)$B$H3Z$7$$"vAGE($J=P2q$$$r$_$D$1$F2<$5$$!#(B   $BCm(B)$B=P2q$$$rA0Ds$K$7$?%5%$%H$N0Y(B10.000$B1_L5NA$O0-5:Ey$rKI$00Y(B $B$KHV9fG'>Z$,I,MW$H$J$j$^$9!#(B                 $BG[?.5qH]$O$3$A$i$N%"%I%l%9$^$G$*4j$$$7$^$9!#(B $B!!!!!!(B       $B"-"-"-"-(B    $B!!!!!!([EMAIL PROTECTED] ----------------------------------------------------------- Please inform the following address if this mail is unnecessary. [EMAIL PROTECTED]