Re: [sniffer] Version 2-3.0i8 published.
I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. I run Mdaemon and I proces about 10K message a day. What things should I look for? How can I know the performance has actually improved? -- Jorge Asch Revilla CONEXION DCR www.conexion.co.cr 800-CONEXION This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] Version 2-3.0i8 published.
On Wednesday, October 20, 2004, 12:19:12 PM, Jorge wrote: I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. JA I run Mdaemon and I proces about 10K message a day. What things should I JA look for? How can I know the performance has actually improved? That's a good question. 10K/day is not very large so you may not see an improvement easily. What you are looking for is higher throughput, or more precisely, that your system is able to process more messages in a given period of time. Systems with heavier loads _should_ see a reduction in their backlog. Systems that have periods of heavy activity should see the these peaks handled more quickly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Version 2-3.0i8 published.
Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog See a reduction of what in their backlog? Can you give an example of how to see this type of measurement? Thanks -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, October 20, 2004 11:44 AM To: Jorge Asch Subject: Re[2]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:19:12 PM, Jorge wrote: I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. JA I run Mdaemon and I proces about 10K message a day. What things JA should I look for? How can I know the performance has actually improved? That's a good question. 10K/day is not very large so you may not see an improvement easily. What you are looking for is higher throughput, or more precisely, that your system is able to process more messages in a given period of time. Systems with heavier loads _should_ see a reduction in their backlog. Systems that have periods of heavy activity should see the these peaks handled more quickly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail scanned for viruses by Declude Virus] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Version 2-3.0i8 published.
If I might butt in ... If you fire up Task Manager on a windows machine (or your favourite ps tool elsewhere), and set the View, Update Speed to High, then sort by the name in reverse, you will see multiple sniffer.exe and one with a PID that doesn't change. That's your persistent instance. What you should see is that all other instances of sniffer.exe wink in and out, with an ever-changing PID. So far, so good. Under a heavy load, your system is slow enough that you will see multiple sniffer.exe instances, and maybe they are using CPU and maybe not, depending on where the CPU load on your system is coming from. If those multiple instances are all using a small amount of RAM, and only the persistent instance uses 4,000 KB of memory, then the code is working well for you. The client instances are loading the rulebase. If some of those instances also have 4,000 KB of memory allocated, then they have gotten impatient and decided to load the rulebase and process the message for themselves. If you have a lot of impatient clients, they will exacerbate your memory and CPU stress, thus the new 2-3.0i8 code that lets them wait longer. Andrew 8) -Original Message- From: Frank Osako [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 20, 2004 9:54 AM To: [EMAIL PROTECTED] Subject: RE: Re[2]: [sniffer] Version 2-3.0i8 published. Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog See a reduction of what in their backlog? Can you give an example of how to see this type of measurement? Thanks -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, October 20, 2004 11:44 AM To: Jorge Asch Subject: Re[2]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:19:12 PM, Jorge wrote: I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. JA I run Mdaemon and I proces about 10K message a day. What things JA should I look for? How can I know the performance has actually improved? That's a good question. 10K/day is not very large so you may not see an improvement easily. What you are looking for is higher throughput, or more precisely, that your system is able to process more messages in a given period of time. Systems with heavier loads _should_ see a reduction in their backlog. Systems that have periods of heavy activity should see the these peaks handled more quickly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail scanned for viruses by Declude Virus] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Version 2-3.0i8 published.
Whups, I missed out an important NOT in the second-to-last paragraph. Corrected version is below: -Original Message- From: Colbeck, Andrew Sent: Wednesday, October 20, 2004 10:29 AM To: '[EMAIL PROTECTED]' Subject: RE: Re[2]: [sniffer] Version 2-3.0i8 published. If I might butt in ... If you fire up Task Manager on a windows machine (or your favourite ps tool elsewhere), and set the View, Update Speed to High, then sort by the name in reverse, you will see multiple sniffer.exe and one with a PID that doesn't change. That's your persistent instance. What you should see is that all other instances of sniffer.exe wink in and out, with an ever-changing PID. So far, so good. Under a heavy load, your system is slow enough that you will see multiple sniffer.exe instances, and maybe they are using CPU and maybe not, depending on where the CPU load on your system is coming from. If those multiple instances are all using a small amount of RAM, and only the persistent instance uses 4,000 KB of memory, then the code is working well for you. The client instances are NOT loading the rulebase. If some of those instances also have 4,000 KB of memory allocated, then they have gotten impatient and decided to load the rulebase and process the message for themselves. If you have a lot of impatient clients, they will exacerbate your memory and CPU stress, thus the new 2-3.0i8 code that lets them wait longer. Andrew 8) -Original Message- From: Frank Osako [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 20, 2004 9:54 AM To: [EMAIL PROTECTED] Subject: RE: Re[2]: [sniffer] Version 2-3.0i8 published. Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog See a reduction of what in their backlog? Can you give an example of how to see this type of measurement? Thanks -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, October 20, 2004 11:44 AM To: Jorge Asch Subject: Re[2]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:19:12 PM, Jorge wrote: I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. JA I run Mdaemon and I proces about 10K message a day. What things JA should I look for? How can I know the performance has actually JA improved? That's a good question. 10K/day is not very large so you may not see an improvement easily. What you are looking for is higher throughput, or more precisely, that your system is able to process more messages in a given period of time. Systems with heavier loads _should_ see a reduction in their backlog. Systems that have periods of heavy activity should see the these peaks handled more quickly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail scanned for viruses by Declude Virus] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re: Re[2]: [sniffer] Version 2-3.0i8 published.
Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog See a reduction of what in their backlog? Can you give an example of how to see this type of measurement? a reduction of mail waiting for processing in your spool of course. for example, right now, my backlog is at around 800, if this new sniffer works the way they say, we should only have around 200-300 in our backlog, which is normal for emails waiting for resend because a host was unavailable, something like that, we have no installed the new updated as of yet, since our production server moves around 800,000 emails per day. we cannot take any chances. -Ken Thanks -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, October 20, 2004 11:44 AM To: Jorge Asch Subject: Re[2]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:19:12 PM, Jorge wrote: I am particularly interested to hear from MDaemon users who should realize a multi-fold improvement in processing speed by using this new version of persistent server. This is one of the critical goals of these modifications and preliminary responses support that we have achieved this goal. JA I run Mdaemon and I proces about 10K message a day. What things JA should I look for? How can I know the performance has actually improved? That's a good question. 10K/day is not very large so you may not see an improvement easily. What you are looking for is higher throughput, or more precisely, that your system is able to process more messages in a given period of time. Systems with heavier loads _should_ see a reduction in their backlog. Systems that have periods of heavy activity should see the these peaks handled more quickly. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail scanned for viruses by Declude Virus] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html --- [This E-mail scanned for viruses by Declude Virus] --- [This E-mail scanned for viruses by Declude Virus] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[4]: [sniffer] Version 2-3.0i8 published.
On Wednesday, October 20, 2004, 12:54:04 PM, Frank wrote: FO Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog FO See a reduction of what in their backlog? Can you give an example of how FO to see this type of measurement? Another good question - I will try to get a solid, detailed answer. I'm not an MDaemon expert so I'm not sure what the best strategies are for measuring throughput performance and backlog (inbound/outbound queue length). Perhaps there are some MDaemon experts on list that can share their strategies for making these measurements? In particular, how best to measure these things when the system in question is not overloaded? Thanks, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Version 2-3.0i8 published.
If we don't run the Mdaemon on our systems and just use the new download, will we also see a speed increase on processing. Thanks for the time. Keith -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, October 20, 2004 1:50 PM To: Frank Osako Subject: Re[4]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:54:04 PM, Frank wrote: FO Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog FO See a reduction of what in their backlog? Can you give an example FO of how to see this type of measurement? Another good question - I will try to get a solid, detailed answer. I'm not an MDaemon expert so I'm not sure what the best strategies are for measuring throughput performance and backlog (inbound/outbound queue length). Perhaps there are some MDaemon experts on list that can share their strategies for making these measurements? In particular, how best to measure these things when the system in question is not overloaded? Thanks, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Version 2-3.0i8 published.
What we did was write a wrapper around sniffer, and fire that wrapper from the Content Filter. that wrapper measures how long each sniffer instance takes. In the previous version, it took way longer when using the persistent version than when not using the persistent version. You would expect it to be the other way around. I could try the new version tomorrow to see if this one is actually faster, but if I don't get around to doing it tomorrow, I can't check it anymore, coz I'm going down under for a month. Regards, Michiel -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: woensdag 20 oktober 2004 19:50 To: Frank Osako Subject: Re[4]: [sniffer] Version 2-3.0i8 published. On Wednesday, October 20, 2004, 12:54:04 PM, Frank wrote: FO Hello _M _ Systems with heavier loads _should_ see a reduction in their backlog FO See a reduction of what in their backlog? Can you give an example FO of how to see this type of measurement? Another good question - I will try to get a solid, detailed answer. I'm not an MDaemon expert so I'm not sure what the best strategies are for measuring throughput performance and backlog (inbound/outbound queue length). Perhaps there are some MDaemon experts on list that can share their strategies for making these measurements? In particular, how best to measure these things when the system in question is not overloaded? Thanks, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Version 2-3.0i8 published.
But did you run the persistent version also? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jorge Asch Sent: woensdag 20 oktober 2004 22:03 To: [EMAIL PROTECTED] Subject: Re: [sniffer] Version 2-3.0i8 published. If you fire up Task Manager on a windows machine (or your favourite ps tool elsewhere), and set the View, Update Speed to High, then sort by the name in reverse, you will see multiple sniffer.exe and one with a PID that doesn't change. That's your persistent instance. I fired up Task Manager. Could't see Sniffer.EXE nor [mylicense].exe as a persistent instance. Could even see the 'clients'. Funny, since I know it is running (since the logs it's being created, and messages are being sniffed) -- Jorge Asch Revilla CONEXION DCR www.conexion.co.cr 800-CONEXION This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[6]: [sniffer] Version 2-3.0i8 published.
On Wednesday, October 20, 2004, 3:05:35 PM, Keith wrote: KJ If we don't run the Mdaemon on our systems and just use the new KJ download, will we also see a speed increase on processing. Thanks for KJ the time. Yes. The changes that have been made should remove administrative overhead and improve the response time of the persistent server on all systems. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Version 2-3.0i8 published.
Exactly, Michiel. And Jorge, it may be stating the obvious, but you may well have to check the tickbox at the bottom of Task Manager to Show processes from all users. I said sniffer.exe merely as an example, the actual executable will be [your licence here].exe or snfrv2r3.exe if you're using the trial. I use FireDaemon to start my persistent process as a service, so it appears as SYSTEM, whereas my desktop session appears as my Windows username. You can see the username under which each executable is running by pulling down View, Select Columns, and ticking Username. Andrew 8) -Original Message- From: Michiel Prins [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 20, 2004 1:15 PM To: [EMAIL PROTECTED] Subject: RE: [sniffer] Version 2-3.0i8 published. But did you run the persistent version also? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jorge Asch Sent: woensdag 20 oktober 2004 22:03 To: [EMAIL PROTECTED] Subject: Re: [sniffer] Version 2-3.0i8 published. If you fire up Task Manager on a windows machine (or your favourite ps tool elsewhere), and set the View, Update Speed to High, then sort by the name in reverse, you will see multiple sniffer.exe and one with a PID that doesn't change. That's your persistent instance. I fired up Task Manager. Could't see Sniffer.EXE nor [mylicense].exe as a persistent instance. Could even see the 'clients'. Funny, since I know it is running (since the logs it's being created, and messages are being sniffed) -- Jorge Asch Revilla CONEXION DCR www.conexion.co.cr 800-CONEXION This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[6]: [sniffer] Version 2-3.0i8 published.
On Wednesday, October 20, 2004, 4:15:09 PM, Michiel wrote: MP What we did was write a wrapper around sniffer, and fire that wrapper from MP the Content Filter. that wrapper measures how long each sniffer instance MP takes. In the previous version, it took way longer when using the persistent MP version than when not using the persistent version. You would expect it to MP be the other way around. The only caution I have about this is that it's only true in a single threaded scenario. On systems that use multiple threads (IMail/Declude, Postfix, Qmail, etc...) one might see long times for single client instances, but many more of them might be completed within a given period. Think of it as a wider road rather than a higher speed limit. In a single threaded environment or one where the number of client instances is highly constrained then there should definitely be an improvement in the time sniffer runs. -- The key to the faster times is a better coordination between the persistent server and the client instance. Here's how that works: In the newest test versions (currently i8) the persistent server advertises it's presence by publishing a .stat file. This reduces overhead by allowing the client instances to avoid scanning the workspace for .SVR files thus reducing overhead. In addition to this savings, the persistent server publishes the best delay for the client(s) to use before checking to see if their job is complete. This ensures that the client is always using an optimal delay. On a busy single threaded system this delay would be roughly equal to the time it took to process the last job (usually a few milliseconds). When the first process is complete the next one should start almost immediately and it will be picked up by the server instance quickly. In theory (and some testing of course) this has the effect of creating a synchronized pipeline between the mail server, sniffer client, and sniffer persistent server. As long as there is a continuous flow of messages each will be processed as quickly as each can be handed off and scanned. In a multithreaded environment there is an additional optimization. In the previous versions a persistent server instance would scan for a job and would stop at the first one that it found. As a result, the entire directory would be scanned for each job that was processed. This is because the persistent server was simply a normal peer-server instance tricked into staying alive for long periods. In the latest test versions a persistent server scans the entire workspace once and picks up all of the available jobs (.QUE files) in that pass. Then it processes all of the jobs before scanning again. As a result the workspace directory is scanned far less frequently on busy systems and all additional overhead associated with that process is eliminated. While a single job may see a longer processing time (outside measurement vs sniffer log) many more jobs can be processed in a given period so the throughput is much better. MP I could try the new version tomorrow to see if this one is actually faster, MP but if I don't get around to doing it tomorrow, I can't check it anymore, MP coz I'm going down under for a month. Hopefully you'll get to this -- I'd love to have the data. In any event, have a great trip! _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
Re[2]: [sniffer] Version 2-3.0i8 published.
On Wednesday, October 20, 2004, 4:03:15 PM, Jorge wrote: If you fire up Task Manager on a windows machine (or your favourite ps tool elsewhere), and set the View, Update Speed to High, then sort by the name in reverse, you will see multiple sniffer.exe and one with a PID that doesn't change. That's your persistent instance. JA I fired up Task Manager. Could't see Sniffer.EXE nor [mylicense].exe as JA a persistent instance. Could even see the 'clients'. JA Funny, since I know it is running (since the logs it's being created, JA and messages are being sniffed) It definitely has to be there if it's producing a log. Sniffer does nothing to hide itself. Did you try sorting by name? _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
