On Wednesday, October 20, 2004, 4:15:09 PM, Michiel wrote: MP> What we did was write a wrapper around sniffer, and fire that wrapper from MP> the Content Filter. that wrapper measures how long each sniffer instance MP> takes. In the previous version, it took way longer when using the persistent MP> version than when not using the persistent version. You would expect it to MP> be the other way around.
The only caution I have about this is that it's only true in a single threaded scenario. On systems that use multiple threads (IMail/Declude, Postfix, Qmail, etc...) one might see long times for single client instances, but many more of them might be completed within a given period. Think of it as a wider road rather than a higher speed limit. In a single threaded environment or one where the number of client instances is highly constrained then there should definitely be an improvement in the time sniffer runs. -- The key to the faster times is a better coordination between the persistent server and the client instance. Here's how that works: In the newest test versions (currently i8) the persistent server advertises it's presence by publishing a .stat file. This reduces overhead by allowing the client instances to avoid scanning the workspace for .SVR files thus reducing overhead. In addition to this savings, the persistent server publishes the best delay for the client(s) to use before checking to see if their job is complete. This ensures that the client is always using an optimal delay. On a busy single threaded system this delay would be roughly equal to the time it took to process the last job (usually a few milliseconds). When the first process is complete the next one should start almost immediately and it will be picked up by the server instance quickly. In theory (and some testing of course) this has the effect of creating a synchronized pipeline between the mail server, sniffer client, and sniffer persistent server. As long as there is a continuous flow of messages each will be processed as quickly as each can be handed off and scanned. In a multithreaded environment there is an additional optimization. In the previous versions a persistent server instance would scan for a job and would stop at the first one that it found. As a result, the entire directory would be scanned for each job that was processed. This is because the persistent server was simply a normal peer-server instance "tricked" into staying alive for long periods. In the latest test versions a persistent server scans the entire workspace once and picks up all of the available jobs (.QUE files) in that pass. Then it processes all of the jobs before scanning again. As a result the workspace directory is scanned far less frequently on busy systems and all additional overhead associated with that process is eliminated. While a single job may see a longer processing time (outside measurement vs sniffer log) many more jobs can be processed in a given period so the throughput is much better. MP> I could try the new version tomorrow to see if this one is actually faster, MP> but if I don't get around to doing it tomorrow, I can't check it anymore, MP> coz I'm going down under for a month. Hopefully you'll get to this -- I'd love to have the data. In any event, have a great trip! _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
