http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
------- Additional Comments From [EMAIL PROTECTED] 2007-08-08 09:58 ------- > This change of network lookups timeouts dramatically increases the time > of scan. > 3.2.1: 333 seconds with skip_rbl_checks=0; 146 seconds skip_rbl_checks=1 > 3.2.2: 643 seconds with skip_rbl_checks=0; 168 seconds skip_rbl_checks=1 Well, you can either have results quick but miss many, or let DNS responses be harvested in reasonable time, which will increase the total elapsed time somewhat, but produce more accurate results. Btw, latency is usually not a problem, as it can be easily compensated by increasing the number of parallel processes, yielding the same maximal throughput the machine can handle. To remind ourselves what is the core problem here: some asynchronous DNS lookup requests carry a completion routine which then spawns additional DNS queries. This happens currently in Plugin::URIDNSBL, but could in principle happen in any completion routine. For example, a lookup on a NS record spawns an additional lookup for an A record, but not before harvest_dnsbl_queries happens, which is quite late, so not much time remains for collecting responses to these new queries. Same goes for URIDNS lookups such as sbl.spamhaus.org, cn-kr.blackholes.us etc. What happened before the patch is that most of these additional lookup results were simply discarded if other rules took long time to complete (longer than rbl_timeout), which was a real waste of good information. A solution is in processing completion routines (which may spawn new queries) much sooner and gradually, as responses trickle in, and not wait until priority 500, which is almost at the end of mail processing. I have a patch (for 3.2.2) which accomplishes this. It is rather substantial (but compatible with the rest of the code), touching AsyncLoop.pm mostly, but also Dns.pm, DnsResolver.pm, Plugin/Check.pm and Plugin/URIDNSBL.pm. It is certainly not something that could go into 3.2 series. I'll let it simmer here on our production machine for a couple of days, just in case, then make it available. Btw, is trunk in C-T-R mode now? Do I open a new bug entry, post a patch, then commit to trunk in few days without a vote (barring objections of course)? ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
