[sniffer] Re: RulePanic on 3741490

[Pete McNeil]

On 1/7/2011 2:00 PM, Darin Cox wrote: Hmmmm....   "Update notifications happen as soon as the rulebase compilers have created a new rulebase."   I don't know what your internal processes are, but if I understand this correctly the rule was created at 5:39am ET, and was compiled into the rulebase somewhere just before 8:53am ET, at which point update notifications were sent.   From the customer point of view, when the rule was created or removed doesn't really matter, and those times are meaningless to us.  What matters is when the rulebases that include them are published/updated, as that is what we key off of for updates. The rulebase compiler that was responsible for your update would have performed it's queries to collect rules for folding. For some time after that the compilers would crunch on that data in order to create the folded token matrix. Both the query and folding operations take time. There are many compilers and many, many rulebase files. The system is adaptive so the times required to perform these operations changes constantly. All that by way of saying that you are essentially correct, but also that there is no single answer we can give that would describe the case for every customer. The rule was removed at about 0853. Some customers rulebase compilation began immediately after that event, others before. All took some time to complete. Part of our rework is to reduce the time required for both phases of this operation so that on average less time is required for a change to appear in active SNF systems. The goal is ultimately to make that as close to instantaneous as possible without incurring performance, availability, scalability, and reliability penalties.   "We have features on the short list to automatically render removed rules inert in near real-time (within seconds)"   Sounds good.  That would definitely be better than notifications for us to be able to put in RulePanics, assuming there's no negative effect to overall performance from checking each rule for active/inactive state.  I assume some sort of push mechanism to all subscribers, to notify their systems that a rule is no longer valid, is what you're planning here. The implementation that is currently planned would operate something like our system making temporary entries into your rule-panic system. Those entries would survive long enough to ensure that your active rulebase contains those changes before the entries expire. Wherever possible we like to leverage tried and true mechanisms (like the rule-panic entry system) when producing new functionality. Also, wherever possible we like to engineer facilities that can be leveraged in multiple ways in future. It's a planning heavy process, but one that pays off in better reliability and greater overall flexibility. (IMO). Best, _M -- Pete McNeil Chief Scientist ARM Research Labs, LLC www.armresearch.com 866-770-1044 x7010 ############################################################# This message is sent to you because you are subscribed to the mailing list <sniffer@sortmonster.com>. This list is for discussing Message Sniffer, Anti-spam, Anti-Malware, and related email topics. For More information see http://www.armresearch.com To unsubscribe, E-mail to: <sniffer-...@sortmonster.com> To switch to the DIGEST mode, E-mail to <sniffer-dig...@sortmonster.com> To switch to the INDEX mode, E-mail to <sniffer-in...@sortmonster.com> Send administrative queries to <sniffer-requ...@sortmonster.com>