[sniffer] Re: xci scanner command
A question on GBUDB utilization. I show a current utilization of 95% (from the log file) which I assume means the amount of memory used from what is set aside for gbudb entries. Is that correct? What happens when more entries are added? Does the GBUdb grow or does it get pruned out? Will gbudb XCI commands (like bad) show up in a log file? On Fri, Feb 13, 2009 at 3:27 PM, Pete McNeil madscient...@armresearch.comwrote: Richard Stupek wrote: So there would not be a real benefit to passing the IP over when it is the is already in the mail having been added by the mail server? Correct. The vast majority of the time a properly configured SNF + GBUdb can learn the original source of the IP even if you have multiple gateways in your infrastructure. You can even teach SNF + GBUdb to learn to see past the infrastructure of other ISPs in many cases. For example you might teach it to see past a DSL provider's outbound servers so that it can map IP reputations based on individual message sources on their network provided they include Received headers you can understand and predict (to some extent). This way GBUdb can provide pinpoint accuracy instead of a rough average of every source on that network. That said, there are still some times where you might want to explicitly define the source IP even if it is present in the Received headers. For example, one of our larger customers has a complex infrastructure. They found that it was easier to explicitly provide the source IP than to train SNF + GBUdb to understand their structure and the inevitable changes that go on through time. Another large customer has developed a very complex system for determining the precise original source for a message even when it is relayed through many large ISPs. They chose to provide that IP rather than have SNF + GBUdb attempt to duplicate that effort. _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: xci scanner command
Richard Stupek wrote: A question on GBUDB utilization. I show a current utilization of 95% (from the log file) which I assume means the amount of memory used from what is set aside for gbudb entries. Is that correct? Yes. What happens when more entries are added? Does the GBUdb grow or does it get pruned out? Both. When more space is needed the ram allocated to GBUdb will grow to accommodate the need. This should happen infrequently. When entries are no longer active they are dropped to make room for additional entries. In practice the GBUdb data size stabilizes quickly and then doesn't change much. Once per day or as otherwise specified by GBUdb condensation triggers the GBUdb data will be condensed. This means that all of the good and bad event counts are divided in half. This has the effect of retaining the ratios (probability of spam) while reducing the event counts (confidence figure). Any ugly entries that drop to zero are dropped from the system (forgotten). The remaining live entries are placed in a new GBUdb and the old one is thrown away. If the GBUdb size can shrink during a condensation run then it will. Will gbudb XCI commands (like bad) show up in a log file? No. These are administrative entries so they don't get reported in scan activity. We may change this later but at present there are no requests for it. Best, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: xci scanner command
Thanks for the info. Is there any diagnostic information available when a gbudb sync occurs? On Tue, Feb 17, 2009 at 4:35 PM, Pete McNeil madscient...@armresearch.comwrote: Richard Stupek wrote: A question on GBUDB utilization. I show a current utilization of 95% (from the log file) which I assume means the amount of memory used from what is set aside for gbudb entries. Is that correct? Yes. What happens when more entries are added? Does the GBUdb grow or does it get pruned out? Both. When more space is needed the ram allocated to GBUdb will grow to accommodate the need. This should happen infrequently. When entries are no longer active they are dropped to make room for additional entries. In practice the GBUdb data size stabilizes quickly and then doesn't change much. Once per day or as otherwise specified by GBUdb condensation triggers the GBUdb data will be condensed. This means that all of the good and bad event counts are divided in half. This has the effect of retaining the ratios (probability of spam) while reducing the event counts (confidence figure). Any ugly entries that drop to zero are dropped from the system (forgotten). The remaining live entries are placed in a new GBUdb and the old one is thrown away. If the GBUdb size can shrink during a condensation run then it will. Will gbudb XCI commands (like bad) show up in a log file? No. These are administrative entries so they don't get reported in scan activity. We may change this later but at present there are no requests for it. Best, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: xci scanner command
Richard Stupek wrote: Thanks for the info. Is there any diagnostic information available when a gbudb sync occurs? You can always see the current status of GBUdb in your status.* files. If you append these logs you can follow the state of the system through time using pre-compiled statistics including the size of GBUdb. This can be done with one entry per minute (status.minute) or one entry per hour (status.hour) or both. In the same status log you can see when the last GBUdb condensation event occurred. No other useful data is produced by a condensation run so none is recorded. If there is something you would like to see then please let us know and we will consider adding features to support your request(s). Best, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: xci scanner command
A question about using the XCI bad command. Assume an email passes through sniffer and does not trigger any rules, I then run it through and determine it is in fact spam. I send a bad command to let sniffer know the IP address had a bad event. Won't the good event that would occur due the spam passing through nullify the bad event? Should I post 2 bad events for each mail that is caught after sniffer? On Tue, Feb 17, 2009 at 5:02 PM, Pete McNeil madscient...@armresearch.comwrote: Richard Stupek wrote: Thanks for the info. Is there any diagnostic information available when a gbudb sync occurs? You can always see the current status of GBUdb in your status.* files. If you append these logs you can follow the state of the system through time using pre-compiled statistics including the size of GBUdb. This can be done with one entry per minute (status.minute) or one entry per hour (status.hour) or both. In the same status log you can see when the last GBUdb condensation event occurred. No other useful data is produced by a condensation run so none is recorded. If there is something you would like to see then please let us know and we will consider adding features to support your request(s). Best, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com