[sniffer] Re: gbx size
also, do you keep stats of the messages collected by your robots ? and do you know what they resulted in ? - Original Message - From: Serge se...@cefib.com To: Message Sniffer Community sniffer@sortmonster.com Sent: Tuesday, June 16, 2009 5:16 PM Subject: [sniffer] gbx size Hello I have a arge increase (x2) of my .gbx file this coincide with me automaticaly routing hi weightFN to snniffer pop box for your robots to pick. Is the 2 above issues related ? if not, why the increase ? if yes, can this result in FP ? what are the consequences, and how long is the effects ? TIA -Serge # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: gbx size
Serge wrote: Hello I have a arge increase (x2) of my .gbx file this coincide with me automaticaly routing hi weightFN to snniffer pop box for your robots to pick. Is the 2 above issues related ? That is very unlikely. I see a few things in your telemetry. You are currently seeing a large number of new IPs. SNF does not appear to remain alive for a full day at a time-- so it never condenses your GBUdb data. That in itself is not a problem as long as you have room in RAM for the data. If you want GBUdb to condense once a day as designed, either allow SNFServer to stay running continuously or set your GBUdb condensation time trigger to a shorter interval than 1 day -- perhaps 10-30 minutes shorter presuming you reboot once per day or something like that. Alternatively you could activate the size trigger and set it near the current size -- or a size you prefer if the 150M default is not appropriate for your system. (You need about twice that much when condensation takes place because a second copy of GBUdb is used to perform the opperation and prevent interference with active scans). Your current GBUdb data size is 83,8Mbytes: timers run started=20090616010038 elapsed=60089/ sync latest=20090616174127 elapsed=40/ save latest=20090616170414 elapsed=2273/ condense latest=1970010100 elapsed=1245174127/ /timers − gbudb size bytes=83886080/ records count=335281/ utilization percent=93.2544/ /gbudb − if not, why the increase ? Most likely you have begun receiving a lot of messages from a new bot net and the new IPs are being added to your GBUdb data. GBUdb will grow as needed within the limits set on your system. The default is about 150Mbytes. if yes, can this result in FP ? Again-- the two issues are not related. Also, GBUdb growth cannot cause false positives. what are the consequences, and how long is the effects ? GBUdb size will grow until it is condensed. SInce your system does not allow SNFServer to run continuously GBUdb will condense when it reaches it's maximum allowed size. You can adjust this if you wish. When GBUdb does condense the size may drop temporarily, but the size will remain roughly stable. If GBUdb were to condense daily as designed then the size might change more frequently and would be related to the number of IPs that are actively communicating with your system over time. Hope this helps, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: gbx size
thanks for the explanation we moved to new hw and are still fine tunning so we do reboot more than once a day what does does condensing do ? something like compressing the file ? or deleting IPs ? if the later, on what criterias ? - Original Message - From: Pete McNeil madscient...@armresearch.com To: Message Sniffer Community sniffer@sortmonster.com Sent: Tuesday, June 16, 2009 5:55 PM Subject: [sniffer] Re: gbx size Serge wrote: Hello I have a arge increase (x2) of my .gbx file this coincide with me automaticaly routing hi weightFN to snniffer pop box for your robots to pick. Is the 2 above issues related ? That is very unlikely. I see a few things in your telemetry. You are currently seeing a large number of new IPs. SNF does not appear to remain alive for a full day at a time-- so it never condenses your GBUdb data. That in itself is not a problem as long as you have room in RAM for the data. If you want GBUdb to condense once a day as designed, either allow SNFServer to stay running continuously or set your GBUdb condensation time trigger to a shorter interval than 1 day -- perhaps 10-30 minutes shorter presuming you reboot once per day or something like that. Alternatively you could activate the size trigger and set it near the current size -- or a size you prefer if the 150M default is not appropriate for your system. (You need about twice that much when condensation takes place because a second copy of GBUdb is used to perform the opperation and prevent interference with active scans). Your current GBUdb data size is 83,8Mbytes: timers run started=20090616010038 elapsed=60089/ sync latest=20090616174127 elapsed=40/ save latest=20090616170414 elapsed=2273/ condense latest=1970010100 elapsed=1245174127/ /timers − gbudb size bytes=83886080/ records count=335281/ utilization percent=93.2544/ /gbudb − if not, why the increase ? Most likely you have begun receiving a lot of messages from a new bot net and the new IPs are being added to your GBUdb data. GBUdb will grow as needed within the limits set on your system. The default is about 150Mbytes. if yes, can this result in FP ? Again-- the two issues are not related. Also, GBUdb growth cannot cause false positives. what are the consequences, and how long is the effects ? GBUdb size will grow until it is condensed. SInce your system does not allow SNFServer to run continuously GBUdb will condense when it reaches it's maximum allowed size. You can adjust this if you wish. When GBUdb does condense the size may drop temporarily, but the size will remain roughly stable. If GBUdb were to condense daily as designed then the size might change more frequently and would be related to the number of IPs that are actively communicating with your system over time. Hope this helps, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: gbx size
Serge wrote: also, do you keep stats of the messages collected by your robots ? and do you know what they resulted in ? We keep the messages in our database for a while so that we (bots and people) can do additional research. Individual messages don't result in anything specific usually. We create rules based on what we are seeing in general and individual messages contribute to that analysis -- often providing fragments and useful structural information. Please let me know if I've properly answered your questions. Thanks, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
[sniffer] Re: gbx size
Serge wrote: thanks for the explanation we moved to new hw and are still fine tunning so we do reboot more than once a day what does does condensing do ? something like compressing the file ? or deleting IPs ? if the later, on what criterias ? Condensing is a way for GBUdb to forget about that past. Here is a link about that: http://www.armresearch.com/support/articles/technology/GBUdb/maintainence.jsp Here is a link about configuring your GBUdb condensation triggers: http://www.armresearch.com/support/articles/software/snfServer/config/node/gbudb/database/condense/index.jsp Best, _M # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: sniffer-...@sortmonster.com To switch to the DIGEST mode, E-mail to sniffer-dig...@sortmonster.com To switch to the INDEX mode, E-mail to sniffer-in...@sortmonster.com Send administrative queries to sniffer-requ...@sortmonster.com
