[sniffer] Re: Updates to log rotation scripts
Hello tfox, Wednesday, October 10, 2007, 9:24:57 PM, you wrote: Thanks, John, for clarifying my question. That's exactly what I meant! I assume additional, more detailed documentation is coming soon that details more of what is required to effectively set everything up... A few lines in a text file for a piece of software as powerful and complicated as Sniffer really makes me nervous, particularly when the Wiki isn't updated either. Documentation is an ongoing project. More is on the way. Did you see this update to the wiki: http://kb.armresearch.com/index.php?title=Message_Sniffer.TechnicalDetails.GBUdb Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Updates to log rotation scripts
Hello Richard, Wednesday, October 10, 2007, 9:58:24 PM, you wrote: When just running it in a cmd window for example, it sure would be nice to know what all those numbers mean. but yeah, the documentation is sparse. Its at least easy to get running in a very basic configuration though. The command line monitor shows from left to right: M/min - Messages Per Minute SP - Capture Rate (Spam Percentage) LR - Latest matched rule ID, indicates how current the rulebase is. [Jobs/Loops Queue] - XCI server stats this second: Jobs = Number of requests processed last second. Loops = Number of times the listener was polled for jobs last second. Queue = Number of jobs queued in the processing channels. W - White/minute. GBUdb overrides to whitelist a pattern matched msg. C - Cautions/minute. GBUdb overrides to blacklist a clean msg. B - Blacks/minute. GBUdb overrides to blacklist a clean msg. T - Truncates/minute. Truncated messages per minute. S - Samples/minute. Virtual spamtrap samples per minute. W, C, B, T, and S events are described here: http://kb.armresearch.com/index.php?title=Message_Sniffer.TechnicalDetails.GBUdb The same information (except for XCI stats) and more can be found in the status reports: licenseid.status.second licenseid.status.minute licenseid.status.hour Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Updates to log rotation scripts
Ok, I have not changed anything in the installation relative to log files, so I can assume, then, that Sniffer is sending you the data you need. Accordingly, I've removed my logrotate script. What files should I be seeing constantly updated in my sniffer directory? I assume additional, more detailed documentation is coming soon that details more of what is required to effectively set everything up... A few lines in a text file for a piece of software as powerful and complicated as Sniffer really makes me nervous, particularly when the Wiki isn't updated either. Documentation is an ongoing project. More is on the way. Did you see this update to the wiki: http://kb.armresearch.com/index.php?title=Message_Sniffer.Tech nicalDetails.GBUdb --- [This E-mail scanned for viruses by Declude Virus] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Updates to log rotation scripts
I think he was asking about the log rotate script that also FTPs a copy up to sniffer. Do we still need to FTP a log to Sniffer? John T -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Tuesday, October 09, 2007 9:28 PM To: Message Sniffer Community Subject: [sniffer] Re: Updates to log rotation scripts Hello tfox, Tuesday, October 9, 2007, 10:23:46 PM, you wrote: What updates/file name changes would be necessary for the log rotation scripts? It is possible to generate old style log files from the new version if you wish. Your current scripts can be used as-is in that case. Hopefully you will be able to make the switch to the new XML based logs. Both log types can be rotated daily by the new engine. Specifically, today's date can be prepended to the log file names. How can we monitor the status of SNF in real time, via the XML pages? The first answer is that the new engine produces a number of status reports - every second, every minute, or every hour. These status reports and logs, though formatted as XML, have been designed to be relatively easy to see in a simple text editor. It does take a little bit of getting used to - but not too much. Is there such a thing as an XML reader? Yep. Your web browser. Just about every web browser can read and translate XML data these days. The trick is -- translate how? You may want to use an XSLT utility, or more likely the XSLT capabilities in your web server environment or even in your web browser alone. For example, you could take one of the status files, copy it to a new file. Add a few lines of text - specifically to add a style-sheet definition and document type so that the XML is complete. Then you should be able to open the resulting file in your favorite browser. (You will have to create an XSL file (style sheet) to translate the XML file into what you want to see.) [[ This is the approach I used to create the rate chart shown in nowSimplePrescale.png, then I moved the whole thing to our web server to make it more automatic. ]] Another way you might go is to import the XML from the log or status report into a database. (Here again you may want/need to prepend a line or two of text to make the XML completely compatible with your environment) Then you would be able to extract reports from your database in the usual way. We're hopeful that folks who are savvy about XML and XSL will create and share useful translations and tools for SNF users. We look forward to supporting that effort. Internally we've done a few quick things to watch the telemetry we get from SNF nodes and our own servers. The approach we've taken is to use the inherent XSLT capabilities of our web/jsp servers and the basic capabilities in IE and Firefox. Attached are some screen shots of live data I am looking at right now. This telemetry comes from one of our spamtrap pre-filters. nowSimplePrescale.png uses a simple XSL file that took me about 20 minutes to throw together while thumbing through a text book. nowNodeDashbaord.png took a bit more work and leverages a flash based live gauge tool that periodically pulls xml data from our internal servers (so it's animated). The flash gadget came from here: http://www.maani.us/gauge/ We will also be creating some monitoring tools and services on our web site to take advantage of the live data provided by the new SNF engine and some of our new back-end tools. If anyone creates any useful XSL, tools, etc then please let us know and we will be happy to post them on our site and create appropriate reciprocal links. Hope this helps, _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Updates to log rotation scripts
Hello John, Wednesday, October 10, 2007, 6:15:18 PM, you wrote: I think he was asking about the log rotate script that also FTPs a copy up to sniffer. Do we still need to FTP a log to Sniffer? When you are running the new engine it is not necessary to upload log files. We collect rulebase activity and effectiveness data directly from the telemetry. _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Updates to log rotation scripts
Thanks, John, for clarifying my question. That's exactly what I meant! I assume additional, more detailed documentation is coming soon that details more of what is required to effectively set everything up... A few lines in a text file for a piece of software as powerful and complicated as Sniffer really makes me nervous, particularly when the Wiki isn't updated either. Thanks, Tom Hello John, Wednesday, October 10, 2007, 6:15:18 PM, you wrote: I think he was asking about the log rotate script that also FTPs a copy up to sniffer. Do we still need to FTP a log to Sniffer? When you are running the new engine it is not necessary to upload log files. We collect rulebase activity and effectiveness data directly from the telemetry. _M --- [This E-mail scanned for viruses by Declude Virus] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer] Re: Updates to log rotation scripts
When just running it in a cmd window for example, it sure would be nice to know what all those numbers mean. but yeah, the documentation is sparse. Its at least easy to get running in a very basic configuration though. On Oct 10, 2007, at 9:24 PM, [EMAIL PROTECTED] wrote: Thanks, John, for clarifying my question. That's exactly what I meant! I assume additional, more detailed documentation is coming soon that details more of what is required to effectively set everything up... A few lines in a text file for a piece of software as powerful and complicated as Sniffer really makes me nervous, particularly when the Wiki isn't updated either. Thanks, Tom Hello John, Wednesday, October 10, 2007, 6:15:18 PM, you wrote: I think he was asking about the log rotate script that also FTPs a copy up to sniffer. Do we still need to FTP a log to Sniffer? When you are running the new engine it is not necessary to upload log files. We collect rulebase activity and effectiveness data directly from the telemetry. _M --- [This E-mail scanned for viruses by Declude Virus] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to sniffer- [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] [This E-mail scanned for viruses by Declude] [This E-mail scanned for viruses by Declude] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
