Re: [CentOS] Log monitoring
On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 Also, current versions of grep have the '-P' flag to allow use of Perl regular expressions for more complex matches. -- Bowie ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
Bowie Bailey wrote: On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 snip Haven't used them, but cactus? splunk? mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
centos-boun...@centos.org wrote: Bowie Bailey wrote: On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 snip Haven't used them, but cactus? splunk? And I think you want -F (not -f) so your tail will follow the file /var/log/messages across logrotates. Insert spiffy .sig here: Life is complex: it has both real and imaginary parts. Life is not measured by the number of breaths we take, but by the moments that take our breath away. //me *** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated** ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
Hi there -- I have been using rsyslog with the LogAnalyzer software to monitor our systems logs. -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Brunner, Brian T. Sent: Wednesday, July 06, 2011 12:07 PM To: CentOS mailing list Subject: Re: [CentOS] Log monitoring centos-boun...@centos.org wrote: Bowie Bailey wrote: On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 snip Haven't used them, but cactus? splunk? And I think you want -F (not -f) so your tail will follow the file /var/log/messages across logrotates. Insert spiffy .sig here: Life is complex: it has both real and imaginary parts. Life is not measured by the number of breaths we take, but by the moments that take our breath away. //me *** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated** ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
Same here, I just recently started using/testing rsyslogd (to mysql [native mysql support is great])+LogAnalyzer web front end for a central log host. So far its been working quite well. Worth checking out Aly Sent from my BlackBerry device on the Rogers Wireless Network ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 03/03/11 1:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product Nagios can probably do all of that. I dunno what you want in those daily/weekly/monthly reports. how many times people logged on and stuff? how many noise packets at your network gateways? the key to any of these systems is configuring the agents to collect the data you want, and deciding whats a security event worthy of an alarm. whether its a commercial system or freeware, you'll be spending a lot of time on that. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 3/3/11 3:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On Thu, Mar 3, 2011 at 2:46 PM, Les Mikesell lesmikes...@gmail.com wrote: On 3/3/11 3:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
After our security team completed POC testing from multiple vendors, we are in the process of implementing LogRhythm in our environment which includes 5000+ servers (Linux, Windows and Solaris). Len Date: Thu, 3 Mar 2011 15:00:53 +0100 From: postnali...@googlemail.com To: centos@centos.org Subject: Re: [CentOS] log monitoring and reporting software On Thu, Mar 3, 2011 at 2:46 PM, Les Mikesell lesmikes...@gmail.com wrote: On 3/3/11 3:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. -- You might want to think about: syslog-ng/rsyslog remote logging + syslog-ng/rsyslog master log receiver + splunk If you find that log messages are getting lost or you need to guarantee that messages arrive you can also consider RELP (supported by rsyslog and possibly by syslog-ng). I actually have experience with writing these types of tools in perl, and found it is not really that hard to do if you have good in-house devops talent at hand. Management and retention of the all that data is the biggest challenge. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
Geoff Galitz wrote: You might want to think about: syslog-ng/rsyslog remote logging + syslog-ng/rsyslog master log receiver + splunk CentOS6 (will) use rsyslog by default and rsyslog is available with CentOS5, so you might want to use rsyslog rather than syslog-ng for CentOS hosts. James Pearson ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 3/3/2011 8:00 AM, Janez Kosmrlj wrote: OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd -- Les Mikesell lesmikes...@gmail.com That's probably not what the OP wanted. Anybody using prelude (http://www.prelude-ids.org)? Rainer ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 3/3/2011 10:22 AM, rai...@ultra-secure.de wrote: It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd That's probably not what the OP wanted. Anybody using prelude (http://www.prelude-ids.org)? If it has to deal with network equipment it won't have access to logs as files anyway - and some syslog handlers can forward the messages if you want both files and real time network processing. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
2011/3/3 Janez Kosmrlj postnali...@googlemail.com: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. syslog + ossec (www.ossec.net) is usually used in high security environments. -- Eero ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
I have deployed LogAnalyzer, and it has been working great in our environment. -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Thursday, March 03, 2011 12:08 PM To: centos@centos.org Subject: Re: [CentOS] log monitoring and reporting software On 3/3/2011 10:22 AM, rai...@ultra-secure.de wrote: It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd That's probably not what the OP wanted. Anybody using prelude (http://www.prelude-ids.org)? If it has to deal with network equipment it won't have access to logs as files anyway - and some syslog handlers can forward the messages if you want both files and real time network processing. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
On Mon, Jan 07, 2008, Joseph L. Casale wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as `messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... We use swatch to monitor various things, mainly security related. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX:(206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 Rights is a fictional abstraction. No one has ``Rights'', neither machines nor flesh-and-blood. Persons... have opportunities, not rights, which they use or do not use. -- Lazarus Long ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
Joseph L. Casale wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as ‘messages’ on your centos servers? I had a hardware failure that happened in between me manually looking (of course…). I would hope it might have a some features to email critical issues etc… Depends on if you're monitoring just one server or a bunch. I'd google for these things: LogWatch epylog big syster oak Then there's various things that read syslog and can read reports for you. Google around for things like syslog-ng, nagios, zenoss, whatnot, if you're looking at larger scope. Jed ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
Bill Campbell wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as `messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... We use swatch to monitor various things, mainly security related. Did you have to do something to it to make it work with centos? I have one running on a machine that collects a lot of router syslogs and it has the annoying habit of resending a bunch of old notifications whenever a new one is noticed. -- Les Mikesell [EMAIL PROTECTED] ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
On Mon, Jan 07, 2008, Les Mikesell wrote: Bill Campbell wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as `messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... We use swatch to monitor various things, mainly security related. Did you have to do something to it to make it work with centos? I have one running on a machine that collects a lot of router syslogs and it has the annoying habit of resending a bunch of old notifications whenever a new one is noticed. Not really. Swatch is pretty straightforward perl, using gnu-tail to watch the end of log file(s). The only issue I've seen is that it will sometimes report old things on occassion when starting if there are matching entries near the end of the files. One place where I used this is on an openldap server that would occassionally get into a ``too many open files'' situation, and swatch would call a routine that restarted slapd when this happened. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX:(206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 Capitalism works primarily because most of the ways that a company can be scum end up being extremely bad for business when there's working competition. -rra ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
