Re: [CentOS] Log rolling with a daemon
On Sat, Dec 21, 2013 at 9:46 PM, Cliff Pratt enkiduonthe...@gmail.com wrote: John's suggestion is still pertinent. You'll need a SIGHUP handler in your script. Logrotate could send the SIGHUP in a postrotate 'script'. Thanks! On Sun, Dec 22, 2013 at 3:15 PM, Larry Martell larry.mart...@gmail.comwrote: On Sat, Dec 21, 2013 at 8:52 PM, John R Pierce pie...@hogranch.com wrote: On 12/21/2013 4:56 PM, Larry Martell wrote: I'm looking for advice or suggestions for rolling log files with a daemon. I have a python script that I daemonized with http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ . Before I daemonized it it was run from a bash script that invoked the underlying python script. It ran the python script, waited for it to complete and then it slept for 5 seconds and ran it again. This was in a infinite loop. In between each invocation it checked the log file and if it was over 10MB it renamed it and then the next invocation started with a new empty log. Since each invocation was a separate run this worked fine. But now the daemonized python script doesn't exit - the same log file is attached to it forever. So my renaming of the file does nothing - the i node doesn't change and it's still logging to the same large file. Anyone have any ideas how I can achieve this sort of log rolling in this situation? send a SIGHUP to syslog and it shoudl re-opent he log files. silly question, but whats wrong with the logrotate daemon thats built into centos? This is not using syslog. If you look at the daemonizing script I gave the link to, you pass in the log files for stdout and stderr, and it does some double fork magic and then associates the given files with them. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Log rolling with a daemon
I'm looking for advice or suggestions for rolling log files with a daemon. I have a python script that I daemonized with http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/. Before I daemonized it it was run from a bash script that invoked the underlying python script. It ran the python script, waited for it to complete and then it slept for 5 seconds and ran it again. This was in a infinite loop. In between each invocation it checked the log file and if it was over 10MB it renamed it and then the next invocation started with a new empty log. Since each invocation was a separate run this worked fine. But now the daemonized python script doesn't exit - the same log file is attached to it forever. So my renaming of the file does nothing - the i node doesn't change and it's still logging to the same large file. Anyone have any ideas how I can achieve this sort of log rolling in this situation? TIA! -lrry ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log rolling with a daemon
On 12/21/2013 4:56 PM, Larry Martell wrote: I'm looking for advice or suggestions for rolling log files with a daemon. I have a python script that I daemonized with http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/. Before I daemonized it it was run from a bash script that invoked the underlying python script. It ran the python script, waited for it to complete and then it slept for 5 seconds and ran it again. This was in a infinite loop. In between each invocation it checked the log file and if it was over 10MB it renamed it and then the next invocation started with a new empty log. Since each invocation was a separate run this worked fine. But now the daemonized python script doesn't exit - the same log file is attached to it forever. So my renaming of the file does nothing - the i node doesn't change and it's still logging to the same large file. Anyone have any ideas how I can achieve this sort of log rolling in this situation? send a SIGHUP to syslog and it shoudl re-opent he log files. silly question, but whats wrong with the logrotate daemon thats built into centos? -- john r pierce 37N 122W somewhere on the middle of the left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log rolling with a daemon
On Sat, Dec 21, 2013 at 8:52 PM, John R Pierce pie...@hogranch.com wrote: On 12/21/2013 4:56 PM, Larry Martell wrote: I'm looking for advice or suggestions for rolling log files with a daemon. I have a python script that I daemonized with http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/. Before I daemonized it it was run from a bash script that invoked the underlying python script. It ran the python script, waited for it to complete and then it slept for 5 seconds and ran it again. This was in a infinite loop. In between each invocation it checked the log file and if it was over 10MB it renamed it and then the next invocation started with a new empty log. Since each invocation was a separate run this worked fine. But now the daemonized python script doesn't exit - the same log file is attached to it forever. So my renaming of the file does nothing - the i node doesn't change and it's still logging to the same large file. Anyone have any ideas how I can achieve this sort of log rolling in this situation? send a SIGHUP to syslog and it shoudl re-opent he log files. silly question, but whats wrong with the logrotate daemon thats built into centos? This is not using syslog. If you look at the daemonizing script I gave the link to, you pass in the log files for stdout and stderr, and it does some double fork magic and then associates the given files with them. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log rolling with a daemon
On 12/21/2013 6:15 PM, Larry Martell wrote: This is not using syslog. If you look at the daemonizing script I gave the link to, you pass in the log files for stdout and stderr, and it does some double fork magic and then associates the given files with them i rarely read links on emails, and am even less likely to do a code analysis for an offhand question unless I'm being paid. -- john r pierce 37N 122W somewhere on the middle of the left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log rolling with a daemon
John's suggestion is still pertinent. You'll need a SIGHUP handler in your script. Logrotate could send the SIGHUP in a postrotate 'script'. Cheers, Cliff On Sun, Dec 22, 2013 at 3:15 PM, Larry Martell larry.mart...@gmail.comwrote: On Sat, Dec 21, 2013 at 8:52 PM, John R Pierce pie...@hogranch.com wrote: On 12/21/2013 4:56 PM, Larry Martell wrote: I'm looking for advice or suggestions for rolling log files with a daemon. I have a python script that I daemonized with http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/ . Before I daemonized it it was run from a bash script that invoked the underlying python script. It ran the python script, waited for it to complete and then it slept for 5 seconds and ran it again. This was in a infinite loop. In between each invocation it checked the log file and if it was over 10MB it renamed it and then the next invocation started with a new empty log. Since each invocation was a separate run this worked fine. But now the daemonized python script doesn't exit - the same log file is attached to it forever. So my renaming of the file does nothing - the i node doesn't change and it's still logging to the same large file. Anyone have any ideas how I can achieve this sort of log rolling in this situation? send a SIGHUP to syslog and it shoudl re-opent he log files. silly question, but whats wrong with the logrotate daemon thats built into centos? This is not using syslog. If you look at the daemonizing script I gave the link to, you pass in the log files for stdout and stderr, and it does some double fork magic and then associates the given files with them. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Log viewing and analysis tools
I have a requirement to allow our security officer to regularly view and analyze the logging and auditing results of one of the machines in our lab. He comes from the Microsoft Windows world and is not a *nix trained person. I know I can configure logwatch. I can also create a script containing various 'aureport' runs into a cron job. Any recommendations for a GUI-based tool that would be easy for him to learn? Dave M ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log viewing and analysis tools
Please check which one suits you more both are web-based Octopussy or loganalyer http://loganalyzer.adiscon.com/ http://sourceforge.net/projects/syslog-analyzer/ On Tue, Aug 28, 2012 at 3:21 PM, David McGuffey davidmcguf...@verizon.netwrote: I have a requirement to allow our security officer to regularly view and analyze the logging and auditing results of one of the machines in our lab. He comes from the Microsoft Windows world and is not a *nix trained person. I know I can configure logwatch. I can also create a script containing various 'aureport' runs into a cron job. Any recommendations for a GUI-based tool that would be easy for him to learn? Dave M ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log viewing and analysis tools
From: David McGuffey davidmcguf...@verizon.net To: centos@centos.org Sent: Tuesday, August 28, 2012 2:51 AM Subject: [CentOS] Log viewing and analysis tools I have a requirement to allow our security officer to regularly view and analyze the logging and auditing results of one of the machines in our lab. He comes from the Microsoft Windows world and is not a *nix trained person. I know I can configure logwatch. I can also create a script containing various 'aureport' runs into a cron job. Any recommendations for a GUI-based tool that would be easy for him to learn? Dave M === Dave: I've been using a free solution called LogAnalyzer, and am pretty happy with it: http://loganalyzer.adiscon.com/ It has a web interface, and uses a database to store all the log info. It can be easily accessed, given specific filtered queries, etc. Check out the Online Demo page to see how it looks. __ If life gives you lemons, keep them-- because hey.. free lemons. ♥ Sticker fixer: http://microflush.org/stuff/stickers/heartFix.html ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log viewing and analysis tools
Need to analyze logs? Have a look at splunk! http://www.splunk.com/ Not sure why anyone would use anything else to tell the truth. :) On Tue, Aug 28, 2012 at 11:49 AM, Joseph Spenner joseph85...@yahoo.comwrote: From: David McGuffey davidmcguf...@verizon.net To: centos@centos.org Sent: Tuesday, August 28, 2012 2:51 AM Subject: [CentOS] Log viewing and analysis tools I have a requirement to allow our security officer to regularly view and analyze the logging and auditing results of one of the machines in our lab. He comes from the Microsoft Windows world and is not a *nix trained person. I know I can configure logwatch. I can also create a script containing various 'aureport' runs into a cron job. Any recommendations for a GUI-based tool that would be easy for him to learn? Dave M === Dave: I've been using a free solution called LogAnalyzer, and am pretty happy with it: http://loganalyzer.adiscon.com/ It has a web interface, and uses a database to store all the log info. It can be easily accessed, given specific filtered queries, etc. Check out the Online Demo page to see how it looks. __ If life gives you lemons, keep them-- because hey.. free lemons. ♥ Sticker fixer: http://microflush.org/stuff/stickers/heartFix.html ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- GPG me!! gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Log monitoring
Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic Thank you Fajar. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 Also, current versions of grep have the '-P' flag to allow use of Perl regular expressions for more complex matches. -- Bowie ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
Bowie Bailey wrote: On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 snip Haven't used them, but cactus? splunk? mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
centos-boun...@centos.org wrote: Bowie Bailey wrote: On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 snip Haven't used them, but cactus? splunk? And I think you want -F (not -f) so your tail will follow the file /var/log/messages across logrotates. Insert spiffy .sig here: Life is complex: it has both real and imaginary parts. Life is not measured by the number of breaths we take, but by the moments that take our breath away. //me *** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated** ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
Hi there -- I have been using rsyslog with the LogAnalyzer software to monitor our systems logs. -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Brunner, Brian T. Sent: Wednesday, July 06, 2011 12:07 PM To: CentOS mailing list Subject: Re: [CentOS] Log monitoring centos-boun...@centos.org wrote: Bowie Bailey wrote: On 7/6/2011 5:37 AM, Fajar Priyanto wrote: Hi all, Currently I do 'tail -f /var/log/messages | grep something' to monitor/tune in my iptables rules. Based on your experience, is there any tools do that better like: - color - grepping multiple keywords - some statistic I don't know about any tools for this, but I did want to point out that grep can handle multiple keywords. $ tail -f /var/log/messages | grep -e keyword1 -e keyword2 -e keyword3 snip Haven't used them, but cactus? splunk? And I think you want -F (not -f) so your tail will follow the file /var/log/messages across logrotates. Insert spiffy .sig here: Life is complex: it has both real and imaginary parts. Life is not measured by the number of breaths we take, but by the moments that take our breath away. //me *** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated** ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log monitoring
Same here, I just recently started using/testing rsyslogd (to mysql [native mysql support is great])+LogAnalyzer web front end for a central log host. So far its been working quite well. Worth checking out Aly Sent from my BlackBerry device on the Rogers Wireless Network ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] log monitoring and reporting software
Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. Thanx ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 03/03/11 1:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product Nagios can probably do all of that. I dunno what you want in those daily/weekly/monthly reports. how many times people logged on and stuff? how many noise packets at your network gateways? the key to any of these systems is configuring the agents to collect the data you want, and deciding whats a security event worthy of an alarm. whether its a commercial system or freeware, you'll be spending a lot of time on that. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 3/3/11 3:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On Thu, Mar 3, 2011 at 2:46 PM, Les Mikesell lesmikes...@gmail.com wrote: On 3/3/11 3:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
After our security team completed POC testing from multiple vendors, we are in the process of implementing LogRhythm in our environment which includes 5000+ servers (Linux, Windows and Solaris). Len Date: Thu, 3 Mar 2011 15:00:53 +0100 From: postnali...@googlemail.com To: centos@centos.org Subject: Re: [CentOS] log monitoring and reporting software On Thu, Mar 3, 2011 at 2:46 PM, Les Mikesell lesmikes...@gmail.com wrote: On 3/3/11 3:12 AM, Janez Kosmrlj wrote: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. -- You might want to think about: syslog-ng/rsyslog remote logging + syslog-ng/rsyslog master log receiver + splunk If you find that log messages are getting lost or you need to guarantee that messages arrive you can also consider RELP (supported by rsyslog and possibly by syslog-ng). I actually have experience with writing these types of tools in perl, and found it is not really that hard to do if you have good in-house devops talent at hand. Management and retention of the all that data is the biggest challenge. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
Geoff Galitz wrote: You might want to think about: syslog-ng/rsyslog remote logging + syslog-ng/rsyslog master log receiver + splunk CentOS6 (will) use rsyslog by default and rsyslog is available with CentOS5, so you might want to use rsyslog rather than syslog-ng for CentOS hosts. James Pearson ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 3/3/2011 8:00 AM, Janez Kosmrlj wrote: OpenNMS is a good snmp monitoring framework with notification/reporting. It doesn't 'collect' logs but you can configure it to receive syslog from other machines and there are a variety of other ways you can pick up data. I'm not sure I'd call it easy to configure, but there are examples on their wiki. http://www.opennms.org It has to collect logs from syslog (or similar service ), because one requirement for certification is log history from all devices in one place. And since we are talking about 1500 devices it should be easy to configure and maintain. It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd -- Les Mikesell lesmikes...@gmail.com That's probably not what the OP wanted. Anybody using prelude (http://www.prelude-ids.org)? Rainer ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
On 3/3/2011 10:22 AM, rai...@ultra-secure.de wrote: It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd That's probably not what the OP wanted. Anybody using prelude (http://www.prelude-ids.org)? If it has to deal with network equipment it won't have access to logs as files anyway - and some syslog handlers can forward the messages if you want both files and real time network processing. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
2011/3/3 Janez Kosmrlj postnali...@googlemail.com: Hi folks, In the company where i work, we are implementing a security standard. A part of this is a log monitoring and reporting software. There are a few requirements, that the software must fulfil: - It must be capable of collecting logs from different devices (Linux machines, network equipment, ...). - it must be capable of sending alarms on security events - it has to generate daily (weekly, monthly) reports - it's a plus if it is easy configurable - it has to have a good support or at least a good community if it is an opensource product So what are you using or at least some recommendations would be nice. An opensource product would be nice, but it's not required. I know i could google it, but it's difficult to decide for a product just from online and marketing presentations. It would be nice to get some real world experience. syslog + ossec (www.ossec.net) is usually used in high security environments. -- Eero ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log monitoring and reporting software
I have deployed LogAnalyzer, and it has been working great in our environment. -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Les Mikesell Sent: Thursday, March 03, 2011 12:08 PM To: centos@centos.org Subject: Re: [CentOS] log monitoring and reporting software On 3/3/2011 10:22 AM, rai...@ultra-secure.de wrote: It doesn't deal with logs as files, but if syslog messages are sent or forwarded to it, it can generate events and notifications from the central configuration. http://www.opennms.org/wiki/Syslogd That's probably not what the OP wanted. Anybody using prelude (http://www.prelude-ids.org)? If it has to deal with network equipment it won't have access to logs as files anyway - and some syslog handlers can forward the messages if you want both files and real time network processing. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log time formats - where is this defined
Albert McCann wrote: -Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Rob Kampen Sent: Sunday, February 27, 2011 3:34 PM To: CentOS mailing list Subject: [CentOS] log time formats - where is this defined One of my servers is using ISO datetime formats (2011-02-27T15:22:15.519857-05:00) in the logs the rest use the default redhat/CentOS format (Feb 27 15:10:21). After a couple of hours searching google I cannot find where this is defined. I know I changed it some months ago as an experiment but forgotten where this was done. the ISO format breaks logwatch - thus I need to revert. I had problems with that, here's the fix: Add to first line of /etc/rsyslog.conf: $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat Exactly my problem - thanks for the heads up - hopefully later versions of logwatch learn how to tell the time. Here's the Bugzilla for this problem: https://bugzilla.redhat.com/show_bug.cgi?id=583621 Al ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos attachment: rkampen.vcf___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log time formats - where is this defined
Spiro Harvey wrote: On Sun, 27 Feb 2011 15:33:57 -0500 Rob Kampen rkam...@kampensonline.com wrote: One of my servers is using ISO datetime formats (2011-02-27T15:22:15.519857-05:00) in the logs the rest use the default redhat/CentOS format (Feb 27 15:10:21). After a couple of hours searching google I cannot find where this is defined. I know I changed it some months ago as an experiment but forgotten where this was done. the ISO format breaks logwatch - thus I need to revert. Have you changed that box to use rsyslog instead of the default syslog? If so, there's a post on HowToForge with some info on how to put it back into traditional format: http://www.howtoforge.com/forums/showthread.php?t=49642 another way to fix the problem - thanks for the link. More detail is in man rsyslog.conf in the TEMPLATES section. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos attachment: rkampen.vcf___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] log time formats - where is this defined
One of my servers is using ISO datetime formats (2011-02-27T15:22:15.519857-05:00) in the logs the rest use the default redhat/CentOS format (Feb 27 15:10:21). After a couple of hours searching google I cannot find where this is defined. I know I changed it some months ago as an experiment but forgotten where this was done. the ISO format breaks logwatch - thus I need to revert. TIA attachment: rkampen.vcf___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log time formats - where is this defined
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Rob Kampen Sent: Sunday, February 27, 2011 3:34 PM To: CentOS mailing list Subject: [CentOS] log time formats - where is this defined One of my servers is using ISO datetime formats (2011-02-27T15:22:15.519857-05:00) in the logs the rest use the default redhat/CentOS format (Feb 27 15:10:21). After a couple of hours searching google I cannot find where this is defined. I know I changed it some months ago as an experiment but forgotten where this was done. the ISO format breaks logwatch - thus I need to revert. I had problems with that, here's the fix: Add to first line of /etc/rsyslog.conf: $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat Here's the Bugzilla for this problem: https://bugzilla.redhat.com/show_bug.cgi?id=583621 Al ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log time formats - where is this defined
On Sun, 27 Feb 2011 15:33:57 -0500 Rob Kampen rkam...@kampensonline.com wrote: One of my servers is using ISO datetime formats (2011-02-27T15:22:15.519857-05:00) in the logs the rest use the default redhat/CentOS format (Feb 27 15:10:21). After a couple of hours searching google I cannot find where this is defined. I know I changed it some months ago as an experiment but forgotten where this was done. the ISO format breaks logwatch - thus I need to revert. Have you changed that box to use rsyslog instead of the default syslog? If so, there's a post on HowToForge with some info on how to put it back into traditional format: http://www.howtoforge.com/forums/showthread.php?t=49642 More detail is in man rsyslog.conf in the TEMPLATES section. -- Spiro Harvey Knossos Networks Ltd (04) 460-2531 : (021) 295-1923 www.knossos.net.nz signature.asc Description: PGP signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] log
My log seems not to be accepted by the list ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log
Maybe the list doesn't accept attachments if that's what you have been sending. Please try using something like pastebin.com and include the URL in your email. On 7/10/10, mj m...@mjw.se wrote: My log seems not to be accepted by the list ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] log rotation not working
Hello,
I've got a Centos 5.4 box that is not rotating it's mail logs. I just
found out about this, the file is considerably large. I've included my
log rotation configs if anyone has any suggestions i'm open to them.
Thanks.
Dave.
/etc/rsyslog.conf:
# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
# don't log clamd messages
*.info;ftp.none;clamd.none;mail.none;authpriv.none;cron.none
/var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
#uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.*/var/log/boot.log
# log ftp stuff separately
ftp.* /var/log/ftp.log
/etc/logrotate.d/syslog:
/var/log/messages /var/log/secure /var/log/maillog /var/log/spooler
/var/log/boot.log /var/log/cron {
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2 /dev/null` 2 /dev/null ||
true
/bin/kill -HUP `cat /var/run/rsyslogd.pid 2 /dev/null` 2 /dev/null ||
true
endscript
}
logrotate.conf:
# see man logrotate for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# uncomment this if you want your log files compressed
compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
monthly
minsize 1M
create 0664 root utmp
rotate 1
}
# system-specific logs may be also be configured here.
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log rotation not working
On Sun, Mar 14, 2010 at 5:10 PM, David Mehler dave.meh...@gmail.com wrote: I've got a Centos 5.4 box that is not rotating it's mail logs. I just found out about this, the file is considerably large. I've included my log rotation configs if anyone has any suggestions i'm open to them. I had a system, set up very minimally by someone else, exhibit this behavior. In my case, turned out that the crontabs package was not installed, which has the general cron config. Check that? --wes ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log rotation not working
Hi, Thanks for your reply. Crontabs package is indeed installed. Thanks. Dave. On 3/14/10, Wes Shull wes.sh...@gmail.com wrote: On Sun, Mar 14, 2010 at 5:10 PM, David Mehler dave.meh...@gmail.com wrote: I've got a Centos 5.4 box that is not rotating it's mail logs. I just found out about this, the file is considerably large. I've included my log rotation configs if anyone has any suggestions i'm open to them. I had a system, set up very minimally by someone else, exhibit this behavior. In my case, turned out that the crontabs package was not installed, which has the general cron config. Check that? --wes ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log rotation not working
On Sunday 14 March 2010 20:38:23 David Mehler wrote: Thanks for your reply. Crontabs package is indeed installed. Various things: 1- Check that indeed crond is running (ps -ef | grep cron) 2- Check that the logrotate script is indeed in the /etc/cron.daily|hourly| weekly directories... 3- the best one: run it manually by doing: logrotate -d -f /etc/logrotate.conf ..and see for yourself why isn't running. HTH, Jorge ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log rotation not working
Hi, Thanks for your reply. Cron is indeed installed and started. I had a logrotate script in cron.daily. When i ran logrotate -d -f logrotate.conf first it failed to complete with an error having to do with ftp, corrected that, reran it, this time it completed successfully but the major file had not rotated. The script claimed rotation of the maillog* files replacing 5 with 4, but the large maillog file didn't go away. Thanks. Dave. On 3/14/10, Jorge Fábregas jorge.fabre...@gmail.com wrote: On Sunday 14 March 2010 20:38:23 David Mehler wrote: Thanks for your reply. Crontabs package is indeed installed. Various things: 1- Check that indeed crond is running (ps -ef | grep cron) 2- Check that the logrotate script is indeed in the /etc/cron.daily|hourly| weekly directories... 3- the best one: run it manually by doing: logrotate -d -f /etc/logrotate.conf ..and see for yourself why isn't running. HTH, Jorge ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log rotation not writing to new logs
--- On Sun, 8/30/09, Dave dave.meh...@gmail.com wrote: Hello, I've got a centos 5.3 machine that is running services http and ftp whih are the two services i've noticed this on. When log rotation happens the old logs are renamed and compressed, but new logs httpd and pure-ftpd have zero lengths. From that poing log writing is not working. I've tried shutting down rsyslog and both httpd and pure-ftpd and restarting them, this is not working. I am open to suggestions. Thanks. Dave. Dave, please post your log rotation configuration. Matthias ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
com Bill Campbell wrote on Mon, 5 Jan 2009 16:02:29 -0800: (which we are running for Zope compatibility as the version of Zope we're running doesn't work with python-2.5.x. you did realize that this is another python compatibility issue, did you ;-) Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Tue, Jan 06, 2009, Kai Schaetzl wrote: com Bill Campbell wrote on Mon, 5 Jan 2009 16:02:29 -0800: (which we are running for Zope compatibility as the version of Zope we're running doesn't work with python-2.5.x. you did realize that this is another python compatibility issue, did you ;-) True enough :-). Bill -- INTERNET: b...@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax:(206) 232-9186 Make no laws whatever concerning speech and, speech will be free; so soon as you make a declaration on paper that speech shall be free, you will have a hundred lawyers proving that freedom does not mean abuse, nor liberty license; and they will define and define freedom out of existence. - Voltarine de Cleyre (1866-1912) ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
Spiro Harvey wrote: Les Mikesell lesmikes...@gmail.com wrote: Don't count on the same stability with python. It has an annoying habit of changing syntax in non-backwards compatible ways with no You seem to be hell-bent (excuse the pun) on turning this into a jihad on scripting languages. Please take the credo of your own favoured religion, sorry, language into account: There's more than one way to do it. Cope. There are hard ways and easy ways. I tend to prefer the easy ways and thought others might too. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Log File Reviewing
I need to review a logfile with Sed and cut out all the lines that start with a certain word, problem is this word begins after some amount of whitespace and unless I search for whitespace at the beginning followed by word I may encounter word somewhere legitimately hence why I don't just search for word only... Anyone know how to make sed accomplish this? Thanks! jlc ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Mon, Jan 05, 2009, Joseph L. Casale wrote:
I need to review a logfile with Sed and cut out all the lines that start with
a certain word, problem
is this word begins after some amount of whitespace and unless I search for
whitespace at the
beginning followed by word I may encounter word somewhere legitimately
hence why
I don't just search for word only...
Anyone know how to make sed accomplish this?
There's always more than one way to do something like this:
sed -n '/^[ \t]*word\s/p' /var/log/messages
pcregrep '^\s*word\b' /var/log/messages
awk '$1 == word{print}' /var/log/messages
Bill
--
INTERNET: b...@celestial.com Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
Voice: (206) 236-1676 Mercer Island, WA 98040-0820
Fax:(206) 232-9186
It is necessary for the welfare of society that genius should be
privileged to utter sedition, to blaspheme, to outrage good taste, to
corrupt the youthful mind, and generally to scandalize one's uncles.
-- George Bernard Shaw
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Mon, 5 Jan 2009, Joseph L. Casale wrote: I need to review a logfile with Sed and cut out all the lines that start with a certain word, problem is this word begins after some amount of whitespace and unless I search for whitespace at the beginning followed by word I may encounter word somewhere legitimately hence why I don't just search for word only... The regex you want is ^[[:space:]]*word -- Paul Heinlein heinl...@madboa.com http://www.madboa.com/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
What about:
perl -ne 'if (/^\s*word/) { print $_; }' logfile
any others?
On Mon, Jan 5, 2009 at 11:45 AM, Joseph L. Casale
jcas...@activenetwerx.com wrote:
I need to review a logfile with Sed and cut out all the lines that start with
a certain word, problem
is this word begins after some amount of whitespace and unless I search for
whitespace at the
beginning followed by word I may encounter word somewhere legitimately
hence why
I don't just search for word only...
Anyone know how to make sed accomplish this?
Thanks!
jlc
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
--
Thx
Joshua Gimer
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
awk '$1 == word{print}' /var/log/messages
This example assumes that word is the first field and that it consists
only of word. If the first field is word1 this won't match.
Fixes for this are
awk '$1 ~ word{print}'
(this matches any occurrance of word in the first field)
or:
awk '/^[[:space:]]*word/ {print}'
(this matches any line starting with whitespace followed immediately by
word)
--
Spiro Harvey Knossos Networks Ltd
021-295-1923www.knossos.net.nz
signature.asc
Description: PGP signature
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
The regex you want is ^[[:space:]]*word Wow, thanks everyone for the help! How does one modify this to also knock out lines that *must* have whitespace followed by a number [0-9]? I can do it using ^[[:space:]]*[0-9] but it also takes out lines w/o whitespace that begin with numbers? I have to buy a book on RegEx's and Sed :) Thanks all! jlc ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
[0-9]? I can do it using ^[[:space:]]*[0-9] but it also takes out lines w/o whitespace that begin with numbers? to match one or more, use + instead of *. * matches 0 or more, + matches 1 or more. I have to buy a book on RegEx's and Sed :) http://www.gnu.org/manual/gawk/gawk.pdf (G)awk is pretty sh!t hot where I work; however we've extended it a bit. :) -- Spiro Harvey Knossos Networks Ltd 021-295-1923www.knossos.net.nz signature.asc Description: PGP signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Mon, 5 Jan 2009, Joseph L. Casale wrote: The regex you want is ^[[:space:]]*word Wow, thanks everyone for the help! How does one modify this to also knock out lines that *must* have whitespace followed by a number [0-9]? I can do it using ^[[:space:]]*[0-9] but it also takes out lines w/o whitespace that begin with numbers? Probably something like ^[[:space:]]\+[0-9] -- though that assumes you're using gawk (since the \+ modifier is GNU-specific). For non-GNU awks, ^[[:space:]][[:space:]]*[0-9] I have to buy a book on RegEx's and Sed :) Good idea! -- Paul Heinlein heinl...@madboa.com http://www.madboa.com/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
to match one or more, use + instead of *. * matches 0 or more, + matches 1 or more. Thanks! I have to buy a book on RegEx's and Sed :) http://www.gnu.org/manual/gawk/gawk.pdf (G)awk is pretty sh!t hot where I work; however we've extended it a bit. :) So gawk does all that sed does and more? I suppose I can start with that in this case, I always wanted a book on regexe's so I think I am going to order O'Reilly's Mastering Regular Expressions, Third Edition. They also have a sed awk, Second Edition book, but its 10+ years old, does that matter, has sed/awk changed any since then? Thanks everyone! jlc ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
So gawk does all that sed does and more? I suppose I can start with Can't really answer that. In 15 years of using UNIX systems, I've never touched sed. :) With Gawk's BEGIN and END blocks you can use it to write full programs, which is kind of nice. that in this case, I always wanted a book on regexe's so I think I am going to order O'Reilly's Mastering Regular Expressions, Third Edition. They also have a sed awk, Second Edition book, but its 10+ years old, does that matter, has sed/awk changed any since then? The link I sent you is the 3rd edition of that book. Dated 2004. The book (Effective AWK Programming) is available completely free, but is also available in dead-tree editions. I printed and bound my PDF and saved a few dollars. -- Spiro Harvey Knossos Networks Ltd 021-295-1923www.knossos.net.nz signature.asc Description: PGP signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Jan 5, 2009, at 2:56 PM, Joseph L. Casale wrote:
The regex you want is ^[[:space:]]*word
Wow, thanks everyone for the help! How does one modify this to also
knock out
lines that *must* have whitespace followed by a number [0-9]? I can
do it using
^[[:space:]]*[0-9] but it also takes out lines w/o whitespace that
begin with
numbers?
^[[:space:]]+[[:digit:]]+
will hit numbers with one or more digits. to restrict the number of
digits, use something like
^[[:space:]]+[[:digit:]]{2}[^[:digit:]]+
that, for example, should only hit lines that consist of at least one
whitespace character, then exactly two digits, then at least one non-
digit character.
-steve
--
If this were played upon a stage now, I could condemn it as an
improbable fiction. - Fabian, Twelfth Night, III,v
smime.p7s
Description: S/MIME cryptographic signature
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Mon, 2009-01-05 at 13:40 -0700, Joseph L. Casale wrote: to match one or more, use + instead of *. * matches 0 or more, + matches 1 or more. Thanks! snip So gawk does all that sed does and more? I suppose I can start with Tons. You can write fairly complex programs with (g)awk. It can combine command line expressions, scripts from files, has formatted print capability, conditional execution, multiple regex selection capabilities and mode. A read of the man page would give you a lot of insight. Think of perl in an earlier form. The original awk was probably what inspired perl. That would be my guess. Since (g)awk is regex based, what you learn for sed, vi(m), etc. is easily transferred into (g)awk, and vice-versa, to a limited degree. that in this case, I always wanted a book on regexe's so I think I am going to order O'Reilly's Mastering Regular Expressions, Third Edition. They also have a sed awk, Second Edition book, but its 10+ years old, does that matter, has sed/awk changed any since then? The man pages will allow you to keep up easily once the fundamentals are in place. Of course, frequency of use affects that greatly. Thanks everyone! jlc snip -- Bill ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
Joseph L. Casale wrote: to match one or more, use + instead of *. * matches 0 or more, + matches 1 or more. Thanks! I have to buy a book on RegEx's and Sed :) http://www.gnu.org/manual/gawk/gawk.pdf (G)awk is pretty sh!t hot where I work; however we've extended it a bit. :) So gawk does all that sed does and more? I suppose I can start with that in this case, I always wanted a book on regexe's so I think I am going to order O'Reilly's Mastering Regular Expressions, Third Edition. They also have a sed awk, Second Edition book, but its 10+ years old, does that matter, has sed/awk changed any since then? Why not just start with perl which does more than sed/awk while using similar syntax (if you want)? -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
Why not just start with perl which does more than sed/awk while using
similar syntax (if you want)?
This is why:
awk '/^[[:space:]]*word/ {print}' logfile
vs
perl -ne 'if (/^\s*word/) { print $_; }' logfile
Which syntax is likely to be easier to remember?
--
Spiro Harvey Knossos Networks Ltd
021-295-1923www.knossos.net.nz
signature.asc
Description: PGP signature
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
Spiro Harvey wrote:
Why not just start with perl which does more than sed/awk while using
similar syntax (if you want)?
This is why:
awk '/^[[:space:]]*word/ {print}' logfile
vs
perl -ne 'if (/^\s*word/) { print $_; }' logfile
Which syntax is likely to be easier to remember?
I never remember the awk syntax because if it is really that simple I'd
use grep with it's implied print. But it's almost never really that
simple and you end up needing things that are difficult in awk but easy
in perl. Perl can use the posix names for character classes too if you
like to type and how can you forget the 'if (expresssion) {action};
syntax? Also you could have omitted the $_ argument to print, since it
is assumed if you are looking for simplicity.
--
Les Mikesell
lesmikes...@gmail.com
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Tue, Jan 06, 2009, Spiro Harvey wrote:
Why not just start with perl which does more than sed/awk while using
similar syntax (if you want)?
This is why:
awk '/^[[:space:]]*word/ {print}' logfile
vs
perl -ne 'if (/^\s*word/) { print $_; }' logfile
Which syntax is likely to be easier to remember?
It depends entirely on what you want to do. For on-liners, sed, awk, and
grep, and pcregrep (grep using perl regular expression syntax which is
considerably more concise than [:space:] and friends) are often the best
tools. For anything more complex, scripting languages such as python and
perl are generally more flexible and easier to use.
I used to some pretty complex shell and awk scripts before learning perl
about 20 years ago. Perl allowed me to do most things in a single language
including fairly low-level system calls that I previously had to do with
compiled ``C'' programs.
I have switched all of my new development primarily to python which I find
far cleaner than perl, and easier to use for large projects. Python uses
perl regular expression syntax so the transition was pretty painless.
Bill
--
INTERNET: b...@celestial.com Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way
Voice: (206) 236-1676 Mercer Island, WA 98040-0820
Fax:(206) 232-9186
The liberties of a people never were, nor ever will be, secure, when the
transactions of their rulers may be concealed from them. -- Patrick Henry
___
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
Bill Campbell wrote: I used to some pretty complex shell and awk scripts before learning perl about 20 years ago. Perl allowed me to do most things in a single language including fairly low-level system calls that I previously had to do with compiled ``C'' programs. And you can probably still run all of your perl scripts unchanged, with the possible exception of @array being interpolated in double-quoted strings which I think started in perl4. I have switched all of my new development primarily to python which I find far cleaner than perl, and easier to use for large projects. Python uses perl regular expression syntax so the transition was pretty painless. Don't count on the same stability with python. It has an annoying habit of changing syntax in non-backwards compatible ways with no provision for running old scripts. If you run your programs on more than one machine you'll end up having to maintain different versions to match the installed interpreters. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
Les Mikesell lesmikes...@gmail.com wrote: Don't count on the same stability with python. It has an annoying habit of changing syntax in non-backwards compatible ways with no You seem to be hell-bent (excuse the pun) on turning this into a jihad on scripting languages. Please take the credo of your own favoured religion, sorry, language into account: There's more than one way to do it. Cope. -- Spiro Harvey Knossos Networks Ltd 021-295-1923www.knossos.net.nz signature.asc Description: PGP signature ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log File Reviewing
On Mon, Jan 05, 2009, Les Mikesell wrote: Bill Campbell wrote: I used to some pretty complex shell and awk scripts before learning perl about 20 years ago. Perl allowed me to do most things in a single language including fairly low-level system calls that I previously had to do with compiled ``C'' programs. And you can probably still run all of your perl scripts unchanged, with the possible exception of @array being interpolated in double-quoted strings which I think started in perl4. I think that was perl-5, but I may well be mistaken. I have found some changes in perl along the way that have required fixing scripts since I started in perl-3.something, but not many. I have switched all of my new development primarily to python which I find far cleaner than perl, and easier to use for large projects. Python uses perl regular expression syntax so the transition was pretty painless. Don't count on the same stability with python. It has an annoying habit of changing syntax in non-backwards compatible ways with no provision for running old scripts. If you run your programs on more than one machine you'll end up having to maintain different versions to match the installed interpreters. I have not run into many (any) compatibility issues with python, but then I have only been doing python for a bit over 4 years now. As I remember, there were some issues with the early versions of python-2.4, but those were in the python builds, not in the syntax of python itself. I tend to stay away of the more esoteric features of languages that are likely to change so don't generally have problems of this type. We don't have problems with multiple versions of packages as we use the ones from the OpenPKG portable packaging system which includes its own versions of python, perl, gcc, berkeley db, etc. avoiding most problems with the underlying distribution/vendor's packages. There were some issues when we moved to CentOS from SuSE in that SuSE ran ran python-2.3.x while CentOS has python-2.4.x which caused some interesting shared library issues with the OpenPKG python-2.4.x (which we are running for Zope compatibility as the version of Zope we're running doesn't work with python-2.5.x. Python-3 definately has backwards compatibility issues, and there are lengthy explanations as to why this is so. Bill -- INTERNET: b...@celestial.com Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way Voice: (206) 236-1676 Mercer Island, WA 98040-0820 Fax:(206) 232-9186 Windows is a computer virus with a user interface!! ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] log sshd activities
hi all, where exactly sshd log files??? this is what i have in /etc/sshsshd_config SyslogFacility AUTHPRIV if i want to log who login/logout sshd, what option do i need to turn on??? thanks, t. hiep ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log sshd activities
Hiep Nguyen wrote: hi all, where exactly sshd log files??? this is what i have in /etc/sshsshd_config SyslogFacility AUTHPRIV if i want to log who login/logout sshd, what option do i need to turn on??? /var/log/secure thanks, t. hiep ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log sshd activities
On Tuesday 11 March 2008 15:33:36 Hiep Nguyen wrote: hi all, where exactly sshd log files??? this is what i have in /etc/sshsshd_config SyslogFacility AUTHPRIV if i want to log who login/logout sshd, what option do i need to turn on??? thanks, t. hiep Logwatch can supply you with a daily report on this - - SSHD Begin Users logging in through sshd: anne: 192.168.0.91 (anne-laptop.lydgate.lan): 4 times -- SSHD End - HTH Anne signature.asc Description: This is a digitally signed message part. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] log outbound port 80 connections
Is there a way to log outbound connections to a specific port (80)? CentOS 4.6. iptables? Thanks Tony Schreiner Boston College ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
On Tue, Feb 05, 2008 at 11:56:48AM -0500, Tony Schreiner wrote: Is there a way to log outbound connections to a specific port (80)? CentOS 4.6. iptables? iptables -A OUTPUT -p tcp --dport 80 -j LOG --log-prefix WWW You might want to tack --syn on there as well to only log the packet initiating the connection instead of packets for the whole stream. Ray ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
On Feb 5, 2008, at 12:00 PM, Ray Van Dolson wrote: On Tue, Feb 05, 2008 at 11:56:48AM -0500, Tony Schreiner wrote: Is there a way to log outbound connections to a specific port (80)? CentOS 4.6. iptables? iptables -A OUTPUT -p tcp --dport 80 -j LOG --log-prefix WWW You might want to tack --syn on there as well to only log the packet initiating the connection instead of packets for the whole stream. Ray Thanks for that. Followup. Can I associate anything in the log record with the process. I see the SPT but, the connection appears to be short, I can't find the port in netstat or lsof (not sure if those apply to source ports). Tony ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
On Feb 5, 2008, at 12:15 PM, John R Pierce wrote: Tony Schreiner wrote: Is there a way to log outbound connections to a specific port (80)? CentOS 4.6. assuming you want to log user web browsing traffic, configuring a Squid transparent proxy at your network border would be the best way. its logfiles are quite similar to those of a webserver, so you can use a wide range of log analysis tools. To get more specific about what's going on. My network services have informed me that the machine is probing other systems at a high rate. An infection of some sort. And I'm trying to track down what's going on. Tony ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
To get more specific about what's going on. My network services have informed me that the machine is probing other systems at a high rate. An infection of some sort. And I'm trying to track down what's going on. The LOG target lets you display the user id of the process I believe, but not the PID. There might be some iptables extensions out there that would do what you're looking for. Don't know them off the top of my head however. Alternately, perhaps you could use SELinux for this? I know its audit logs would give you the level of detail you're looking for, but getting the policy written for it might be challenging. Ray ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
On Tue, Feb 05, 2008, Tony Schreiner wrote: On Feb 5, 2008, at 12:15 PM, John R Pierce wrote: Tony Schreiner wrote: Is there a way to log outbound connections to a specific port (80)? CentOS 4.6. assuming you want to log user web browsing traffic, configuring a Squid transparent proxy at your network border would be the best way. its logfiles are quite similar to those of a webserver, so you can use a wide range of log analysis tools. To get more specific about what's going on. My network services have informed me that the machine is probing other systems at a high rate. An infection of some sort. And I'm trying to track down what's going on. In that case, you might want to use ``lsof -i :80'' to see processes using port 80. Once one has an interesting PID, then using ``lsof -p PID'' will show everything that process is using including the full path to the executing program. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX:(206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 The only logical reason to take guns away from responsible people is to give irresponsible people an edge in the perpetration of their crimes against us. -- The Idaho Observer, Vol. 1, No. 2 February 1997 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
Tony Schreiner wrote: assuming you want to log user web browsing traffic, configuring a Squid transparent proxy at your network border would be the best way. its logfiles are quite similar to those of a webserver, so you can use a wide range of log analysis tools. To get more specific about what's going on. My network services have informed me that the machine is probing other systems at a high rate. An infection of some sort. And I'm trying to track down what's going on. ah. tcpdump -i ethX tcp port 80 (and prepare for a flood of data). ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
On Tue, Feb 05, 2008 at 09:29:30AM -0800, John R Pierce wrote: Tony Schreiner wrote: assuming you want to log user web browsing traffic, configuring a Squid transparent proxy at your network border would be the best way. its logfiles are quite similar to those of a webserver, so you can use a wide range of log analysis tools. To get more specific about what's going on. My network services have informed me that the machine is probing other systems at a high rate. An infection of some sort. And I'm trying to track down what's going on. ah. tcpdump -i ethX tcp port 80 (and prepare for a flood of data). If you decide to use tcpdump at all, maybe just limit to SYN packets as well: tcpdump -n -i ethX 'tcp port 80 and tcp[tcpflags] tcp-syn != 0' Ray ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] log outbound port 80 connections
On Tuesday 05 February 2008 12:00, Ray Van Dolson wrote: iptables -A OUTPUT -p tcp --dport 80 -j LOG --log-prefix WWW I was thinking more along these lines for a rule: iptables -A OUTPUT -p tcp --dport 80 -m state --state NEW -j LOG --log-prefix [WWW] : --log-tcp-options --log-ip-options -- Regards Robert Smile... it increases your face value! Linux User #296285 http://counter.li.org ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Log Monitoring Recomendation
Given my experience in Linux is limited currently, what do you guys use to monitor logs such as 'messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... Thanks! jlc ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
On Mon, Jan 07, 2008, Joseph L. Casale wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as `messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... We use swatch to monitor various things, mainly security related. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX:(206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 Rights is a fictional abstraction. No one has ``Rights'', neither machines nor flesh-and-blood. Persons... have opportunities, not rights, which they use or do not use. -- Lazarus Long ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
Joseph L. Casale wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as ‘messages’ on your centos servers? I had a hardware failure that happened in between me manually looking (of course…). I would hope it might have a some features to email critical issues etc… Depends on if you're monitoring just one server or a bunch. I'd google for these things: LogWatch epylog big syster oak Then there's various things that read syslog and can read reports for you. Google around for things like syslog-ng, nagios, zenoss, whatnot, if you're looking at larger scope. Jed ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
Bill Campbell wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as `messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... We use swatch to monitor various things, mainly security related. Did you have to do something to it to make it work with centos? I have one running on a machine that collects a lot of router syslogs and it has the annoying habit of resending a bunch of old notifications whenever a new one is noticed. -- Les Mikesell [EMAIL PROTECTED] ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Log Monitoring Recomendation
On Mon, Jan 07, 2008, Les Mikesell wrote: Bill Campbell wrote: Given my experience in Linux is limited currently, what do you guys use to monitor logs such as `messages' on your centos servers? I had a hardware failure that happened in between me manually looking (of course...). I would hope it might have a some features to email critical issues etc... We use swatch to monitor various things, mainly security related. Did you have to do something to it to make it work with centos? I have one running on a machine that collects a lot of router syslogs and it has the annoying habit of resending a bunch of old notifications whenever a new one is noticed. Not really. Swatch is pretty straightforward perl, using gnu-tail to watch the end of log file(s). The only issue I've seen is that it will sometimes report old things on occassion when starting if there are matching entries near the end of the files. One place where I used this is on an openldap server that would occassionally get into a ``too many open files'' situation, and swatch would call a routine that restarted slapd when this happened. Bill -- INTERNET: [EMAIL PROTECTED] Bill Campbell; Celestial Software LLC URL: http://www.celestial.com/ PO Box 820; 6641 E. Mercer Way FAX:(206) 232-9186 Mercer Island, WA 98040-0820; (206) 236-1676 Capitalism works primarily because most of the ways that a company can be scum end up being extremely bad for business when there's working competition. -rra ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
