> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Jose Sanchez > Sent: Friday, November 27, 2009 5:13 AM > To: rsyslog-users > Subject: Re: [rsyslog] filter logger tags from syslog > > Hello, > > Thanks again for the great response. > It's actually working! rsyslog is removing the "logger:" thing and all > the nasty stuff from it automatically, how come?
you really need to read through the property doc. I know the doc is not great, but with some persistence you'll find anything you need (and doc contributions are always very welcome!). Understanding properties is key to getting complex configurations right. > Is it because we are > not adding any tag in the template? Yes, because the template specifies which properties are used (and how). > Im still not understanding how > rsyslog removes the logger thing. > > Ok, Im now getting the proper output and like David said Im now getting > issues with filtering the apache logs with all the rsyslog messages. > I've tried to use the following filter but for some reason is not > working and Im not 100% if this is the best solution to use, > > This is how I had set it up, > > $template line,"%msg%\n" > if $msg contains 'GET' then /var/log/apache.test.log;line > *.* /var/log/test.log;line If you want to split the messages, you need to discard the one that you just wrote: if $msg contains 'GET' then /var/log/apache.test.log;line & ~ Helpful read is http://www.rsyslog.com/doc-queues_analogy.html HTH Rainer > > Not sure if Im on the right path, any help will be appreciated. > I have also tried the "if" sentence without specifying the template > name. > > Many Thanks. > > --- On Thu, 11/26/09, [email protected] <[email protected]> wrote: > > > From: [email protected] <[email protected]> > > Subject: Re: [rsyslog] filter logger tags from syslog > > To: "rsyslog-users" <[email protected]> > > Date: Thursday, November 26, 2009, 6:38 PM > > On Thu, 26 Nov 2009, Jose Sanchez > > wrote: > > > > > Hello, > > > > > > I appreciate all the responses. > > > Im not sure how can I can acconplish options 1) and 2) > > automatically. > > > For option 3) the thing is I need "combined" log type > > so I cannot reform this. > > > > > > Im trying to centralize an access_log file from one > > server to the rsyslog server and I need to completely remove > > the tags I mentioned on my previous post. > > > I have also tried using a perl script mentioned at the > > botton of this email, but it salso arriving with a tag, > > "apache_syslog:" as showed below, > > > > > > "apache_syslog: XXX.XXX.XXX.XXX - - > > [26/Nov/2009:18:23:02 -0600] \"GET /.." > > > > > > Basically, this log will be parsed by awstats which is > > pretty much stricted with the log format so that's why I > > need a clean log sent from the apache server to the rsyslog > > server. > > > > don't forget that you need to filter these messages into a > > seperate file, > > otherwise you will have your apache combined log messages > > mixed with other > > syslog messages (which will really confuse awstats) > > > > option 1 is what Rainer suggested > > > > option 2 is to run the log through another step before > > awstats runs, > > something along the lines of > > > > cut -c 16- file |cut -f 3- -d ' ' |awstats > > > > the first cut removes the timestamp (always 15 characters, > > but with a > > variable number of spaces in it), the second cut removes > > the servername > > and the syslog tag ('logger:' in your first example) > > > > David Lang > > > > > Thank you very much for all the help. > > > > > > Below is the Perl script: > > > > > > #!/usr/local/bin/perl > > > # script: apache-access-logger > > > > > > use Sys::Syslog; > > > $SERVER_NAME = shift || ''; > > > > > > $PRIORITY = 'info'; > > > $FACILITY = 'local1'; > > > > > > Sys::Syslog::setlogsock('unix'); > > > openlog ($SERVER_NAME,'ndelay', $FACILITY); > > > > > > while (<>) { > > > chomp; > > > syslog($PRIORITY,$_); > > > } > > > closelog; > > > > > > --- On Thu, 11/26/09, [email protected] <[email protected]> > > wrote: > > > > > >> From: [email protected] <[email protected]> > > >> Subject: Re: [rsyslog] filter logger tags from > > syslog > > >> To: "rsyslog-users" <[email protected]> > > >> Date: Thursday, November 26, 2009, 2:21 AM > > >> On Wed, 25 Nov 2009, Jose Sanchez > > >> wrote: > > >> > > >>> Hello, > > >>> > > >>> I've been using classic syslog for > > centralizing apache > > >> access logs from one server to a remote syslog > > server, the > > >> thing is syslog adds some nasty tags before the > > lines in the > > >> access logs and I cant get them off, ie: > > >>> > > >>> "Nov 25 21:25:37 server1 logger:" > > >>> > > >>> I would like to know if rsyslog has the option > > to > > >> filter this kind of stuff, I just want to have the > > logs sent > > >> to the syslog server exactly like I was saving > > them in a > > >> local access.log file. > > >>> > > >>> Thanks in advance. > > >> > > >> 'logger:' is added by the logger program that > > apache is > > >> using to send the > > >> logs to syslog. > > >> > > >> a properly formatted syslog message will include > > a > > >> timestamp and what > > >> server it came from (note that the apache logs do > > _not_ > > >> tell you what > > >> virtual server the log comes from, it usually uses > > a > > >> different file for > > >> each log, so when you mix them into syslog you > > won't be > > >> able to tell them > > >> apart) > > >> > > >> so you have three basic options > > >> > > >> 1. let logger do it's default thing and then use > > a > > >> formatting command to > > >> strip off the 'syslogie' parts to get back to the > > apache > > >> default in the > > >> file > > >> > > >> 2. leave the 'syslogie' parts in when you write it > > to a > > >> file and have your > > >> analysis tool strip them out > > >> > > >> 3. reformat the apache log message so that you put > > useful > > >> information in > > >> the 'syslogie' parts of the message. > > >> > > >> you can move the timestamp to the beginning (you > > can do > > >> this with or > > >> without the timezone, the format obviously > > differs > > >> slightly) > > >> > > >> you can put the name of the virtual host in the > > server > > >> field > > >> > > >> you can replace 'logger:' with something like > > apache[80]: > > >> or apache[443]: > > >> > > >> I am going to be setting up something along the > > lines of #3 > > >> in the next > > >> few weeks. I figure I will also want to tinker > > with other > > >> things in the > > >> log message. there are items that apache can log, > > but does > > >> not log by > > >> default (I believe that how long it took to > > process the > > >> request is one of > > >> these), and also since syslog defaults to limiting > > log > > >> messages to 1-2K > > >> (depending on your impementation), there are some > > fields > > >> that I would want > > >> to move late in the message so that if they get > > very long > > >> they don't cause > > >> other fields to be lost due to truncation (URL and > > referrer > > >> fields can be > > >> several K long by themselves) > > >> > > >> David Lang > > >> _______________________________________________ > > >> rsyslog mailing list > > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > > >> http://www.rsyslog.com > > >> > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
