On Thu, 26 Nov 2009, Jose Sanchez wrote:
> Hello,
>
> I appreciate all the responses.
> Im not sure how can I can acconplish options 1) and 2) automatically.
> For option 3) the thing is I need "combined" log type so I cannot reform this.
>
> Im trying to centralize an access_log file from one server to the rsyslog
> server and I need to completely remove the tags I mentioned on my previous
> post.
> I have also tried using a perl script mentioned at the botton of this email,
> but it salso arriving with a tag, "apache_syslog:" as showed below,
>
> "apache_syslog: XXX.XXX.XXX.XXX - - [26/Nov/2009:18:23:02 -0600] \"GET /.."
>
> Basically, this log will be parsed by awstats which is pretty much stricted
> with the log format so that's why I need a clean log sent from the apache
> server to the rsyslog server.
don't forget that you need to filter these messages into a seperate file,
otherwise you will have your apache combined log messages mixed with other
syslog messages (which will really confuse awstats)
option 1 is what Rainer suggested
option 2 is to run the log through another step before awstats runs,
something along the lines of
cut -c 16- file |cut -f 3- -d ' ' |awstats
the first cut removes the timestamp (always 15 characters, but with a
variable number of spaces in it), the second cut removes the servername
and the syslog tag ('logger:' in your first example)
David Lang
> Thank you very much for all the help.
>
> Below is the Perl script:
>
> #!/usr/local/bin/perl
> # script: apache-access-logger
>
> use Sys::Syslog;
> $SERVER_NAME = shift || '';
>
> $PRIORITY = 'info';
> $FACILITY = 'local1';
>
> Sys::Syslog::setlogsock('unix');
> openlog ($SERVER_NAME,'ndelay', $FACILITY);
>
> while (<>) {
> chomp;
> syslog($PRIORITY,$_);
> }
> closelog;
>
> --- On Thu, 11/26/09, [email protected] <[email protected]> wrote:
>
>> From: [email protected] <[email protected]>
>> Subject: Re: [rsyslog] filter logger tags from syslog
>> To: "rsyslog-users" <[email protected]>
>> Date: Thursday, November 26, 2009, 2:21 AM
>> On Wed, 25 Nov 2009, Jose Sanchez
>> wrote:
>>
>>> Hello,
>>>
>>> I've been using classic syslog for centralizing apache
>> access logs from one server to a remote syslog server, the
>> thing is syslog adds some nasty tags before the lines in the
>> access logs and I cant get them off, ie:
>>>
>>> "Nov 25 21:25:37 server1 logger:"
>>>
>>> I would like to know if rsyslog has the option to
>> filter this kind of stuff, I just want to have the logs sent
>> to the syslog server exactly like I was saving them in a
>> local access.log file.
>>>
>>> Thanks in advance.
>>
>> 'logger:' is added by the logger program that apache is
>> using to send the
>> logs to syslog.
>>
>> a properly formatted syslog message will include a
>> timestamp and what
>> server it came from (note that the apache logs do _not_
>> tell you what
>> virtual server the log comes from, it usually uses a
>> different file for
>> each log, so when you mix them into syslog you won't be
>> able to tell them
>> apart)
>>
>> so you have three basic options
>>
>> 1. let logger do it's default thing and then use a
>> formatting command to
>> strip off the 'syslogie' parts to get back to the apache
>> default in the
>> file
>>
>> 2. leave the 'syslogie' parts in when you write it to a
>> file and have your
>> analysis tool strip them out
>>
>> 3. reformat the apache log message so that you put useful
>> information in
>> the 'syslogie' parts of the message.
>>
>> you can move the timestamp to the beginning (you can do
>> this with or
>> without the timezone, the format obviously differs
>> slightly)
>>
>> you can put the name of the virtual host in the server
>> field
>>
>> you can replace 'logger:' with something like apache[80]:
>> or apache[443]:
>>
>> I am going to be setting up something along the lines of #3
>> in the next
>> few weeks. I figure I will also want to tinker with other
>> things in the
>> log message. there are items that apache can log, but does
>> not log by
>> default (I believe that how long it took to process the
>> request is one of
>> these), and also since syslog defaults to limiting log
>> messages to 1-2K
>> (depending on your impementation), there are some fields
>> that I would want
>> to move late in the message so that if they get very long
>> they don't cause
>> other fields to be lost due to truncation (URL and referrer
>> fields can be
>> several K long by themselves)
>>
>> David Lang
>> _______________________________________________
>> rsyslog mailing list
>> http://lists.adiscon.net/mailman/listinfo/rsyslog
>> http://www.rsyslog.com
>>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
