switching to rsyslog everywhere is definantly the most flexible option
(and probably the best from a long-term support point of view).
what Rainer was talking about wasn't the ruleset parser capibility, but
rather the parser module capibility
http://www.rsyslog.com/doc/messageparser.html
and an example one is at http://www.rsyslog.com/doc/pmlastmsg.html
there isn't currently a module that will fix up this malformed input, it's
something that probably isn't that hard to do (it does take learning how
to insert data into a log entry rather than just changeing or deleting it
like the prior parsers I've written, but I've got instructions on how to
do that, I just haven't done it yet)
David Lang
On Sun, 22 May 2011, Kaiwang Chen wrote:
After a quick scan through Message parsers in rsyslog,
http://www.rsyslog.com/doc/messageparser.html, I was able to locate a
$RulesetParser directive to manipulate the parser chain bound a
specific ruleset. Also noticed the statements under the section "Can I
use different parser chains for different devices":
Note that it is currently under evaluation if rsyslog will support
binding parser chains to specific inputs directly, without depending
on the ruleset. There are some concerns that this may not be necessary
but adds considerable complexity to the configuration. So this may or
may not be possible in the future. In any case, if we decide to add
it, input modules need to support it, so this functionality would
require some time to implement.
Bad news is that
$RulesetParser(http://www.rsyslog.com/doc/rsconf1_rulesetparser.html)
is a 5.3.4+ feature. And multiple
ruleset(http://www.rsyslog.com/doc/multi_ruleset.html) is a 4.5.0+ or
5.1.1+ feature.
I can't figure out whether imudp, which is the only choice for legacy
syslogd sender, supports multi-ruleset or not. Looks like it is
impossible to register multiple rulsets to receive from single input,
say tcp/514.
As for my case, a quick solution would be upgrade all CentOS5 server
to use rsyslogd, and if any bridge device generates mailformated logs,
I would setup a relay and use output module to reformat it.
Thanks,
Kaiwang
2011/5/22 Rainer Gerhards <[email protected]>:
yes
-----Original Message-----
From: [email protected] [mailto:rsyslog-
[email protected]] On Behalf Of Kaiwang Chen
Sent: Saturday, May 21, 2011 6:24 PM
To: rsyslog-users
Subject: Re: [rsyslog] HOSTNAME and programname extraction failure,
when rsyslog as receiver, stock syslog as sender
So rsyslogd is currently lame in the case of missing both timestamp
and hostname. Is it possible to assign a different parser for specific
log source to get things right in output module?
Thanks,
Kaiwang
2011/5/21 Rainer Gerhards <[email protected]>:
-----Original Message-----
From: [email protected] [mailto:rsyslog-
[email protected]] On Behalf Of [email protected]
Sent: Saturday, May 21, 2011 11:53 AM
To: rsyslog-users
Subject: Re: [rsyslog] HOSTNAME and programname extraction failure,
when rsyslog as receiver, stock syslog as sender
Rainer,
if the message is missing a timestamp, why does the default
parser
assume that there is a hostname there?
I would assume that if there is no timestamp there isn't a hostname
either
(which would address this particular issue)
did you have some experiance in the past that pushed you to the
current
implementation?
I don't have the specifics at hand, but as far as I remember there
were cases
where absence of timestamp does NOT indicate absence of tag and/or
hostname.
I am very hesitant to touch the default legacy parser, as the
heuristics
works pretty well since > 2 years. All other malformed messages
reported were
really badly malformed. So I think the clean path would be to write a
parser
module for such dateless but otherwise correct messages...
Rainer
David Lang
On Fri, 20 May 2011, Kaiwang Chen wrote:
Date: Fri, 20 May 2011 09:35:15 +0800
From: Kaiwang Chen <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] HOSTNAME and programname extraction
failure,
when rsyslog as receiver, stock syslog as sender
Yes, it's
3c 34 36 3e 65 78 69 74 69 6e 67 20 6f 6e 20 73 69 67 6e 61 6c 20
31
35 0a
< 4 6 > e x i t i n g _ o n _ s i
g
n a l _ 1 5 \n
where spaces are represented by underscores.
Thanks,
Kaiwang
2011/5/20 <[email protected]>:
if it's being generated without a timestamp, rsyslog should be
able
to
detect that. can you get a raw log and verify that?
David Lang
On Fri, 20 May 2011, Kaiwang Chen wrote:
Date: Fri, 20 May 2011 00:10:27 +0800
From: Kaiwang Chen <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: Re: [rsyslog] HOSTNAME and programname extraction
failure,
when rsyslog as receiver, stock syslog as sender
Looks like it's the stock sysklog in CentOS5.6 that generated
bad
formatted logs(without timestamp and syslogtag), and with a
rsyslog
3.x installation I have no choice but not use stock sysklog.
Thanks,
Kaiwang
2011/5/19 Rainer Gerhards <[email protected]>:
I addition to what David already said:
http://www.rsyslog.com/doc/syslog_parsing.html
Rainer
-----Original Message-----
From: [email protected] [mailto:rsyslog-
[email protected]] On Behalf Of Kaiwang Chen
Sent: Wednesday, May 18, 2011 2:02 PM
To: [email protected]
Subject: [rsyslog] HOSTNAME and programname extraction
failure,
when
rsyslog as receiver, stock syslog as sender
Hello,
I was trying to configure rsyslog(rsyslog-3.22.1-3.el5_5.1) as
receiver, stock syslog(sysklogd-1.4.1-46.el5) as sender.
The rsyslogd listened on udp/514, and used dynamic filenames
with
protocol23 message formatting:
$ModLoad imudp
$UDPServerRun 514
$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
$template
DynFile,"/var/log/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/rsyslog.log"
*.* ?DynFile
The sender generated several entries(3rd, 4th) violating the
"syslogtag: message" convention
May 18 19:40:17 dns1 kernel: Kernel logging (proc) stopped.
May 18 19:40:17 dns1 kernel: Kernel log daemon terminating.
May 18 19:40:18 dns1 exiting on signal 15
May 18 19:40:18 dns1 syslogd 1.4.1: restart.
May 18 19:40:18 dns1 kernel: klogd 1.4.1, log source =
/proc/kmsg
started.
Take 3rd entry for example, the receiver mistook 'exiting' and
'on' to
be %HOSTNAME% and %programname%:
<6>1 2011-05-18T19:40:12.592370+08:00 bogon kernel - - -
Kernel
logging (proc) stopped.
<6>1 2011-05-18T19:40:12.592693+08:00 bogon kernel - - -
Kernel
log
daemon terminating.
<46>1 2011-05-18T19:40:13.697115+08:00 exiting on - - -
signal
15
<46>1 2011-05-18T19:40:13.806302+08:00 syslogd 1.4.1 - - -
restart.
<6>1 2011-05-18T19:40:13.811331+08:00 bogon kernel - - -
klogd
1.4.1,
log source = /proc/kmsg started.
So, it went to /var/log/hosts/exiting/2011/05/18/rsyslog.log,
and
that's definitely the wrong place.
How to deal with this case?
Thanks,
Kaiwang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com
