you can tie a ruleset to a particular input, so all messages that arrive via
that input only see the rules in that ruleset.
to change the action 2 to something meaningful, add name='foo' in the action()
statement and you will then get foo instead of 'action 2' in the pstats output
pstats only knows about what is going on on the server it's running on, it can't
tell you what happened on other systems.
does this answer your questions?
David Lang
On Wed, 24 Feb 2021, odrzen via rsyslog wrote:
Date: Wed, 24 Feb 2021 21:59:52 +0000
From: odrzen via rsyslog <[email protected]>
Reply-To: odrzen <[email protected]>,
rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Cc: odrzen <[email protected]>
Subject: Re: [rsyslog] The right way to include more log files?
First of all, Thank you very much for all this analysis.
So, if I define my own rulesets in some rsyslog "client" to send this
messages/logs to a central rsyslog server, then from the server side, I don't need to
define something, right ?
Do these only apply to the local machine. Right ?
Another very easy example: if a machine doesn't run many services, but only
1-2, and we want to send the logs to a remote machine, in this case, too, would
you suggest creating rulesets ?
Well .. is it for better performance and easier management in various
situations ?
Can I use these rulesets and actions even more ?
Additionally, rulesets can have own queues which help dealing with reliability
issues.
Can I even use them on rsyslog (central) server to have better statistics ?
For example, I have now done this configuration:
module(load="impstats"
interval="600"
severity="7"
log.syslog="off")
global(senders.keepTrack="on")
dyn_stats(name="msg_per_host")
and I get the following results:
```
Wed Feb 24 23:23:20 2021: global: origin=dynstats msg_per_host.ops_overflow=0
msg_per_host.new_metric_add=0 msg_per_host.no_metric=0
msg_per_host.metrics_purged=0 msg_per_host.ops_ignored=0
msg_per_host.purge_triggered=142
Wed Feb 24 23:23:20 2021: imuxsock: origin=imuxsock submitted=0
ratelimit.discarded=0 ratelimit.numratelimiters=0
Wed Feb 24 23:23:20 2021: dynafile cache Auditlog: origin=omfile
requests=1611936 level0=979202 missed=26491 evicted=26362 maxused=10
closetimeouts=0
Wed Feb 24 23:23:20 2021: action 0: origin=core.action processed=1611936
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: dynafile cache RemoteLogs: origin=omfile
requests=69023867 level0=7424680 missed=930036 evicted=929906 maxused=10
closetimeouts=0
Wed Feb 24 23:23:20 2021: action 1: origin=core.action processed=69023867
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 2: origin=core.action processed=67365327
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 3: origin=core.action processed=1611936
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 4: origin=core.action processed=21400
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 5: origin=core.action processed=25204
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 6: origin=core.action processed=0
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 7: origin=core.action processed=0
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 8: origin=core.action processed=0
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: msg_per_host: origin=dynstats.bucket
Wed Feb 24 23:23:20 2021: imudp(*:514): origin=imudp submitted=0
Wed Feb 24 23:23:20 2021: imudp(*:514): origin=imudp submitted=0
Wed Feb 24 23:23:20 2021: imtcp(6514): origin=imtcp submitted=68743761
Wed Feb 24 23:23:20 2021: resource-usage: origin=impstats utime=6521531271
stime=8494790327 maxrss=20592 minflt=198177 majflt=47 inblock=106768
oublock=89664368 nvcsw=132661919 nivcsw=31780
Wed Feb 24 23:23:20 2021: main Q: origin=core.queue size=0 enqueued=69023867
full=0 discarded.full=0 discarded.nf=0 maxqsize=2284
Wed Feb 24 23:23:20 2021: imudp(w0): origin=imudp called.recvmmsg=0
called.recvmsg=0 msgs.received=0
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_1.com messages=14731
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_2.com messages=4885409
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_3.com messages=18499
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_4.com messages=18946
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_5.com messages=91936
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_6.com messages=33897
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_7.com messages=7901260
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_8.com messages=21403
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_9.com messages=4887634
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_10.com messages=4886624
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_11.com messages=103785
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_12.com messages=17910
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_13.com messages=7990763
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_14.com messages=89795
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_15.com messages=14669914
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_16.com messages=8018009
```
From the beginning I was wondering what the following action mean:
```
action 1: origin=core.action
action 2: origin=core.action
[...]
```
if I set in your example in the action, the parameter "name=something" then, in
(central) rsyslog server I will see this action statistics ?
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, February 24, 2021 9:31 AM, Mariusz Kruk via rsyslog
<[email protected]> wrote:
A ruleset is a "subprogram" processing your messages. If you have not
defined a specific ruleset, every message is getting processed by the
main rsyslog ruleset and the messages you read are in no way split
between different ruleset (thus all are processed according to the same
rules).
If you define multiple inputs with own rulesets, the events coming from
a particular input are processed only by the rules included in that ruleset.
Why would you want to do that? Let's imagine that you have an
installation where you want to log local events "normally" to a file but
you also receive events from the network and want to forward them to
some central log management solution. You could of course set up a
complicated set of filters which would choose only some subset of the
events to be written to files and another subset of the events to be
forwarded somewhere else, but it's way easier to just make an input
listening on a network port and tie it to a ruleset with action
forwarding it to another server.
Rulesets are a way of managing event processing flows. Additionally,
rulesets can have own queues which help dealing with reliability issues.
And again - if you have not set any rulesets explicitly, all your events
are processed by an implicit "main" ruleset containing all the filters
you put in the config file.
On 24.02.2021 08:01, odrzen via rsyslog wrote:
> Thanks Cyril for the awesome explanations and help. I understood some things
even better.
> I would like to ask you about the ruleset. I try the following and ( until
now ) works perfect:
>
> Module(load="imfile" mode="inotify")
>
> input(type="imfile"
> File="/var/log/httpd/*log"
> Tag="apache:"
>
>
> Why you also configure ruleset ?
> ( Although I read about the "facility", but I still don't fully understand
it, but this is another story, I will read it again. )
> Personally I have not set ruleset. So, is any specific implied by default or
not ?
> What do I gain by defining a specific one like you in your very nice example ?
> For example, Can I have better statistics on my messages ? Or.. something
else ?
> Thank you very much. I really appreciate your help.
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Monday, February 22, 2021 12:53 PM, [email protected] wrote:
>
> > Hi
> > Your OS most likely uses rsyslog for other things than just
messages/syslog. It often also handles maillog, cron-logs etc. Now in order to send
all the logs that rsyslog handles to a central logserver you can use the following
config:
> > . @123.123.123.123:514
> > If you use two @ signs it will use TCP and in the example above with just
one @ sign rsyslog will send the logs using UDP. You could also use the newer syntax
to achieve this but it's just very simple this way.
> > To send logs of other software that does not use rsyslog for logging I use
the imfile module to read those logfiles and then send the logs to the central
logserver. You can use the following config to do this:
> > Module(load="imfile" mode="inotify")
> > ruleset(name="fwdapachetocentrallog"){
> > action(type="omfwd"
> > template="RSYSLOG_TraditionalForwardFormat"
> > queue.type="LinkedList"
> > queue.filename="fwd_q_apache"
> > queue.size="100000"
> > action.resumeRetryCount="-1"
> > queue.saveonshutdown="on"
> > Target="123.123.123.123" Port="514" Protocol="udp")
> > }
> > Input(type="imfile" tag="apache/access.log" file="/var/log/httpd/local-access_log"
facility="local1" ruleset="fwdapachetocentrallog")
> > Input(type="imfile" tag="apache/err.log" file="/var/log/httpd/local-error_log"
facility="local2" ruleset="fwdapachetocentrallog")
> > Best,
> > Cyril
> >
> > > ----- Original message -----
> > > From: "odrzen via rsyslog" [email protected]
> > > Sent by: "rsyslog" [email protected]
> > > To: "rsyslog-users" [email protected]
> > > Cc: "odrzen" [email protected]
> > > Subject: [rsyslog] The right way to include more log files?
> > > Date: Mon, Feb 22, 2021 8:50 AM
> > > I am new with rsyslog and I have the following concern:
> > > How can I send from rsyslog clients, logs of services other than those
written in /var/log/messagess ?
> > > For example the logs from :
> > >
> > > - /var/log/service_x/*
> > > - /var/log/service_y/*
> > >
> > > Can I configure the rsyslog client to include more paths to log files ?
> > > I tried the following with apache logs, through a pipe to logger utility(
/etc/apache2/..virtualHost.conf ):
> > >
> > > CustomLog "| /bin/sh -c '/usr/bin/tee -a /var/log/httpd/local-access_log |
/usr/bin/logger -thttpd -plocal1.notice'" combined
> > > ErrorLog "|/bin/sh -c '/usr/bin/tee -a /var/log/httpd/local-error_log |
/usr/bin/logger -thttpd -plocal1.err'"
> > >
> > >
> > > and without any configuration either to the rsyslog client or to the
rsyslog server I receive the logs on my ryslog server.
> > > But, I don't like this way, and I don't think it's the best way to do it.
In addition, what will I do in other services ( databases for example ) ?
> > > Thanks in advance,
> > >
> > > rsyslog mailing list
> > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > http://www.rsyslog.com/professional-services/
> > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
