First of all, Thank you very much for all this analysis.
So, if I define my own rulesets in some rsyslog "client" to send this
messages/logs to a central rsyslog server, then from the server side, I don't
need to define something, right ?
Do these only apply to the local machine. Right ?
Another very easy example: if a machine doesn't run many services, but only
1-2, and we want to send the logs to a remote machine, in this case, too, would
you suggest creating rulesets ?
Well .. is it for better performance and easier management in various
situations ?
Can I use these rulesets and actions even more ?
> Additionally, rulesets can have own queues which help dealing with
> reliability issues.
Can I even use them on rsyslog (central) server to have better statistics ?
For example, I have now done this configuration:
module(load="impstats"
interval="600"
severity="7"
log.syslog="off")
global(senders.keepTrack="on")
dyn_stats(name="msg_per_host")
and I get the following results:
```
Wed Feb 24 23:23:20 2021: global: origin=dynstats msg_per_host.ops_overflow=0
msg_per_host.new_metric_add=0 msg_per_host.no_metric=0
msg_per_host.metrics_purged=0 msg_per_host.ops_ignored=0
msg_per_host.purge_triggered=142
Wed Feb 24 23:23:20 2021: imuxsock: origin=imuxsock submitted=0
ratelimit.discarded=0 ratelimit.numratelimiters=0
Wed Feb 24 23:23:20 2021: dynafile cache Auditlog: origin=omfile
requests=1611936 level0=979202 missed=26491 evicted=26362 maxused=10
closetimeouts=0
Wed Feb 24 23:23:20 2021: action 0: origin=core.action processed=1611936
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: dynafile cache RemoteLogs: origin=omfile
requests=69023867 level0=7424680 missed=930036 evicted=929906 maxused=10
closetimeouts=0
Wed Feb 24 23:23:20 2021: action 1: origin=core.action processed=69023867
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 2: origin=core.action processed=67365327
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 3: origin=core.action processed=1611936
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 4: origin=core.action processed=21400
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 5: origin=core.action processed=25204
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 6: origin=core.action processed=0
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 7: origin=core.action processed=0
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: action 8: origin=core.action processed=0
failed=0 suspended=0 suspended.duration=0 resumed=0
Wed Feb 24 23:23:20 2021: msg_per_host: origin=dynstats.bucket
Wed Feb 24 23:23:20 2021: imudp(*:514): origin=imudp submitted=0
Wed Feb 24 23:23:20 2021: imudp(*:514): origin=imudp submitted=0
Wed Feb 24 23:23:20 2021: imtcp(6514): origin=imtcp submitted=68743761
Wed Feb 24 23:23:20 2021: resource-usage: origin=impstats utime=6521531271
stime=8494790327 maxrss=20592 minflt=198177 majflt=47 inblock=106768
oublock=89664368 nvcsw=132661919 nivcsw=31780
Wed Feb 24 23:23:20 2021: main Q: origin=core.queue size=0 enqueued=69023867
full=0 discarded.full=0 discarded.nf=0 maxqsize=2284
Wed Feb 24 23:23:20 2021: imudp(w0): origin=imudp called.recvmmsg=0
called.recvmsg=0 msgs.received=0
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_1.com messages=14731
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_2.com messages=4885409
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_3.com messages=18499
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_4.com messages=18946
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_5.com messages=91936
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_6.com messages=33897
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_7.com messages=7901260
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_8.com messages=21403
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_9.com messages=4887634
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_10.com messages=4886624
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_11.com messages=103785
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_12.com messages=17910
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_13.com messages=7990763
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_14.com messages=89795
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_15.com messages=14669914
Wed Feb 24 23:23:20 2021: _sender_stat: sender=example_16.com messages=8018009
```
From the beginning I was wondering what the following action mean:
```
action 1: origin=core.action
action 2: origin=core.action
[...]
```
if I set in your example in the action, the parameter "name=something" then, in
(central) rsyslog server I will see this action statistics ?
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, February 24, 2021 9:31 AM, Mariusz Kruk via rsyslog
<[email protected]> wrote:
> A ruleset is a "subprogram" processing your messages. If you have not
> defined a specific ruleset, every message is getting processed by the
> main rsyslog ruleset and the messages you read are in no way split
> between different ruleset (thus all are processed according to the same
> rules).
>
> If you define multiple inputs with own rulesets, the events coming from
> a particular input are processed only by the rules included in that ruleset.
>
> Why would you want to do that? Let's imagine that you have an
> installation where you want to log local events "normally" to a file but
> you also receive events from the network and want to forward them to
> some central log management solution. You could of course set up a
> complicated set of filters which would choose only some subset of the
> events to be written to files and another subset of the events to be
> forwarded somewhere else, but it's way easier to just make an input
> listening on a network port and tie it to a ruleset with action
> forwarding it to another server.
>
> Rulesets are a way of managing event processing flows. Additionally,
> rulesets can have own queues which help dealing with reliability issues.
>
> And again - if you have not set any rulesets explicitly, all your events
> are processed by an implicit "main" ruleset containing all the filters
> you put in the config file.
>
> On 24.02.2021 08:01, odrzen via rsyslog wrote:
>
> > Thanks Cyril for the awesome explanations and help. I understood some
> > things even better.
> > I would like to ask you about the ruleset. I try the following and ( until
> > now ) works perfect:
> >
> > Module(load="imfile" mode="inotify")
> >
> > input(type="imfile"
> > File="/var/log/httpd/*log"
> > Tag="apache:"
> >
> >
> > Why you also configure ruleset ?
> > ( Although I read about the "facility", but I still don't fully understand
> > it, but this is another story, I will read it again. )
> > Personally I have not set ruleset. So, is any specific implied by default
> > or not ?
> > What do I gain by defining a specific one like you in your very nice
> > example ?
> > For example, Can I have better statistics on my messages ? Or.. something
> > else ?
> > Thank you very much. I really appreciate your help.
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Monday, February 22, 2021 12:53 PM, [email protected] wrote:
> >
> > > Hi
> > > Your OS most likely uses rsyslog for other things than just
> > > messages/syslog. It often also handles maillog, cron-logs etc. Now in
> > > order to send all the logs that rsyslog handles to a central logserver
> > > you can use the following config:
> > > . @123.123.123.123:514
> > > If you use two @ signs it will use TCP and in the example above with just
> > > one @ sign rsyslog will send the logs using UDP. You could also use the
> > > newer syntax to achieve this but it's just very simple this way.
> > > To send logs of other software that does not use rsyslog for logging I
> > > use the imfile module to read those logfiles and then send the logs to
> > > the central logserver. You can use the following config to do this:
> > > Module(load="imfile" mode="inotify")
> > > ruleset(name="fwdapachetocentrallog"){
> > > action(type="omfwd"
> > > template="RSYSLOG_TraditionalForwardFormat"
> > > queue.type="LinkedList"
> > > queue.filename="fwd_q_apache"
> > > queue.size="100000"
> > > action.resumeRetryCount="-1"
> > > queue.saveonshutdown="on"
> > > Target="123.123.123.123" Port="514" Protocol="udp")
> > > }
> > > Input(type="imfile" tag="apache/access.log"
> > > file="/var/log/httpd/local-access_log" facility="local1"
> > > ruleset="fwdapachetocentrallog")
> > > Input(type="imfile" tag="apache/err.log"
> > > file="/var/log/httpd/local-error_log" facility="local2"
> > > ruleset="fwdapachetocentrallog")
> > > Best,
> > > Cyril
> > >
> > > > ----- Original message -----
> > > > From: "odrzen via rsyslog" [email protected]
> > > > Sent by: "rsyslog" [email protected]
> > > > To: "rsyslog-users" [email protected]
> > > > Cc: "odrzen" [email protected]
> > > > Subject: [rsyslog] The right way to include more log files?
> > > > Date: Mon, Feb 22, 2021 8:50 AM
> > > > I am new with rsyslog and I have the following concern:
> > > > How can I send from rsyslog clients, logs of services other than those
> > > > written in /var/log/messagess ?
> > > > For example the logs from :
> > > >
> > > > - /var/log/service_x/*
> > > > - /var/log/service_y/*
> > > >
> > > > Can I configure the rsyslog client to include more paths to log files ?
> > > > I tried the following with apache logs, through a pipe to logger
> > > > utility( /etc/apache2/..virtualHost.conf ):
> > > >
> > > > CustomLog "| /bin/sh -c '/usr/bin/tee -a
> > > > /var/log/httpd/local-access_log | /usr/bin/logger -thttpd
> > > > -plocal1.notice'" combined
> > > > ErrorLog "|/bin/sh -c '/usr/bin/tee -a
> > > > /var/log/httpd/local-error_log | /usr/bin/logger -thttpd -plocal1.err'"
> > > >
> > > >
> > > > and without any configuration either to the rsyslog client or to the
> > > > rsyslog server I receive the logs on my ryslog server.
> > > > But, I don't like this way, and I don't think it's the best way to do
> > > > it. In addition, what will I do in other services ( databases for
> > > > example ) ?
> > > > Thanks in advance,
> > > >
> > > > rsyslog mailing list
> > > > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > > > http://www.rsyslog.com/professional-services/
> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
> > > > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
> > > > if you DON'T LIKE THAT.
> >
> > rsyslog mailing list
> > https://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> > LIKE THAT.
>
> rsyslog mailing list
> https://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
> sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T
> LIKE THAT.
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
