Thx, it is sufficient. But it's strange, I don't see anything obviously causing a lot of time. Could you, for an experiment, do:
global(net.enableDNS="off") and see if that changes the picture? Rainer El lun., 5 oct. 2020 a las 15:04, Kype Ahamed (<[email protected]>) escribió: > > Hi Rainer > > 1 minute generates too large a file to be submitted. I have reduced my > message rate and ran it for a few seconds > > Hope its sufficient > > > Regards, > Scorsese P. > > On Mon, Oct 5, 2020 at 6:30 PM Rainer Gerhards <[email protected]> > wrote: >> >> That sounds strange. Could you create a debug log and post. If so, let >> it run for a minute or so during a busy time, so that we can see what >> actually happens during processing. >> >> Rainer >> >> El lun., 5 oct. 2020 a las 11:59, Kype Ahamed >> (<[email protected]>) escribió: >> > >> > Hi Rainer >> > >> > This is running on a Intel CPU Virtual Machine (specifically an i5-6600 >> > CPU) >> > >> > Rsyslog version: rsyslogd 8.24.0-57.el7_9 >> > >> > >> > Regards, >> > Scorsese P. >> > >> > >> > On Mon, Oct 5, 2020 at 2:37 PM Rainer Gerhards via rsyslog >> > <[email protected]> wrote: >> >> >> >> Nothing in the config really looks like it would need a lot of processing >> >> time. >> >> >> >> Which machine is this run on (ARM?) and what rsyslog version is used? >> >> >> >> Rainer >> >> >> >> El dom., 4 oct. 2020 a las 17:12, Yuri Bushmelev via rsyslog >> >> (<[email protected]>) escribió: >> >> > >> >> > Hello! >> >> > >> >> > From what I see you have 5 conditions. 4 of them doing full-scan of >> >> > $msg on >> >> > every incoming message. What I'd suggest is to parse the message first >> >> > using a `mmnormalize`/`mmfields`/`mmpstructdata` module. Then you can >> >> > refer >> >> > to fields extracted in your conditions instead to prevent full message >> >> > scan. >> >> > >> >> > I can guess you may be using the iptables message format. So you may >> >> > check >> >> > this liblognorm field type: >> >> > https://www.liblognorm.com/files/manual/configuration.html#iptables >> >> > >> >> > >> >> > On Sun, 4 Oct 2020 at 21:30, Kype Ahamed via rsyslog < >> >> > [email protected]> wrote: >> >> > >> >> > > Hello, >> >> > > >> >> > > I am using rsyslog mainly as a syslog relay, to forward messages from >> >> > > 1 >> >> > > source device to multiple destination devices. >> >> > > >> >> > > Right now i am receiving about 50k messages per second, and i noticed >> >> > > CPU >> >> > > usage is constantly above 80% >> >> > > >> >> > > Are there any further tweaks that can be done to below config to >> >> > > reduce the >> >> > > CPU usage? >> >> > > >> >> > > ------------------------------------------------- >> >> > > rsyslog config file: >> >> > > >> >> > > module(load="impstats" >> >> > > interval="20" >> >> > > severity="7" >> >> > > log.syslog="off" >> >> > > log.file="/var/log/impstats.log") >> >> > > >> >> > > global(parser.escapecontrolcharactertab="off") >> >> > > >> >> > > # Load Modules # >> >> > > module(load="imudp" TimeRequery="5" BatchSize="64") >> >> > > >> >> > > # rsyslog Templates # >> >> > > template(name="testMachineHeader" type="string" >> >> > > string="%TIMESTAMP:::date-rfc3164% testMachine >> >> > > %rawmsg:::drop-last-lf%\n") >> >> > > template(name="rawTemplate" type="string" >> >> > > string="%rawmsg:::drop-last-lf%\n") >> >> > > >> >> > > # rsyslog Input Modules # >> >> > > input(type="imudp" >> >> > > port="10514" >> >> > > ruleset="forwardToDestRule" >> >> > > device="eth0" >> >> > > ) >> >> > > >> >> > > >> >> > > >> >> > > # rsyslog RuleSets # >> >> > > ruleset(name="forwardToDestRule" >> >> > > queue.type="fixedArray" >> >> > > queue.size="25000" >> >> > > ) { >> >> > > if ($msg contains "interface=inbound" and $msg contains >> >> > > "source=10.1.1.1") >> >> > > then { >> >> > > action(type="omfwd" >> >> > > Target="10.1.1.5" >> >> > > Port="514" >> >> > > Protocol="tcp" >> >> > > Device="eth0" >> >> > > queue.type="fixedArray" >> >> > > queue.size="50000" >> >> > > queue.dequeueBatchSize="1024" >> >> > > template="testMachineHeader") >> >> > > } >> >> > > else{ >> >> > > action(type="omfwd" >> >> > > Target="10.1.1.6" >> >> > > Port="514" >> >> > > Protocol="udp" >> >> > > Device="eth0" >> >> > > queue.type="fixedArray" >> >> > > queue.size="50000" >> >> > > action.resumeRetryCount="-1" >> >> > > template="rawTemplate") >> >> > > } >> >> > > >> >> > > if ($msg contains "interface=outbound" and $msg contains >> >> > > "source=10.1.1.1") >> >> > > then { >> >> > > if ($msg contains "proto=17") then { >> >> > > action(type="omfwd" >> >> > > Target="10.1.1.7" >> >> > > Port="514" >> >> > > Protocol="udp" >> >> > > Device="eth0" >> >> > > queue.type="linkedlist" >> >> > > queue.size="50000" >> >> > > action.resumeRetryCount="-1" >> >> > > template="rawTemplate") >> >> > > } >> >> > > } >> >> > > >> >> > > } >> >> > > >> >> > > ------------------------------------------------- >> >> > > >> >> > > Top -H output: >> >> > > >> >> > > top - 08:53:26 up 1:55, 1 user, load average: 2.32, 1.50, 1.07 >> >> > > Threads: 112 total, 7 running, 105 sleeping, 0 stopped, 0 zombie >> >> > > %Cpu(s): 23.4 us, 53.5 sy, 0.0 ni, 3.5 id, 0.4 wa, 0.0 hi, 19.1 >> >> > > si, >> >> > > 0.0 st >> >> > > KiB Mem : 7972668 total, 7475888 free, 250088 used, 246692 >> >> > > buff/cache >> >> > > KiB Swap: 4063228 total, 4063228 free, 0 used. 7449764 >> >> > > avail Mem >> >> > > >> >> > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ >> >> > > COMMAND >> >> > > 2567 root 20 0 639540 16836 12416 R 32.4 0.2 1:11.53 >> >> > > rs:action 2 que >> >> > > 2566 root 20 0 639540 16836 12416 R 30.1 0.2 1:11.32 >> >> > > rs:action 1 que >> >> > > 2551 root 20 0 639540 16836 12416 R 14.1 0.2 0:30.68 >> >> > > in:imudp >> >> > > 2565 root 20 0 639540 16836 12416 R 11.1 0.2 0:31.94 >> >> > > rs:forwardToDes >> >> > > 600 root 20 0 100676 38508 38184 S 6.2 0.5 0:33.62 >> >> > > systemd-journal >> >> > > >> >> > > ------------------------------------------------- >> >> > > >> >> > > impstats output: >> >> > > >> >> > > Sun Oct 4 08:46:49 2020: global: origin=dynstats >> >> > > Sun Oct 4 08:46:49 2020: imuxsock: origin=imuxsock submitted=0 >> >> > > ratelimit.discarded=0 ratelimit.numratelimiters=0 >> >> > > Sun Oct 4 08:46:49 2020: action 0: origin=core.action processed=0 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 1: origin=core.action >> >> > > processed=1341545 >> >> > > failed=0 suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 2: origin=core.action >> >> > > processed=1341545 >> >> > > failed=0 suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 3: origin=core.action processed=4022 >> >> > > failed=0 suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 4: origin=core.action processed=1 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 5: origin=core.action processed=0 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 6: origin=core.action processed=0 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 7: origin=core.action processed=0 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 8: origin=core.action processed=0 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: action 9: origin=core.action processed=0 >> >> > > failed=0 >> >> > > suspended=0 suspended.duration=0 resumed=0 >> >> > > Sun Oct 4 08:46:49 2020: imudp(*:10514): origin=imudp >> >> > > submitted=1341849 >> >> > > Sun Oct 4 08:46:49 2020: imudp(*:10514): origin=imudp submitted=0 >> >> > > Sun Oct 4 08:46:49 2020: resource-usage: origin=impstats >> >> > > utime=9190927 >> >> > > stime=25608171 maxrss=12244 minflt=166970 majflt=0 inblock=0 >> >> > > oublock=4368 >> >> > > nvcsw=37503 nivcsw=339 >> >> > > Sun Oct 4 08:46:49 2020: action 0 queue: origin=core.queue size=0 >> >> > > enqueued=0 full=0 discarded.full=0 discarded.nf=0 maxqsize=0 >> >> > > Sun Oct 4 08:46:49 2020: action 1 queue: origin=core.queue size=0 >> >> > > enqueued=1341545 full=0 discarded.full=0 discarded.nf=0 maxqsize=5227 >> >> > > Sun Oct 4 08:46:49 2020: action 2 queue: origin=core.queue size=252 >> >> > > enqueued=1341545 full=0 discarded.full=0 discarded.nf=0 maxqsize=6051 >> >> > > Sun Oct 4 08:46:49 2020: forwardToDestRule: origin=core.queue >> >> > > size=304 >> >> > > enqueued=1341849 full=0 discarded.full=0 discarded.nf=0 maxqsize=1003 >> >> > > Sun Oct 4 08:46:49 2020: main Q: origin=core.queue size=0 >> >> > > enqueued=4023 >> >> > > full=0 discarded.full=0 discarded.nf=0 maxqsize=64 >> >> > > Sun Oct 4 08:46:49 2020: imudp(w0): origin=imudp >> >> > > called.recvmmsg=40859 >> >> > > called.recvmsg=0 msgs.received=1341849 >> >> > > >> >> > > ------------------------------------------------- >> >> > > >> >> > > >> >> > > Regards, >> >> > > Scorsese P. >> >> > > _______________________________________________ >> >> > > rsyslog mailing list >> >> > > https://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > > http://www.rsyslog.com/professional-services/ >> >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> > > myriad >> >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> > > DON'T LIKE THAT. >> >> > > >> >> > >> >> > >> >> > -- >> >> > Yury Bushmelev >> >> > _______________________________________________ >> >> > rsyslog mailing list >> >> > https://lists.adiscon.net/mailman/listinfo/rsyslog >> >> > http://www.rsyslog.com/professional-services/ >> >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> >> > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >> >> > if you DON'T LIKE THAT. >> >> _______________________________________________ >> >> rsyslog mailing list >> >> https://lists.adiscon.net/mailman/listinfo/rsyslog >> >> http://www.rsyslog.com/professional-services/ >> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> >> DON'T LIKE THAT. _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
