I don't see anything obviously wrong, except that the forwarding action has action.resumeRetryCount="-1" which means eternal blocking. Remove that param to make rsyslog throw away the messages when transmission fails after a retry. But the queue sizes are low (1000) and default for the main queue (whatever it was on that version). I do not see anything that could consume 2GiB of main memory.
Conclusion: may actually be a bug. I suggest to move on to 8.2001.0 and see if the problem persists. I guess it will go away. HTH Rainer El mié., 5 feb. 2020 a las 13:04, Vini Harimoorthy (<[email protected]>) escribió: > > > Please find the attached configurations. > > On Wed, Feb 5, 2020 at 3:25 PM Rainer Gerhards <[email protected]> > wrote: >> >> 2G+ memory is of course not a usual result, and it will not happen >> with default config (we keep around 10.000 msgs in queues). I guess if >> you show us your full config, we will be able to spot the problem. >> >> Also note that 8.36 is veeeeery old - maybe there indeed is a memory >> leak which is long-fixed. >> >> HTH >> Rainer >> >> El mié., 5 feb. 2020 a las 6:19, Vini Harimoorthy via rsyslog >> (<[email protected]>) escribió: >> > >> > Is this expected ?? How to l avoid buffering messages in rsyslogd ? Since >> > it is not acceptable that rsyslog consuming 2G + memory in 4G memory >> > system, >> > Which parameter determines this behavior ? I tried with queue size & >> > retrying logic, did;t seem to be working. >> > >> > >> > I have tried to load Impstats module. But it couldn't find module and logs >> > are below. >> > Is there anything I need to enable to load the module ? >> > >> > >> > Feb 05 05:16:37 rsyslogd[8064]: *could not load module >> > '/usr/lib/rsyslog/impstats.so', dlopen: /usr/lib/rsyslog/impstats.so: >> > *cannot >> > open shared object file: No such file or directory [v8.36.0 try >> > http://www.rsyslog.com/e/2066 ] >> > Feb 05 05:16:37 systemd[1]: Started System Logging Service. >> > Feb 05 05:16:37 rsyslogd[8064]: imuxsock: Acquired UNIX socket >> > '/run/systemd/journal/syslog' (fd 3) from systemd. [v8.36.0] >> > Feb 05 05:16:37 rsyslogd[8064]: [origin software="*rsyslogd" >> > swVersion="8.36.0"* x-pid="8064" x-info="http://www.rsyslog.com";] start >> > >> > >> > On Wed, Feb 5, 2020 at 1:57 AM David Lang via rsyslog < >> > [email protected]> wrote: >> > >> > > enable impstats to see the queue sizes and resources consumed >> > > >> > > David Lang >> > > >> > > On Tue, 4 Feb 2020, John Chivian via rsyslog wrote: >> > > >> > > > Date: Tue, 4 Feb 2020 07:45:35 -0600 >> > > > From: John Chivian via rsyslog <[email protected]> >> > > > To: [email protected] >> > > > Cc: John Chivian <[email protected]> >> > > > Subject: Re: [rsyslog] rsyslogd memory leak >> > > > >> > > > That's probably not a memory leak, but rather rsyslog queuing the >> > > messages in >> > > > memory until the remote system is again available and the queue can be >> > > > drained. >> > > > >> > > > Regards, >> > > > >> > > > >> > > > On 2/4/20 7:32 AM, Vini Harimoorthy via rsyslog wrote: >> > > >> I am seeing rsyslogd memory leak when remote server is unreachable. >> > > >> >> > > >> I tried to change resumeRetryCount ="0" & queue.size ="0", so that >> > > rsyslogd >> > > >> won't care much about retrying when remote server becomes reachable >> > > again. >> > > >> >> > > >> When remote server is reachable, it stays at 1.5MB. >> > > >> */home/admin# ps_mem | grep rsyslog* >> > > >> * 1.5 MiB + 31.5 KiB = 1.5 MiB rsyslogd* >> > > >> >> > > >> >> > > >> When remote server is unreachable, it is increasing by 1 MB. And it >> > > >> goes >> > > >> upto 2GB after 24 hours . >> > > >> >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> 1.6 MiB + 31.5 KiB = 1.6 MiB rsyslogd >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> 1.6 MiB + 31.5 KiB = 1.6 MiB rsyslogd >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> 1.6 MiB + 31.5 KiB = 1.6 MiB rsyslogd >> > > >> /home/admin# >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> 1.6 MiB + 31.5 KiB = 1.7 MiB rsyslogd >> > > >> >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> * 2.4 MiB + 31.5 KiB = 2.5 MiB rsyslogd* >> > > >> >> > > >> /home/admin# date >> > > >> Tue Feb 4 13:23:13 UTC 2020 >> > > >> >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> * 3.8 MiB + 31.5 KiB = 3.8 MiB rsyslogd* >> > > >> >> > > >> >> > > >> /home/admin# date >> > > >> Tue Feb 4 13:26:32 UTC 2020 >> > > >> /home/admin# ps_mem | grep rsyslog >> > > >> * 5.2 MiB + 31.5 KiB = 5.2 MiB rsyslogd* >> > > >> >> > > >> >> > > >> When I made the remote server reachable again, it is reducing again >> > > >> >> > > >> :/home/admin# date >> > > >> Tue Feb 4 13:29:16 UTC 2020 >> > > >> :/home/admin# ps_mem | grep rsyslog >> > > >> 2.6 MiB + 31.5 KiB = 2.6 MiB rsyslogd >> > > >> >> > > >> >> > > >> Could you please check this issue . Below is my remote configurations >> > > >> >> > > >> */home/admin# cat /etc/rsyslog.remote.conf* >> > > >> $DefaultNetstreamDriverCAFile >> > > >> /fs/security/certmgr/apps/syslog-client/syslog-client.crt >> > > >> $DefaultNetstreamDriverCertFile >> > > >> /fs/security/certmgr/apps/syslog-client/syslog-client.crt >> > > >> $DefaultNetstreamDriverKeyFile >> > > >> /fs/security/certmgr/apps/syslog-client/syslog-client.key >> > > >> set $!updatepri = $syslogseverity + 184; >> > > >> $template remoteFmt,"<%$!updatepri%>1 %TIMESTAMP:::date-rfc3339% >> > > %HOSTNAME% >> > > >> %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA%%msg%\n" >> > > >> ruleset(name="logger1"){ >> > > >> action(type="omfwd" Target="20.20.20.2" Port="514" Protocol="tcp" >> > > >> NetworkNamespace="swns" template="remoteFmt"Address="None" >> > > >> *queue.type="LinkedList" >> > > >> action.resumeRetryCount="-1" *queue.size="1000") >> > > >> } >> > > >> >> > > >> if($msg contains "Event|") then { >> > > >> if $syslogseverity <= 6 then { >> > > >> call logger1 >> > > >> } >> > > >> } >> > > >> >> > > >> >> > > > >> > > > _______________________________________________ >> > > > rsyslog mailing list >> > > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > > http://www.rsyslog.com/professional-services/ >> > > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >> > > > myriad >> > > of >> > > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T >> > > > LIKE THAT. >> > > > >> > > _______________________________________________ >> > > rsyslog mailing list >> > > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > > http://www.rsyslog.com/professional-services/ >> > > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > > DON'T LIKE THAT. >> > > >> > >> > >> > -- >> > *Regards,* >> > *Vinitha H* >> > _______________________________________________ >> > rsyslog mailing list >> > http://lists.adiscon.net/mailman/listinfo/rsyslog >> > http://www.rsyslog.com/professional-services/ >> > What's up with rsyslog? Follow https://twitter.com/rgerhards >> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> > DON'T LIKE THAT. > > > > -- > Regards, > Vinitha H > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
