2015-12-16 13:48 GMT+01:00 Ciprian Hacman <[email protected]>: > Hi, > > I upgraded a server to Rsyslog 8.15 last night and today the process was > using almost 200MB of RAM (raising steadily). > Tried running the process in Valgrind to see if I get an idea about what > happens, but wasn't that much help for me.
That's because debug symbols are unloaded at module unload time. This makes valgrind stacktraces unusable. Nevertheless, the information looks very promising. Can you build rsyslog yourself for that box? All we need is the --enable-valgrind option, which will essentially remove the module unloads and make the stacktrace usable. Rainer > > If someone has better debugging skills, I pasted the output here. Not sure > if I let it run enough or leave it running longer. > https://gist.github.com/hakman/44afddaf4eb67cda28c6 > > Thanks, > Ciprian > > > -- > Performance Monitoring * Log Analytics * Search Analytics > Solr & Elasticsearch Support * http://sematext.com/ > > On Tue, Dec 15, 2015 at 2:41 PM, David Lang <[email protected]> wrote: > >> On Tue, 15 Dec 2015, Ciprian Hacman wrote: >> >> Hi David, >>> >>> maxMessageSize="10000" >>> queue.size="10000" - main queue >>> queue.size="10000" - elasticsearch queue >>> >>> Based on my calculations this brings me to a max of 200MB of memory, maybe >>> a little more depending on how maxMessageSize is calculated. >>> >>> I read logs from a file and push them to Elasticsearch (on the same >>> network), so TCP is the only possibility. This server has a very simple >>> setup. >>> >>> If I don't find the reason for this issue, I might have to go implement >>> the >>> forwarding to a central location and push to Elasticsearch from there. >>> >> >> There are advantages to sending things to a central server. >> >> it's one place to queue data, so you can either allocate more ram, or go >> to disk as needed without impacting other workloads. >> >> it's more efficient, the central server is more likely to have larger >> batches of data to feed to ES, and ES only needs to be running one thread >> processing inbound data >> >> while it is one more point to have to look at, I think it simplifies >> troubleshooting as all the communication to ES (and therefor all the errors >> for such communication) happen in one place instead of distributed. >> >> anyway, let's see how things look with 8.15 >> >> >> David Lang >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
