Hi, I upgraded a server to Rsyslog 8.15 last night and today the process was using almost 200MB of RAM (raising steadily). Tried running the process in Valgrind to see if I get an idea about what happens, but wasn't that much help for me.
If someone has better debugging skills, I pasted the output here. Not sure if I let it run enough or leave it running longer. https://gist.github.com/hakman/44afddaf4eb67cda28c6 Thanks, Ciprian -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Tue, Dec 15, 2015 at 2:41 PM, David Lang <[email protected]> wrote: > On Tue, 15 Dec 2015, Ciprian Hacman wrote: > > Hi David, >> >> maxMessageSize="10000" >> queue.size="10000" - main queue >> queue.size="10000" - elasticsearch queue >> >> Based on my calculations this brings me to a max of 200MB of memory, maybe >> a little more depending on how maxMessageSize is calculated. >> >> I read logs from a file and push them to Elasticsearch (on the same >> network), so TCP is the only possibility. This server has a very simple >> setup. >> >> If I don't find the reason for this issue, I might have to go implement >> the >> forwarding to a central location and push to Elasticsearch from there. >> > > There are advantages to sending things to a central server. > > it's one place to queue data, so you can either allocate more ram, or go > to disk as needed without impacting other workloads. > > it's more efficient, the central server is more likely to have larger > batches of data to feed to ES, and ES only needs to be running one thread > processing inbound data > > while it is one more point to have to look at, I think it simplifies > troubleshooting as all the communication to ES (and therefor all the errors > for such communication) happen in one place instead of distributed. > > anyway, let's see how things look with 8.15 > > > David Lang > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
