Thank you both for your answers, I'll update to Logstash 5.4.0, the persistent input queue are now GA! And normally it will be able to handle burst. Plus I'll be able to see the input queue status via the Logstash API.
I'll keep you informed. Ludovic -----Message d'origine----- De : David Lang [mailto:[email protected]] Envoyé : mardi 23 mai 2017 09:26 À : phrogz via rsyslog <[email protected]> Cc : phrogz <[email protected]> Objet : Re: [rsyslog] Slow log processing well, the fundamental problem is that logstash is not keeping up, so rsyslog has it's internal queues build up. once the queues are full, rsyslog only accepts new messages at the rate that the queus can be drained. unless you set rebindinterval, you will only be making one connection to logstash and the load balancer will not have a chance to send any traffic to the second instance. I'm not sure how much this will matter, as logstash doesn't have any internal queueing, so the normal strategy of sending a burst of traffic to one, disconnecting and reconnecting to let the load balancer work and send another burst of traffic may not really work as the logstash instances have no way of taking a bust and working through the backlog. look at your logstash instances and you will probably find that one is maxed out. David Lang _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
