in rsyslog, batches don't wait to be filled. the algorithm is designed for
low-latency, in that each time a batch is processed, it takes from 1-batchsize
messages (how ever many are available) and processes them.
so if you only have one message arrive per minute, each message will be
processed individually. You have to have messages arriving faster than they can
be individually processed before the batches will contain more than one message.
David Lang
On Wed, 18 Nov 2015, Otis Gospodnetić wrote:
Date: Wed, 18 Nov 2015 17:05:02 -0500
From: Otis Gospodnetić <[email protected]>
Reply-To: rsyslog-users <[email protected]>
To: rsyslog-users <[email protected]>
Subject: [rsyslog] omelasticsearch: send every N seconds even if bulk not
full?
Hi,
omelasticseearch has options like:
queue.type="linkedlist"
queue.size="5000"
queue.dequeuebatchsize="300"
action.resumeretrycount="-1")
What happens when the volume of logs is very low. For example, imagine a
log file where a new log event shows up only once a minute.
Looking at
http://www.rsyslog.com/doc/v8-stable/configuration/modules/omelasticsearch.html
I don't see a mention of "timeout" or "send every N seconds even if the
specified bulk size has not been reached yet".
Is there an option for that?
Thanks,
Otis
--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
