On Mon, 21 Oct 2013, Rainer Gerhards wrote:
I think it's fine to just document for now.
I think the 'right' answer is to create a magic variable that is the
number of messages in the batch. This will allow you to have a counter of
the number of messages.
now, the counter will not give every message a unique value if you have
batch size > 1, but this is actually pretty much the case with other things
related to the batch (like timestamps on received messages), so it's not
unusual.
As far as load balancing goes, balancing a batch of messages at a time is
actually 'good enough', and since it saves you manipulating individual
messages, it's a performance win as well.
I think this variable shuffling is the right approach to load balancing. As
I wrote yesterday, I think it is fairly easy to do it inside the engine. If
I had not been lost in the vars, I would have started with this today and
were positive I could have finished it quickly. If the remaining time
permits, I'll probably still try to squeeze it in.
Since this can be dealt with by just setting the batch size = 1, I think
that it's acceptable.
yup, except for performance (the ES http request with batch size 1 ;)). But
your are basically right for most things.
well, most of the batch stuff involves an efficiency/precision trade-off, so
this seems reasonable.
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
