On Tue, 22 Oct 2013, Pavel Levshin wrote:
22.10.2013 14:39, David Lang:
actualy it will, you see 256 is the max batch size, rsyslog will grab
_up_to_ 256 messages for the batch, so if you incrament per message you
will seldom be incrimenting by 256 (unless your system is always running
behind)
but if I am understanding things correctly, putting set $/counter =
$/counter +1; in your config will end up incrimenting counter by 1 per
batch, not by the number of messages in the batch, with the race condition
that two different threads may end up racing and ending up with $/counter
only going up by 1 when it should go up by 2.
No, unfortunately, you are wrong twice.
First, at loaded server you will get full batches from time to time. And,
then, next batch will probably get to the same destination action. This will
lead to avalanche overload.
a couple batches going to the same destination will cause that destination to
slow, but unless the full batches continue for quite a while, you will end up
recovering.
If we, by chance, get a partial batch, it will be good for load balancing, as
it brings some randomization.
exactly, I'm saying that unless the system is really badly overloaded, partial
batches are going to be relatively common.
remember that the threads all want to start work as quickly as possible, so they
do not wait at all, you would have to have a large enough overload to have all
the threads doing max size batches at once, just a few of them won't cause real
problems.
Second, globals are incremented once per message, not per batch.
I stand corrected on that.
Yes, it would be better if the spreading factor and batchsize were not multiples
of each other, but I don't think that's very hard to do. batch sizes tend to be
nice round numbers (especially with the defaults being nice round binary
numbers), just pick an odd spreading factor and a power of 2 batchsize and you
are guaranteed to be safe :-)
David Lang
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
