On Wed, 8 Apr 2015, chenlin rao wrote:
rainer:
What's main queue dequeuebatchsize meaning? I should define the
options in `action()`, `ruleset()` or `$MainMsgQueuedequeuebatchsize`?
legacy format would be $MainMsgQueuedequeuebatchsize
new format would be as part of the main_queue() definition
http://www.rsyslog.com/doc/master/rainerscript/queue_parameters.html
David Lang
I'm not so clear about ruleset queue and action queue. Especially one
time, I defined `queue.workerthreads="5"` both in my two different
rulesets, then found reciever got two copy...
And I didn't find too much example about those things, hope a better
document... Thanks very much!!
2015-04-08 17:28 GMT+08:00 Rainer Gerhards <[email protected]>:
2015-04-08 9:34 GMT+02:00 Radu Gheorghe <[email protected]>:
Hello,
I have three questions about action queues:
1) if I have a message modifier action (e.g. mmnormalize), should I add
an
action queue for better performance?
no, simply because that wouldn't work as expected. Remember that a
queue runs async, so the modification would not get back to the
original message object.
2) what's the flow of messages if I have both mmnormalize and, say,
omelasticsearch? Messages flow from the main queue to mmnormalize's queue
(if any) and then back to the main queue to be processed by
omelasticsearch?
I guess I already answered this ;) With a queue, message objects are
duplicated and never be injected back. that's the prime reason you
want a queue.
3) how does queue.dequeuebatchsize work if I have an action queue. For
example:
* I have a main queue with dequeuebatchsize=100
* I have an omelasticsearch action with no queue (direct) and
dequeuebatchsize=1000 and BulkMode="on"
dequeuebatchsize has no effect in direct mode.
How many messages are sent to ES in a single bulk? 100? 1000? 1?
at most 100 (fewer if the batch doesn't get full).
I can do a test to answer this last question, but I'd like to understand
how multiple actions and direct queues work. For example, I didn't notice
any difference in performance (during a benchmark) with and without an
action queue with mmnormalize. But maybe the bottleneck was somewhere
else.
queues always *cost* performance. If you need more concurrency, you
should adjust the main or ruleset queue accordingly.
HTH
Rainer
Best regards,
Radu
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
