rainer:
What's main queue dequeuebatchsize meaning? I should define the
options in `action()`, `ruleset()` or `$MainMsgQueuedequeuebatchsize`?
I'm not so clear about ruleset queue and action queue. Especially one
time, I defined `queue.workerthreads="5"` both in my two different
rulesets, then found reciever got two copy...
And I didn't find too much example about those things, hope a better
document... Thanks very much!!
2015-04-08 17:28 GMT+08:00 Rainer Gerhards <[email protected]>:
> 2015-04-08 9:34 GMT+02:00 Radu Gheorghe <[email protected]>:
> > Hello,
> >
> > I have three questions about action queues:
> > 1) if I have a message modifier action (e.g. mmnormalize), should I add
> an
> > action queue for better performance?
>
> no, simply because that wouldn't work as expected. Remember that a
> queue runs async, so the modification would not get back to the
> original message object.
>
> > 2) what's the flow of messages if I have both mmnormalize and, say,
> > omelasticsearch? Messages flow from the main queue to mmnormalize's queue
> > (if any) and then back to the main queue to be processed by
> omelasticsearch?
>
> I guess I already answered this ;) With a queue, message objects are
> duplicated and never be injected back. that's the prime reason you
> want a queue.
>
> > 3) how does queue.dequeuebatchsize work if I have an action queue. For
> > example:
> >
> > * I have a main queue with dequeuebatchsize=100
> > * I have an omelasticsearch action with no queue (direct) and
> > dequeuebatchsize=1000 and BulkMode="on"
>
> dequeuebatchsize has no effect in direct mode.
>
> >
> > How many messages are sent to ES in a single bulk? 100? 1000? 1?
>
> at most 100 (fewer if the batch doesn't get full).
>
> > I can do a test to answer this last question, but I'd like to understand
> > how multiple actions and direct queues work. For example, I didn't notice
> > any difference in performance (during a benchmark) with and without an
> > action queue with mmnormalize. But maybe the bottleneck was somewhere
> else.
>
> queues always *cost* performance. If you need more concurrency, you
> should adjust the main or ruleset queue accordingly.
>
> HTH
> Rainer
> >
> > Best regards,
> > Radu
> > --
> > Performance Monitoring * Log Analytics * Search Analytics
> > Solr & Elasticsearch Support * http://sematext.com/
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com/professional-services/
> > What's up with rsyslog? Follow https://twitter.com/rgerhards
> > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
> DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
