On Thu, 26 Sep 2013, Rainer Gerhards wrote:
On Thu, Sep 26, 2013 at 9:06 AM, David Lang <[email protected]> wrote:
On Thu, 26 Sep 2013, Rainer Gerhards wrote:
On Thu, Sep 26, 2013 at 12:31 AM, David Lang <[email protected]> wrote:
am I correct in thinking that this isn't going to help if each action is
in it's own queue?
Depends on the action, but in this case it hurts. Action queues only
make
sense for really slow actions. Where "slow" means things like databases or
even remote hosts that may be offline for a while. In this setup, I'd
expect that the action queue has nothing to do at all - even the filters
are evaluated by the main queue threads (because that needs to happen
before the action is called). As I said, I think the filters are the
culprit here (and still do so).
Keep in mind that action queues have an overhead:
- need to duplicate the messsage (to guard against modification by other
actions downstream)
- keep real queue structures
- overhead of mutex operations
I would assume that the overhead here is much higher than the processing
cost for omfile. So I conclude that action queues slow down things.
Ok, I understand what you are saying, but he is getting much better
performance with a small number of main queues and action queues per action
than he was getting with no action queues and more main queue threads.
mmmhh.. have I overlooked something (may be, especially as I was busy the
past days)? My current understanding is that I recommended several times to
drop the action queues, but that never happend. At least I have not seen
any pstats without them. This is why I am so persistive with this point. I
would like to see *the exact same config* with and without the action
queues. That config must of course include imudp running at realtime
priority.
If I have overlooked this, would you be able to point me to the correct
reply?
This topic is a mess to try and find things (and there was a lot of stuff tested
with just the throughput numbers reported, no pstats or top reports)
Robert, could you test the same config you tried earlier (2 main threads), but
with the action queues removed entirely?
he did have a test with 8 main queue threads and no action queue threads, and it
was unable to keep up with 250K logs/sec
The reason I was looking at trying to use rulesets was to be able to split
the test overhead between different threads (and have a
queue per ruleset rather than per action)
what is the best way to do this?
At least in theory, there is no such good way. The reason again simply is
that omfile is such a quick action (not counting IO), that the putting it
into an action queue creates probably 5 times overhead (even more if we run
into contention situations).
Splitting of filter evaluation to several rulesets does no better than
running the main queue on multiple threads.
ahh, I thought that there were other ways to do this
a couple of interesting things to note in the top results.
the imudp thread is comeing close to maxing out a cpu
the main queue threads have been eating about the same CPU (per thread) with or
without action queues and with 2 or 8 main threads
David Lang
Again, this is *theory*. I would like to see some solid indication that the
theory is wrong (see test to be carried out above). Even if it proves to be
wrong, splitting off to rulesets would probably not make much sense.
Instead it would be better to try to find out where the theory is wrong,
and correct that.
But I really can't say more unless I see this with and without action queue
behaviour (with otherwise *unchanged* config during the tests).
Sorry I have no better answer.
Rainer
David Lang
______________________________**_________________
rsyslog mailing list
http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog>
http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/>
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
