On Thu, Sep 26, 2013 at 9:06 AM, David Lang <[email protected]> wrote: > On Thu, 26 Sep 2013, Rainer Gerhards wrote: > > On Thu, Sep 26, 2013 at 12:31 AM, David Lang <[email protected]> wrote: >> >> am I correct in thinking that this isn't going to help if each action is >>> in it's own queue? >>> >>> >>> Depends on the action, but in this case it hurts. Action queues only >> make >> sense for really slow actions. Where "slow" means things like databases or >> even remote hosts that may be offline for a while. In this setup, I'd >> expect that the action queue has nothing to do at all - even the filters >> are evaluated by the main queue threads (because that needs to happen >> before the action is called). As I said, I think the filters are the >> culprit here (and still do so). >> >> Keep in mind that action queues have an overhead: >> >> - need to duplicate the messsage (to guard against modification by other >> actions downstream) >> - keep real queue structures >> - overhead of mutex operations >> >> I would assume that the overhead here is much higher than the processing >> cost for omfile. So I conclude that action queues slow down things. >> > > Ok, I understand what you are saying, but he is getting much better > performance with a small number of main queues and action queues per action > than he was getting with no action queues and more main queue threads. >
mmmhh.. have I overlooked something (may be, especially as I was busy the past days)? My current understanding is that I recommended several times to drop the action queues, but that never happend. At least I have not seen any pstats without them. This is why I am so persistive with this point. I would like to see *the exact same config* with and without the action queues. That config must of course include imudp running at realtime priority. If I have overlooked this, would you be able to point me to the correct reply? >The reason I was looking at trying to use rulesets was to be able to split the test overhead between different threads (and have a > queue per ruleset rather than per action) > > what is the best way to do this? > At least in theory, there is no such good way. The reason again simply is that omfile is such a quick action (not counting IO), that the putting it into an action queue creates probably 5 times overhead (even more if we run into contention situations). Splitting of filter evaluation to several rulesets does no better than running the main queue on multiple threads. Again, this is *theory*. I would like to see some solid indication that the theory is wrong (see test to be carried out above). Even if it proves to be wrong, splitting off to rulesets would probably not make much sense. Instead it would be better to try to find out where the theory is wrong, and correct that. But I really can't say more unless I see this with and without action queue behaviour (with otherwise *unchanged* config during the tests). Sorry I have no better answer. Rainer > > David Lang > > ______________________________**_________________ > rsyslog mailing list > http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> > http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
