Thank you for getting back. Makes sense. One more question related to this. If we dont need logs in the order they were received, then what is beneficial - multiple threads in ruleset queue vs multiple threads in action queue. I read your blog but still have some doubts. Assume we don’t need any filtering. And there are 2 omfwd actions sending logs to tcp destinations within the ruleset. In this scenario - should i create a ruleset queue with 4 threads(example) or create the actions each with 2 threads(example). Or is there a benefit to having both. Of course am assuming large mps like 500k.
Regards, Sid On Wed, Apr 7, 2021 at 12:51 AM Rainer Gerhards <[email protected]> wrote: > The actions are simply far too fast to see any difference. For that > case, a queue would just add overhead. It's not for actions that fast. > > HTH > Rainer > > El mié, 7 abr 2021 a las 5:02, siddhartha dewan via rsyslog > (<[email protected]>) escribió: > > > > Hi, > > this is my 1st time reaching out to the public community - so pls advise > if > > am doing something wrong. > > My question is related to ruleset queues and parallelism. After reviewing > > all the documentation on queues and reading them multiple times - the > whole > > idea behind the queue with a fixed or linked list was to introduce > > asynchronous processing of actions within the ruleset which also helps > with > > handling costly filter logic. > > > > In order to test the benefits of queues - i created a simple ruleset > > without a queue 1st: > > ruleset(name="test") { > > action(type="omfile" file="/var/log/mtu3.log") > > action(type="omfile" file="/var/log/mtu4.log") > > } > > > > As per the documentation - my expectation was - because i didn't define > any > > queues - the action would be processed synchronously - or - it will be > > processed 1 after another. Meaning - all the syslog packets will be > stored > > in the main queue - which then will invoke the 1st action and once the > 1st > > action is complete - it will then invoke the 2nd action. > > > > To my surprise - the data was written in parallel to both these files. > And > > now i am totally confused - if the data is being written in parallel - > how > > exactly is the cpu thread working: > > is it writing to both files at once? If yes - then what really is the > > benefit of a dedicated queue? > > In the documentation - the biggest benefit seems to be avoiding a slow > > output - and also helping provide parallel threads to do work while > > increasing performance. > > > > Yes - I understand that the slow output will eventually cause a backlog > in > > the main queue causing the entire rsyslog to hang - but is that the only > > benefit of a queue? Performance doesnt really seem to be a benefit if the > > main queue can already do things in parallel - till rsyslog eventually > > hangs due to backlog. > > > > Your team has done an awesome job helping out with an open source > solution. > > Pls keep it up. > > _______________________________________________ > > rsyslog mailing list > > https://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list https://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
