Well thanks guys its a lot of help, I will further test but I have three questions at the moment, one I want you guys to see my threaded top but the only way I have found to output to file is "top -b -n 5 > topoutput" but I don't see all rsyslog threads, so any suggestions would be great, second, i didn't have an intention to run an action 2 on one single rule, how do I create an action queue for each rules? And lastly how can I run imudp in real-time priority?
Thanks Rainer Gerhards <[email protected]> wrote: >If you wanna test something, it may be interesting to see if you can >get >imudp thread to 100% cpu and dropping msgs at that point. That would >prove >it would make sense to add a thread pool. > >Sent from phone, thus brief. >Am 19.09.2013 20:13 schrieb "David Lang" <[email protected]>: > >> On Thu, 19 Sep 2013, Rainer Gerhards wrote: >> >> No matter how many inputs you define, imudp runs on one thread. I've >never >>> seen it outperform a single core ( in v7). >>> >> >> I'm glad I asked, it means that I don't waste time testing it :-) >> >> David Lang >> >> Sent from phone, thus brief. >>> Am 19.09.2013 20:08 schrieb "David Lang" <[email protected]>: >>> >>> On Thu, 19 Sep 2013, Rainer Gerhards wrote: >>>> >>>> I know you said that you don't think it's DNS lookups, but am I >correct >>>> in >>>> >>>>> thinking that if it is a problem with DNS lookups you would have >the >>>>>> same >>>>>> symptoms? the imudp thread would not be eating CPU as it's >blocked >>>>>> waiting >>>>>> for the response from the DNS server. >>>>>> >>>>>> >>>>>> yes, BUT: the DNS lookup is done on the main message queue >thread. >>>>> imudp >>>>> never does it. It just sets a flag that it hasn't done it. >Actually, the >>>>> resolution later in processing only occurs if a property is >accessed >>>>> that >>>>> needs it. >>>>> >>>>> Early version of rsyslog (v5 and below?) did do it on the imudp >thread, >>>>> but >>>>> that's long gone away. >>>>> >>>>> >>>>> Ok, good to know about the change. >>>> >>>> I wonder if there's a way to run more than one imudp thread on the >port. >>>> >>>>> the inbound packets would not be evenly split between them, but if >one >>>>>> is >>>>>> stalled, the other may be able to pull a packet. >>>>>> >>>>>> >>>>>> >>>>> That would require redesign and I am rather skeptic about it. The >reason >>>>> is >>>>> that imudp runs in a very thight loop that essentially does pull >>>>> messages >>>>> off the receive buffers and puts them into the queue. No real >waits >>>>> involved here. >>>>> >>>>> But of course multiple threads may have the result of getting more >/ or >>>>> quicker CPU power assigned. However, for that use case I think >realtime >>>>> priority is a much better solution. I would even expect that two >imudp >>>>> threads would contend each other for the main queue thread, as it >is a >>>>> really thight loop... BUT - it's my estimate, not tested reality. >To >>>>> test >>>>> it, I would probably need two to three days and I don't think they >were >>>>> wisely spent at this moment. >>>>> >>>>> >>>> Ok, I agree it's not worth the effort right now. >>>> >>>> off the top of your head, will rsyslog complain if you try to start >two >>>> inputs that use the same ip/port? >>>> >>>> I can probably also test this with some iptables trickery to split >the >>>> inbound traffic across two IPs, and bind an imudp thread to each >IP. >>>> >>>> But that's only worth trying after we get some more info about this >guy's >>>> performance. >>>> >>>> David Lang >>>> ______________________________****_________________ >>>> rsyslog mailing list >>>> >http://lists.adiscon.net/****mailman/listinfo/rsyslog<http://lists.adiscon.net/**mailman/listinfo/rsyslog> >>>> ><http:**//lists.adiscon.net/mailman/**listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>>> > >>>> >http://www.rsyslog.com/****professional-services/<http://www.rsyslog.com/**professional-services/> >>>> ><http://**www.rsyslog.com/professional-**services/<http://www.rsyslog.com/professional-services/> >>>> > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >you >>>> DON'T LIKE THAT. >>>> >>>> ______________________________**_________________ >>> rsyslog mailing list >>> >http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >>> >http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >you >>> DON'T LIKE THAT. >>> >>> ______________________________**_________________ >> rsyslog mailing list >> >http://lists.adiscon.net/**mailman/listinfo/rsyslog<http://lists.adiscon.net/mailman/listinfo/rsyslog> >> >http://www.rsyslog.com/**professional-services/<http://www.rsyslog.com/professional-services/> >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >you >> DON'T LIKE THAT. >> >_______________________________________________ >rsyslog mailing list >http://lists.adiscon.net/mailman/listinfo/rsyslog >http://www.rsyslog.com/professional-services/ >What's up with rsyslog? Follow https://twitter.com/rgerhards >NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST >if you DON'T LIKE THAT. -- Sent from my mobile device _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
