hello, interesting thread, thanks.
one different question about compression : IIRC rsyslog provides two mechanisms for compression. Both of them would require that rsyslog is the client and the server. Ive just implemented a syslog server in python (using tornado async framework, code will be released this week). Supporting plain TCP and RELP was not difficult, as good specs are available. If i wanted to support rsyslog compression, are there any docs to start with? or just read the code? regards, stephane Le 10 mars 2015 18:42, "David Lang" <[email protected]> a écrit : > On Tue, 10 Mar 2015, Joe Blow wrote: > > I definitely need some more workers to send the logs, i'm getting 100% cpu >> on rsyslog threads and that's all this box is doing. >> > > not surprising, compression is rather CPU intensive > > What's the best way to add threads to an omfwd action? My action looks >> like this: >> > > The documentation page on queues unfortuantly doesn't show the new format > syntax, but since you've figured out the other parameters, take a look at > http://www.rsyslog.com/doc/v8-stable/concepts/queues.html# > worker-thread-pools and see if you can get it working from this. > > I think it's probably going to be queue.workerthreads="2" and/or queue. > workerthreadminimummessages="N" > > experiment with the minimum messages to set it as high as possible before > the cpu gets maxed out > > By the way, if you add name="something" in your action parameter, I > believe that the thread will show up with that name (at least if it's got a > queue) > > David Lang > > if $rawmsg contains "MyHeaderMsg|" then {action( >> Type="omfwd" >> Target="127.0.0.1" >> Port="1514" >> Protocol="tcp" >> Template="RAW" >> RebindInterval="250" >> compression.mode="stream:always" >> queue.dequeuebatchsize="100000" >> queue.type="fixedarray" >> queue.filename="output.rsq" >> queue.highwatermark="1200000000" >> queue.lowwatermark="150000000" >> queue.discardmark="1500000000" >> queue.maxdiskspace="100g" >> queue.size="1500000000" >> queue.saveonshutdown="on" >> action.resumeretrycount="-1")stop} >> >> Cheers, >> >> JB >> >> On Tue, Mar 10, 2015 at 11:58 AM, Rainer Gerhards < >> [email protected]> >> wrote: >> >> 2015-03-10 16:55 GMT+01:00 Rainer Gerhards <[email protected]>: >>> >>> 2015-03-10 16:53 GMT+01:00 David Lang <[email protected]>: >>>> >>>> On Tue, 10 Mar 2015, Rainer Gerhards wrote: >>>>> >>>>> >>>>> the version you compiled yourself, was it from the v8.8.0 tag or a >>>>>>> >>>>>> later >>> >>>> checkout from git that didn't have that tag changed yet? >>>>>>> >>>>>>> >>>>>> >>>>>> The .adX versions are Adiscon interim builds. When the project >>>>>> switched >>>>>> to >>>>>> the 6-weekly release cycle, Adiscon got some trouble because support >>>>>> customers often require new features or a bugfix in package form more >>>>>> timely. In this case, a special .adX version is released. They are >>>>>> very >>>>>> close to the "regular" version. There are not .adX tags in the public >>>>>> >>>>> git >>> >>>> repos because these versions are specially patched. >>>>>> >>>>>> I guess I didn't give that information before, sorry for that. >>>>>> >>>>>> >>>>> Since the .adX builds show up in the PPA repository from the nightly >>>>> builds, why not just make 8.8.1 instead of 8.8.0.ad1? >>>>> >>>> >>>> >>>> Well, we said we wanted a 6-week release cycle, except for emergencies. >>>> >>> If >>> >>>> I (the project) now release 8.8.1 just because Adiscon needs a new >>>> >>> package, >>> >>>> we are back to the old model where we release "when it is ready". >>>> >>>> This would also mean I would have much more work to do, because I would >>>> need to create a proper release, whereas otherwise I can just ask >>>> someone >>>> in Adiscon to build new packages. And that was really a good effect from >>>> the 6-week cycle for me... >>>> >>>> >>> >>> Oh, I need to add that nobody ever showed up when it came to help with >>> building packages. So the packages that currently exist are Adiscon >>> packages, not real project packages (in the sense that the project takes >>> care of them). If there is need to make that distinction clear, we can do >>> so. So far, I thought there is no big value in that. Adiscon doesn't want >>> to sponsor package development, but is happy with anyone (not only >>> support >>> customers) using what is created for the support customers. >>> >>> Rainer >>> _______________________________________________ >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
