David , can you tell me the difference bet omprog and mmexternal. both can be used to call the external plugin? my purpose is two feed raw log/msg to some external plugin and do the parsing? which one will do the purpose for me?
On Tue, Jul 22, 2014 at 6:22 AM, David Lang <[email protected]> wrote: > In any language, there is a huge variation in the performance of things > like this, depending on implementation details, you will have to try it and > see. > > > David Lang > > On Mon, 21 Jul 2014, Zeshan wrote: > > we were using ruby files in case of logstash for normalization , now we >> want to reuse them in rsyslog , by calling them through omprog and writing >> the data to mongoDB. >> >> >> On Mon, Jul 21, 2014 at 4:35 PM, David Lang <[email protected]> wrote: >> >> It probably depends mostly on what code you are using for your >>> normalization. >>> >>> David Lang >>> >>> >>> On Mon, 21 Jul 2014, Rainer Gerhards wrote: >>> >>> On Mon, Jul 21, 2014 at 11:43 AM, masoom alam <[email protected]> >>> >>>> wrote: >>>> >>>> David: >>>> >>>>> >>>>> How the performance will affected in the case where we want to pass the >>>>> control to do some normalization - name,value pair conversion by >>>>> Rsyslog. >>>>> We want to get rid of Logstash. >>>>> >>>>> >>>>> depending on what you do it's "not too bad", but of course it depends >>>>> on >>>>> >>>> the circumstances. However, you do not want to use output modules but >>>> you >>>> want to use the message modification internface, implemented via >>>> mmexternal. See these links: >>>> >>>> https://github.com/rsyslog/rsyslog/blob/master/plugins/ >>>> external/INTERFACE.md >>>> >>>> http://www.rsyslog.com/doc/master/configuration/modules/mmexternal.html >>>> >>>> HTH >>>> Rainer >>>> >>>> >>>> Thanks >>>> >>>>> >>>>> >>>>> On Mon, Jul 21, 2014 at 2:20 PM, David Lang <[email protected]> wrote: >>>>> >>>>> On Mon, 21 Jul 2014, Zeshan wrote: >>>>> >>>>>> >>>>>> Thanks David, >>>>>> >>>>>> >>>>>>> It worked now with the following lines in my rsyslog.conf >>>>>>> >>>>>>> $ModLoad omprog >>>>>>> $actionomprogbinary /var/log/test.sh >>>>>>> *.* :omprog:;RSYSLOG_TraditionalFileFormat >>>>>>> >>>>>>> >>>>>>> that's the legacy format, the new format I gave below does the same >>>>>> >>>>>> thing, >>>>> >>>>> but is clearer in many ways. It allos allows you to give parameters to >>>>>> >>>>>> your >>>>> >>>>> program (the $actionomprogbinary value can't have any spaces in it) >>>>>> >>>>>> now that you have this working, you can change the format of data >>>>>> passed >>>>>> to your script by using a different template, and change ot program >>>>>> >>>>>> itself >>>>> >>>>> to any other language. >>>>>> >>>>>> David Lang >>>>>> >>>>>> >>>>>> I was doing exactly the same except that the script was in >>>>>> /opt/test.sh >>>>>> >>>>>> and >>>>>>> now it is in /var/log/test.sh >>>>>>> >>>>>>> Thanks for all the kindness >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 21, 2014 at 1:42 PM, David Lang <[email protected]> wrote: >>>>>>> >>>>>>> Ok, looking at the documentation >>>>>>> >>>>>>> http://www.rsyslog.com/doc/omprog.html >>>>>> >>>>> >>>>> >>>>>> try >>>>>>>> >>>>>>>> Module (load="omprog") >>>>>>>> action(type="omprog" binary="/pathto/test.sh" template="RSYSLOG_ >>>>>>>> TraditionalFileFormat") >>>>>>>> >>>>>>>> this will send the script every log entry that arrives on the >>>>>>>> rsyslog >>>>>>>> server >>>>>>>> >>>>>>>> if you run this with the -dn flags, we should see test.sh and the >>>>>>>> prog >>>>>>>> output module show up with the first log entry that it processes. >>>>>>>> >>>>>>>> David Lang >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, 21 Jul 2014, Zeshan wrote: >>>>>>>> >>>>>>>> Thanks David for your reply >>>>>>>> >>>>>>>> >>>>>>>> ok , i have script named test.sh having following code >>>>>>>>> >>>>>>>>> #!/bin/bash >>>>>>>>> echo "$@" >/var/log/myoutput >>>>>>>>> >>>>>>>>> and it is is executable. >>>>>>>>> >>>>>>>>> Now what should me rsyslog.conf file. and how to test this whole >>>>>>>>> procedure >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Jul 21, 2014 at 1:09 PM, Muhammad Asif Ihsan < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> David, now I have its binary and what should I code in >>>>>>>>> rsyslog.conf >>>>>>>>> >>>>>>>>>> for >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> loading and redirecting to my executable myfile. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> rsyslog mailing list >>>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>>> myriad >>>>>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>>>>> >>>>>>>>>> you >>>>>>>>> >>>>>>>> >>>>> DON'T LIKE THAT. >>>>>> >>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> >>>>>>>>>> rsyslog mailing list >>>>>>>>>> >>>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>>> >>>>>>>>> myriad >>>>>>>> >>>>>>> >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> >>>>>>> DON'T LIKE THAT. >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> >>>>>>>>> rsyslog mailing list >>>>>>>>> >>>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>>> >>>>>>>> myriad >>>>>>> >>>>>> >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> >>>>>>> DON'T LIKE THAT. >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> >>>>>>>> rsyslog mailing list >>>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>>> http://www.rsyslog.com/professional-services/ >>>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>>> myriad >>>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if >>>>>>> you >>>>>>> DON'T LIKE THAT. >>>>>>> >>>>>>> _______________________________________________ >>>>>>> >>>>>>> rsyslog mailing list >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>>> http://www.rsyslog.com/professional-services/ >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>>> myriad >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>>> DON'T LIKE THAT. >>>>>> >>>>>> _______________________________________________ >>>>>> >>>>> rsyslog mailing list >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>>> http://www.rsyslog.com/professional-services/ >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a >>>>> myriad >>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>>> DON'T LIKE THAT. >>>>> >>>>> _______________________________________________ >>>>> >>>> rsyslog mailing list >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>>> http://www.rsyslog.com/professional-services/ >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>>> DON'T LIKE THAT. >>>> >>>> _______________________________________________ >>>> >>> rsyslog mailing list >>> http://lists.adiscon.net/mailman/listinfo/rsyslog >>> http://www.rsyslog.com/professional-services/ >>> What's up with rsyslog? Follow https://twitter.com/rgerhards >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >>> DON'T LIKE THAT. >>> >>> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad >> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you >> DON'T LIKE THAT. >> >> _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
