On Fri, Jan 31, 2014 at 8:14 AM, Nathan Brown <[email protected]> wrote:
> What if a plugin wants to reliably handle events? For example: I am > currently using a relp protocol implementation to ensure I handle > every event reliably and this plugin architecture sounds like an > intriguing way to get rid of the tcp overhead and relp window sizing. > I would want to be able to replicate the same offerings as relp in > this interface though. > > I can also imagine more exotic scenarios where a plugin is used as > part of a ruleset to decide which branch to take. Like selecting which > aggregator node to send an event to or whether to forward an event in > general. > > All of these scenarios require a more structured protocol, and the > right answer might be that this architecture isn't for that. > > Well, you could probably build this on top of the interface. But it's far beyond the current "just make it simple to write a connector" approach. This however requires the feedback verbs, which are not yet defined. Current focus is on enable people to write simple connectors. Quite honestly, I think the target base is people who can write automation scripts, but are no real developers. And what you describe would be far above their head. Digging a bit deeper, I wonder what you want to save over RELP? If you require it's feature, I think it is very hard to come up with some protocol that does not look very similar to RELP... Rainer > > On Jan 30, 2014, at 10:47 PM, Rainer Gerhards <[email protected]> > wrote: > > > > On Fri, Jan 31, 2014 at 7:37 AM, Rainer Gerhards > > <[email protected]>wrote: > > > >>> On Fri, Jan 31, 2014 at 6:50 AM, Nathan Brown <[email protected]> > wrote: > >>> > >>> Fully agree that some plugins will be brain dead simple and will only > >>> require an incredibly small feature set of what the plugin system may > >>> offer. But having multiple communications options, certainly ones that > >>> are entirely unstructured, will be difficult to maintain. Forward > > I forgot to answer this: it's not difficult to maintain, as we simply use > > the template system that's already in place, and needs to be in place for > > all further versions in any case. So there is exactly 0% overhead for > this. > > > > Rainer > > > > > >> compatibility of a plugin that may start simple but eventually will > >>> want to grow into something more complex is a good enough reason to > >>> have a structured plugin protocol. Let alone rsyslog's side that may > >>> increase the feature set that it can offer a plugin. JSON has proven > >>> it's a good candidate for this sort of thing, lightweight with decent > >>> native types. > >> IMO rsyslog is still about speed, even though we sacrifice it for the > sake > >> of making it easy to write plugins. I agree that JSON is the most > versatile > >> format and something to recommend for the general case. HOWEVER, there > is > >> quite a bit of overhead associated with crafting an all-JSON > representation > >> of a message object. > >> > >> So I think we should not FORCE plugins to use JSON format. There are > many > >> cases where this makes sense, but there are also many cases where simply > >> -and much faster!- formats make more sense. Given the simplicity of the > >> interface, I think anyone should understand what he is doing. Even more > so, > >> for real novices, getting the JSON parser to work might be a stumbling > >> stone, whereas pulling pre-formatted data from stdin makes them happy. > >> > >> This is why I think we should not enforce the use of JSON on the > interface. > >> > >> Rainer > >> > >>>> On Jan 30, 2014, at 9:11 PM, David Lang <[email protected]> wrote: > >>>> > >>>> The thing is that with the rsyslog templates, you can do a lot of > >>> formatting and manipulation of the data before you send it to the > module. > >>> so you should configure rsyslog to send your module whatever it can > handle > >>> the best. > >>>> > >>>> If you have typical needs (take this entire line and send it there) > you > >>> can get by with a very simple output module. If you have very complex > >>> needs, you may need to use JSON to structure things so that it's always > >>> clear what's what. > >>>> > >>>> the author of the output module needs to document what that module > >>> requires as it's input. > >>>> > >>>> overall this looks pretty good. > >>>> > >>>> David Lang > >>>> > >>>>> On Wed, 29 Jan 2014, Nathan Brown wrote: > >>>>> > >>>>> I would avoid having multiple communication options and use only line > >>>>> delimited JSON to talk between the plugin and rsyslog. > >>>>> > >>>>> > >>>>> On Wed, Jan 29, 2014 at 3:21 AM, Rainer Gerhards > >>>>> <[email protected]>wrote: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> I've written a short presentation on writing rsyslog output plugins. > >>> It is > >>>>>> targeted towards people who would like to know at least a bit about > >>> how > >>>>>> this works. I also plan to do an ultra-fasttrack document for those > >>> that > >>>>>> just want to get it going somehow and don't about how it works. They > >>> are > >>>>>> *not* the focus of the current presentation: > >>>>>> > >>>>>> http://www.slideshare.net/rainergerhards1/external-plugins > >>>>>> > >>>>>> I will include this information in the Fedora DevConf Talk next > week, > >>> and > >>>>>> would really like your feedback. Where can we improve it, how to > make > >>> even > >>>>>> more clear that it's simple. Note that I say "v8" only because it > >>> already > >>>>>> covers a bit of what we will have in a month or so (the stdout > >>> feedback). > >>>>>> When I give the presentation, I'll tell that omprog in older > versions > >>> works > >>>>>> mostly the same. > >>>>>> > >>>>>> Thanks, > >>>>>> Rainer > >>>>>> _______________________________________________ > >>>>>> rsyslog mailing list > >>>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>>> http://www.rsyslog.com/professional-services/ > >>>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>> myriad > >>>>>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if > you > >>>>>> DON'T LIKE THAT. > >>>>> _______________________________________________ > >>>>> rsyslog mailing list > >>>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>>> http://www.rsyslog.com/professional-services/ > >>>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if > >>> you DON'T LIKE THAT. > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com/professional-services/ > >>>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > >>> myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if > >>> you DON'T LIKE THAT. > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com/professional-services/ > >>> What's up with rsyslog? Follow https://twitter.com/rgerhards > >>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad > >>> of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > >>> DON'T LIKE THAT. > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
