On Fri, 31 Jan 2014, Rainer Gerhards wrote:
On Fri, Jan 31, 2014 at 8:14 AM, Nathan Brown <[email protected]> wrote:
What if a plugin wants to reliably handle events? For example: I am
currently using a relp protocol implementation to ensure I handle
every event reliably and this plugin architecture sounds like an
intriguing way to get rid of the tcp overhead and relp window sizing.
I would want to be able to replicate the same offerings as relp in
this interface though.
I can also imagine more exotic scenarios where a plugin is used as
part of a ruleset to decide which branch to take. Like selecting which
aggregator node to send an event to or whether to forward an event in
general.
All of these scenarios require a more structured protocol, and the
right answer might be that this architecture isn't for that.
Well, you could probably build this on top of the interface. But it's far
beyond the current "just make it simple to write a connector" approach.
This however requires the feedback verbs, which are not yet defined.
Current focus is on enable people to write simple connectors. Quite
honestly, I think the target base is people who can write automation
scripts, but are no real developers. And what you describe would be far
above their head.
Digging a bit deeper, I wonder what you want to save over RELP? If you
require it's feature, I think it is very hard to come up with some protocol
that does not look very similar to RELP...
I think that what he's looking for is very similar to RELP, but not over the
network (which would make it a bit simpler).
I think for a local connection, we can get away with a synchronous connection as
long as we support the rsyslog internal batching to the external module. but
this really is looking for the protocol buffers/thrift type interface that I was
talking about doing for the second version of this.
David Lang
Rainer
On Jan 30, 2014, at 10:47 PM, Rainer Gerhards <[email protected]>
wrote:
On Fri, Jan 31, 2014 at 7:37 AM, Rainer Gerhards
<[email protected]>wrote:
On Fri, Jan 31, 2014 at 6:50 AM, Nathan Brown <[email protected]>
wrote:
Fully agree that some plugins will be brain dead simple and will only
require an incredibly small feature set of what the plugin system may
offer. But having multiple communications options, certainly ones that
are entirely unstructured, will be difficult to maintain. Forward
I forgot to answer this: it's not difficult to maintain, as we simply use
the template system that's already in place, and needs to be in place for
all further versions in any case. So there is exactly 0% overhead for
this.
Rainer
compatibility of a plugin that may start simple but eventually will
want to grow into something more complex is a good enough reason to
have a structured plugin protocol. Let alone rsyslog's side that may
increase the feature set that it can offer a plugin. JSON has proven
it's a good candidate for this sort of thing, lightweight with decent
native types.
IMO rsyslog is still about speed, even though we sacrifice it for the
sake
of making it easy to write plugins. I agree that JSON is the most
versatile
format and something to recommend for the general case. HOWEVER, there
is
quite a bit of overhead associated with crafting an all-JSON
representation
of a message object.
So I think we should not FORCE plugins to use JSON format. There are
many
cases where this makes sense, but there are also many cases where simply
-and much faster!- formats make more sense. Given the simplicity of the
interface, I think anyone should understand what he is doing. Even more
so,
for real novices, getting the JSON parser to work might be a stumbling
stone, whereas pulling pre-formatted data from stdin makes them happy.
This is why I think we should not enforce the use of JSON on the
interface.
Rainer
On Jan 30, 2014, at 9:11 PM, David Lang <[email protected]> wrote:
The thing is that with the rsyslog templates, you can do a lot of
formatting and manipulation of the data before you send it to the
module.
so you should configure rsyslog to send your module whatever it can
handle
the best.
If you have typical needs (take this entire line and send it there)
you
can get by with a very simple output module. If you have very complex
needs, you may need to use JSON to structure things so that it's always
clear what's what.
the author of the output module needs to document what that module
requires as it's input.
overall this looks pretty good.
David Lang
On Wed, 29 Jan 2014, Nathan Brown wrote:
I would avoid having multiple communication options and use only line
delimited JSON to talk between the plugin and rsyslog.
On Wed, Jan 29, 2014 at 3:21 AM, Rainer Gerhards
<[email protected]>wrote:
Hi all,
I've written a short presentation on writing rsyslog output plugins.
It is
targeted towards people who would like to know at least a bit about
how
this works. I also plan to do an ultra-fasttrack document for those
that
just want to get it going somehow and don't about how it works. They
are
*not* the focus of the current presentation:
http://www.slideshare.net/rainergerhards1/external-plugins
I will include this information in the Fedora DevConf Talk next
week,
and
would really like your feedback. Where can we improve it, how to
make
even
more clear that it's simple. Note that I say "v8" only because it
already
covers a bit of what we will have in a month or so (the stdout
feedback).
When I give the presentation, I'll tell that omprog in older
versions
works
mostly the same.
Thanks,
Rainer
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if
you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST
if
you DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a
myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad
of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you
DON'T LIKE THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of
sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE
THAT.
