Re: [DISCUSS] Generic Syslog Parsing capability for parsers

2018-03-26 Thread Ali Nazemian
Just adding more details regarding what different parts are: There are three stages here that need to be understood: 1- pre-parsing 2- chain of parsing (wrapping one type of message in another format) 3- post-parsing aka normalization Pre-parsing stage is where we need to specify what specific

Re: [DISCUSS] Generic Syslog Parsing capability for parsers

2018-03-20 Thread zeo...@gmail.com
So I've kept my ear to the ground regarding this topic for a while now, and had some conversations a year or so ago about the idea as well. At the very least, I think having the concept of a pre-parser is a good one, if not chaining an arbitrary number of parsers together. I see this as an

Re: [DISCUSS] Generic Syslog Parsing capability for parsers

2018-03-20 Thread Otto Fowler
I think the chaining of parsers, or ability to compose parsers is a good idea, but with reference to the pr mentioned, I would have some number of StellarChainLinks as opposed re-implementing stellar in chainlinks. Although it is NiFi-y. But since I write Processors too, that is fine. On March

Re: [DISCUSS] Generic Syslog Parsing capability for parsers

2018-03-20 Thread Simon Elliston Ball
It seems like parser chaining is becomes a hot topic on the repo too with https://github.com/apache/metron/pull/969#partial-pull-merging I would like to discuss the option, and how we might architect, of configuring parsers to

[DISCUSS] Generic Syslog Parsing capability for parsers

2018-03-20 Thread Otto Fowler
I entered METRON–1453 a little while ago while working on the PR#579 . "We have several parsers now, with many imaginable that are based on syslog, where the format is SYSLOG HEADER MESSAGE. With