Metron does not have a generic Syslog Parser. Nifi has Syslog parsing ( either Records or standard Processor ), in two modes.
ParseSyslog is the original, where regex’s are used to parse the syslog RFC3164 and RFC5424, but only extracts the common fields ( so the ‘additional info’ like program id, message id, structured data in 5424 is in the MSG ). I have recently added a record reader for that method as well ( Nifi PR#2900 <https://github.com/apache/nifi/pull/2900>). Syslog5424Reader(records) and ParseSyslog5424 are new and instead of using regexes they use a new library simple-syslog–5424 <https://github.com/palindromicity/simple-syslog-5424> I wrote that parses RFC5424 messages completely ( note properly formatted RFC 5424 messages ) see Nifi PR#2805 <https://github.com/apache/nifi/pull/2805> and Nifi PR#2816 <https://github.com/apache/nifi/pull/2816> using an antlr grammar. You should be able to pick the manner best for you and parse that out in Nifi if you choose. Metron parses syslog as required in specific parsers that have messages assumed to be embedded in syslog. What I have been talking about in METRON–1453 <https://issues.apache.org/jira/browse/METRON-1453> and other places is separating out the syslog from the parser, such that the parsers don’t need to know that the message is delivered embedded in syslog. The new parser chaining work would give us an avenue to this, and as you can see here MetronPR#1099 <https://github.com/apache/metron/pull/1099#issuecomment-405701948> I have put that case forward. If that hits, I think that we’d be able to : 1. parse plain syslog to metron 2. parse plain syslog as a transform and then have less complicated, more specific parsers for the msg part. We may end up having syslog parsers and transforms at the end of this. In the mean time, if you wish to parse plain syslog in Metron, you will have to use grok, which doesn’t get structured data. If you want the complete 5424 set of data, then you can open a jira for creating a parser using simple-syslog–5424. On July 20, 2018 at 04:23:36, Farrukh Naveed Anjum ([email protected]) wrote: Hi, I am trying to index the Syslog using CEF Parser with Nifi. It does not give any error though, transport data to kafa without indexing it. It keepg giving FAILED in Spout. I believe indexing Syslog are most basic usecase for all. But metron fails to do it with each in standard format. I tried bro for it. But even it keeps giving PARSER Error. Any help ? Fast will be apperciated. -- With Regards Farrukh Naveed Anjum
