Oziel, The ListenSyslog processor [1] will apply the regular expression and extract each of the fields into flow file attributes. From there you could use AttributesToJSON [2] with "Include Core Attributes" set to false, that should give you fields named "syslog.hostname" for example. You could use JoltTransformJSON [3] if you need to rename/reorganize the fields, or if you need more complex logic and are familiar with a scripting language such as Groovy, Javascript, Python, Ruby, or Lua, you could use ExecuteScript [4] to build a custom JSON output, I have examples of custom JSON transformations using Groovy [5], Javascript [6], and Jython [7].
If you are looking for the regular expression(s) to do the parsing yourself (perhaps with ExtractText [8]), you can find them here [9]. Regards, Matt [1] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ListenSyslog/index.html [2] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.AttributesToJSON/index.html [3] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.JoltTransformJSON/index.html [4] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.script.ExecuteScript/index.html [5] http://funnifi.blogspot.com/2016/02/executescript-json-to-json-conversion.html [6] http://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited.html [7] http://funnifi.blogspot.com/2016/03/executescript-json-to-json-revisited_14.html [8] https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.ExtractText/index.html [9] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/syslog/SyslogParser.java#L36 On Tue, Jan 17, 2017 at 5:28 PM, Hinojosa, Ozzie <[email protected]> wrote: > Hi, > > I am in the process of reading the Apache NiFi documentation and setting up a > single instance to obtain basic working knowledge of the platform. But in the > meantime I need an answer regarding Syslog messages and guidance on which > educational resources would be best to leverage in order to understand NiFi > in depth and in the shortest time possible. > > My questions regarding Syslog. How can I parse the FlowFile contents produce > by a ListenSyslog processor. By parse, I mean apply regular expressions and > transform the entire contents to JSON. I know extensibility is an integral > part of the platform, but again I am new to NiFi and I am looking to confirm > the feasibility of parsing Syslog messages; to add additional complexity > Netflows would have the same requirement as the Syslog messages. What is > available out of the box for Netflows and JFlows? > > Thanks, much appreciated. > > Oziel Hinojosa > Security Technologies Analyst > Strategy, Service Delivery & Programs (SSDP) > [email protected] > > CITC - IRSM > 1600 Smith St. HOU160-23042B > Houston, TX 77002 > > Tel +1 713 754 4749 > > Use http://csocbehavior.chevron.com/ to give me or others CSOC Behavioral > Feedback, my CAI is OZNH. >
