One more resource to add to Matt's list: https://blogs.apache.org/nifi/entry/storing_syslog_events_in_hbase
Thanks, Bryan On Tue, Jan 17, 2017 at 6:35 PM, Matt Burgess <[email protected]> wrote: > Oziel, > > The ListenSyslog processor [1] will apply the regular expression and > extract each of the fields into flow file attributes. From there you > could use AttributesToJSON [2] with "Include Core Attributes" set to > false, that should give you fields named "syslog.hostname" for > example. You could use JoltTransformJSON [3] if you need to > rename/reorganize the fields, or if you need more complex logic and > are familiar with a scripting language such as Groovy, Javascript, > Python, Ruby, or Lua, you could use ExecuteScript [4] to build a > custom JSON output, I have examples of custom JSON transformations > using Groovy [5], Javascript [6], and Jython [7]. > > If you are looking for the regular expression(s) to do the parsing > yourself (perhaps with ExtractText [8]), you can find them here [9]. > > Regards, > Matt > > [1] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.standard.ListenSyslog/index.html > [2] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.standard.AttributesToJSON/index.html > [3] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.standard.JoltTransformJSON/index.html > [4] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.script.ExecuteScript/index.html > [5] http://funnifi.blogspot.com/2016/02/executescript-json-to- > json-conversion.html > [6] http://funnifi.blogspot.com/2016/03/executescript-json-to- > json-revisited.html > [7] http://funnifi.blogspot.com/2016/03/executescript-json-to- > json-revisited_14.html > [8] https://nifi.apache.org/docs/nifi-docs/components/org. > apache.nifi.processors.standard.ExtractText/index.html > [9] https://github.com/apache/nifi/blob/master/nifi-nar- > bundles/nifi-standard-bundle/nifi-standard-processors/src/ > main/java/org/apache/nifi/processors/standard/syslog/SyslogParser.java#L36 > > On Tue, Jan 17, 2017 at 5:28 PM, Hinojosa, Ozzie > <[email protected]> wrote: > > Hi, > > > > I am in the process of reading the Apache NiFi documentation and setting > up a single instance to obtain basic working knowledge of the platform. But > in the meantime I need an answer regarding Syslog messages and guidance on > which educational resources would be best to leverage in order to > understand NiFi in depth and in the shortest time possible. > > > > My questions regarding Syslog. How can I parse the FlowFile contents > produce by a ListenSyslog processor. By parse, I mean apply regular > expressions and transform the entire contents to JSON. I know extensibility > is an integral part of the platform, but again I am new to NiFi and I am > looking to confirm the feasibility of parsing Syslog messages; to add > additional complexity Netflows would have the same requirement as the > Syslog messages. What is available out of the box for Netflows and JFlows? > > > > Thanks, much appreciated. > > > > Oziel Hinojosa > > Security Technologies Analyst > > Strategy, Service Delivery & Programs (SSDP) > > [email protected] > > > > CITC - IRSM > > 1600 Smith St. HOU160-23042B > > Houston, TX 77002 > > > > Tel +1 713 754 4749 > > > > Use http://csocbehavior.chevron.com/ to give me or others CSOC > Behavioral Feedback, my CAI is OZNH. > > >
