Joe, My question wasn't very clear but I think you still managed to provide the insight I needed!
I will give it a go creating a reader to see if I get a better view on how those work. Cheers On Sat, May 6, 2017 at 1:22 PM, Joe Witt <[email protected]> wrote: > Andre > > If I understand the question correctly the ask is how can you extend > the existing set of readers and writers. This is accomplished by > implementing controller services. You can build them in Java or you > can build them using the scripting language support for record > readers/writers. > > You'd want a reader understands how to deserialize the bytes of a CEF > formatted file into record objects and a writer that understands how > to take record objects and serialize them into CEF. Same for evtx. > As you deserialize and parse the events you identify fields and values > and create the record structure. > > Any of the processors that support the record concept have a reader > and writer configured on them. If you wanted to end up with JSON > records then you'd plug-in a json writer. You can get a sense of much > of this from unit tests and I'd also be happy to put up a template > that shows off how to configure a flow that reads from NiFi's own > provenance stream, turns it into JSON, Avro, plaintext, and xml all at > once. I'll try to do that tomorrow. That should make it a lot more > clear. > > Thanks > > On Fri, May 5, 2017 at 10:51 PM, Andre <[email protected]> wrote: > > All, > > > > I was doing some reading during my spare time and the record related > > feature set is truly exciting. > > > > However I was wondering, how should one extend the existing range of > > readers? > > > > I use for example ParseCEF and ParseEvtx. CEF while not particularly > simple > > to parse and validate using RegEx'es (and Grok) is a reasonably > structured > > format (with the same field supporting 9 different date formats...) > > > > ParseCEF currently exposes the parsed and validade CEF payload either to > > attributes or JSON to content file. And hence my question: > > > > Is this the desired path of action: > > > > ParseCEF -> JSON content -> ConvertRecord (using JsonPathReader) -> > whatever > > > > ? > > > > Or should extension make more sense? > > > > > > Cheers >
