Andre If I understand the question correctly the ask is how can you extend the existing set of readers and writers. This is accomplished by implementing controller services. You can build them in Java or you can build them using the scripting language support for record readers/writers.
You'd want a reader understands how to deserialize the bytes of a CEF formatted file into record objects and a writer that understands how to take record objects and serialize them into CEF. Same for evtx. As you deserialize and parse the events you identify fields and values and create the record structure. Any of the processors that support the record concept have a reader and writer configured on them. If you wanted to end up with JSON records then you'd plug-in a json writer. You can get a sense of much of this from unit tests and I'd also be happy to put up a template that shows off how to configure a flow that reads from NiFi's own provenance stream, turns it into JSON, Avro, plaintext, and xml all at once. I'll try to do that tomorrow. That should make it a lot more clear. Thanks On Fri, May 5, 2017 at 10:51 PM, Andre <[email protected]> wrote: > All, > > I was doing some reading during my spare time and the record related > feature set is truly exciting. > > However I was wondering, how should one extend the existing range of > readers? > > I use for example ParseCEF and ParseEvtx. CEF while not particularly simple > to parse and validate using RegEx'es (and Grok) is a reasonably structured > format (with the same field supporting 9 different date formats...) > > ParseCEF currently exposes the parsed and validade CEF payload either to > attributes or JSON to content file. And hence my question: > > Is this the desired path of action: > > ParseCEF -> JSON content -> ConvertRecord (using JsonPathReader) -> whatever > > ? > > Or should extension make more sense? > > > Cheers
