Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/3130
This is interesting stuff, I need to read more up on DFDL, might make a
good schema definition language as an alternative to Avro schemas. Also at
first glance it seems the Daffodil infoset is analogous to a NiFi Record for
data, and/or a NiFi RecordSchema for its schema. Having an infoset as the
underlying data model seems to overlap a bit with how the NiFi Record stuff
works, for example we could have a PCAPReader and any writer such as
JSONWriter, then the ConvertRecord processor can convert PCAP to JSON. I think
someone was working on a PCAP reader, not sure how they handled the global
stuff. If Avro schemas are not sufficient to capture this kind of model, then
we should definitely look at DFDL as an alternative (we've been considering XSD
as a more expressive alternative).
Quick note about JOLT, the "normal" use case is for JSON-to-JSON
conversion, but the underlying library does the transformation on a POJO, not
JSON. So the usual entry point for JOLT is to give it a JSON string, it
converts to a POJO, runs the transformation, and takes the resulting POJO and
writes it out as JSON. In NiFi I wrote JoltTransformRecord, which instead of
explicit JSON uses the NiFi Record API, so it gets each record (in whichever
format) as a NiFi Record, translates it to a POJO, then runs the JOLT
transforms, then translates back into a NiFi Record, which is written out in
whatever format is specified by the RecordWriter.
---
