All, Benjamin has already done a lot of good work and it would very helpful we can agree how to move on. https://issues.apache.org/jira/browse/NIFI-901
My first post was naive, there are much more things to consider. It is probably impossible to select only one “correct data exchange format” what all processors should use. But can we agree one or two preferred data format what SQL and NoSQL processors should support. And all other other formats are supported using converter processors. I my opinion preferred data exchange format should: 1. Support schema in the way or another. 2. Support streaming. 3. Support different data types (String, numeric types, date/time, binary) 4. Serialization/deserialization should be fast and efficient. 5. Widely used and has strong supporters. 6. Can be used in transformations, filtering, join, split, etc. 7. Can be converted to/and from other formats relatively easily. Nice to have: 1. Nested data structures. For example Orders can contain order rows. Or maybe we recommend all SQL and NoSQL processors should support two or more input/output formats and user can select format using configuration? Or separate sets of processors for different formats? Thanks Toivo -- View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Common-data-exchange-formats-and-tabular-data-tp3508p4337.html Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
