[
https://issues.apache.org/jira/browse/NIFI-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272307#comment-14272307
]
Ryan Blue commented on NIFI-238:
--------------------------------
bq. Would it be feasible to simply transform the FlowFile content from what it
was to some Kite formatted/output?
Not really. Kite is more than a file format, it's more like a file layout
format instead. A dataset managed by Kite will typically be partitioned by some
function of the data (like timestamp => year/month/day) and Kite transparently
takes care of managing the data. A dataset could also be backed by HBase
instead of files.
If we want to make this more modular, which is a good call, then I think the
right way is to split out the conversion and storage into two Kite processors.
One that writes to a dataset but accepts only Avro files (which is Kite's
object model) and one that converts CSV to Avro given some schema. I'll like
that idea so I'll make those changes unless you think that's misguided.
> Add processors to write datasets using Kite
> -------------------------------------------
>
> Key: NIFI-238
> URL: https://issues.apache.org/jira/browse/NIFI-238
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Ryan Blue
>
> I think it would be great to have a set of processors that parse incoming
> flow files and add the data to Kite datasets.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
