[
https://issues.apache.org/jira/browse/NIFI-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024918#comment-16024918
]
Joseph Witt commented on NIFI-3759:
-----------------------------------
Understood. I think what needs to happen to go this route then is a processor
which is designed to do datatype specific puts to HDFS with optional support
for appending to an existing file. PutHDFS enjoys being completely about
writing bytes and is not concerned with the file type/format/schema and we
should keep it that way so as not to create new class/version issues for this
extremely heavily used/general purpose processor.
I'd recommend building a PutHDFSRecord processor which takes the record
reader/record writer interface. It will need to be able to read from the
exising file in HDFS to assess/understand the schema/formats at play so it can
enforce consistency. The foundations to do all this have been laid in the
recent 1.2.0 release and with the work already on master.
> Enable Avro append for Put HDFS
> -------------------------------
>
> Key: NIFI-3759
> URL: https://issues.apache.org/jira/browse/NIFI-3759
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.1.0, 1.1.1
> Reporter: Jonas Hartwig
> Priority: Minor
>
> It would be nice where Nifi already supports working with AVRO to enable hdfs
> append working correctly with avro files coming from MergeContent processor.
> I suggest making a similar choice in as in MergeContent to be able to choose
> a file format which applies file format specific logic when required.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
