[
https://issues.apache.org/jira/browse/NIFI-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024938#comment-16024938
]
Bryan Bende commented on NIFI-3759:
-----------------------------------
Just adding to Joe's comment...
As part of the Parquet work, I created AbstractPutHDFSRecord which should have
80-90% of the logic:
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-hadoop-record-utils/src/main/java/org/apache/nifi/processors/hadoop/AbstractPutHDFSRecord.java
Then implementations just need to provide an HDFSWriter by implementing this
method:
{code}
public abstract HDFSRecordWriter createHDFSRecordWriter(
final ProcessContext context, final FlowFile flowFile, final
Configuration conf, final Path path, final RecordSchema schema) throws
IOException, SchemaNotFoundException;
{code}
We could go the route of having PutHDFSAvro that extends AbstractPutHDFSRecord
and provides an HDFSRecordWriter that knows how to deal with writing Avro.
We could also try to have a generic PutHDFSRecord with a strategy for choosing
the destination format and then it would create the appropriate HDFSWriter
based on the format.
> Enable Avro append for Put HDFS
> -------------------------------
>
> Key: NIFI-3759
> URL: https://issues.apache.org/jira/browse/NIFI-3759
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
> Affects Versions: 1.1.0, 1.1.1
> Reporter: Jonas Hartwig
> Priority: Minor
>
> It would be nice where Nifi already supports working with AVRO to enable hdfs
> append working correctly with avro files coming from MergeContent processor.
> I suggest making a similar choice in as in MergeContent to be able to choose
> a file format which applies file format specific logic when required.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
