[
https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt reopened NIFI-12923:
-----------------------------
> PutHDFS to support appending avro data
> --------------------------------------
>
> Key: NIFI-12923
> URL: https://issues.apache.org/jira/browse/NIFI-12923
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Balázs Gerner
> Assignee: Balázs Gerner
> Priority: Major
> Fix For: 2.0.0-M3, 1.26.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The goal of this ticket is to extend the PutHDFS processor with the ability
> to append avro records. The processor already provides an option to set
> 'append' as conflict resolution strategy, but that does not work correctly in
> case of avro files, because the serialized avro file cannot be deserialized
> again (because the binary content is invalid).
> Some notes about the implementation:
> * The user needs to explicitly select avro as file format and append as
> conflict resolution mode to enable 'avro append' mode, otherwise regular
> append mode will work just as before. There is no auto detection of mimetype
> for the incoming flowfile.
> * The records of the incoming flowfile and the ones in the existing avro
> file need to conform to the same avro schema, otherwise the append operation
> fails with incompatible schema.
> * The 'avro append' mode should only work when compression type is set to
> 'none', if any other compression type is selected in 'avro append' mode the
> user should get a validation error.
> The changes will have to be added to *support/nifi-1.x* branch also.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
