[
https://issues.apache.org/jira/browse/NIFI-12923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17839086#comment-17839086
]
ASF subversion and git services commented on NIFI-12923:
--------------------------------------------------------
Commit 3239f59746f7f2edf9375b60682bdd8ad7a6a7e9 in nifi's branch
refs/heads/main from Balázs Gerner
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=3239f59746 ]
NIFI-12923 Added append avro mode to PutHDFS
NIFI-12923 remove var keyword
NIFI-12923 change property name
NIFI-12923 Added property dependency for append_mode
Signed-off-by: Matt Burgess <[email protected]>
This closes #8544
> PutHDFS to support appending avro data
> --------------------------------------
>
> Key: NIFI-12923
> URL: https://issues.apache.org/jira/browse/NIFI-12923
> Project: Apache NiFi
> Issue Type: Improvement
> Reporter: Balázs Gerner
> Assignee: Balázs Gerner
> Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The goal of this ticket is to extend the PutHDFS processor with the ability
> to append avro records. The processor already provides an option to set
> 'append' as conflict resolution strategy, but that does not work correctly in
> case of avro files, because the serialized avro file cannot be deserialized
> again (because the binary content is invalid).
> Some notes about the implementation:
> * The user needs to explicitly select avro as file format and append as
> conflict resolution mode to enable 'avro append' mode, otherwise regular
> append mode will work just as before. There is no auto detection of mimetype
> for the incoming flowfile.
> * The records of the incoming flowfile and the ones in the existing avro
> file need to conform to the same avro schema, otherwise the append operation
> fails with incompatible schema.
> * The 'avro append' mode should only work when compression type is set to
> 'none', if any other compression type is selected in 'avro append' mode the
> user should get a validation error.
> The changes will have to be added to *support/nifi-1.x* branch also.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
