[
https://issues.apache.org/jira/browse/NIFI-9260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438903#comment-17438903
]
ASF subversion and git services commented on NIFI-9260:
-------------------------------------------------------
Commit 9b724df6d94eb40be321cea07dfbabf02092de0f in nifi's branch
refs/heads/main from Bence Simon
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9b724df ]
NIFI-9260 Making the 'write and rename' behaviour optional for PutHDFS
This closes #5423.
Signed-off-by: Peter Turcsanyi <[email protected]>
> Making "write and rename" behaviour optional in PutHDFS
> -------------------------------------------------------
>
> Key: NIFI-9260
> URL: https://issues.apache.org/jira/browse/NIFI-9260
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Simon Bence
> Assignee: Simon Bence
> Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> In some cases the current "write and rename" behaviour of PutHDFS
> undesirable. Particularly if the the server is backed with AWS S3 which has
> no "native" rename but copies file content when renaming, serious perfomance
> issues might arise.
> In order to avoid this, I propose to make this behaviour optional and make it
> possible for the PutHDFS to write directly to the final destination. Of
> course, depending on the use case, there is the risk of reading
> "half-written" files in this manner, but in some scenarios this is not a
> danger. Also, in case of S3 this should not be an issue as files in S3 are
> visible only after the "write" operation is finished
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
