[
https://issues.apache.org/jira/browse/NIFI-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026768#comment-17026768
]
Joe Witt commented on NIFI-7073:
--------------------------------
Clearly line
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L356
is attempting to handle the case where the hdfs client tells us the remote
side cannot handle the given request. And it attempts to clean it and throws
an exception. The exception you're suggesting occurred (difficult to be sure
without the stack trace) means additional cases (not documented?) can do this
as well. Which suggests we should just change
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L356
to catch throwable and do that logic so that in *any* case where attempting to
close the stream results in an exception we assume the whole operation failed.
> PutHDFS silently ignores exceptions when closing temporary file output stream
> -----------------------------------------------------------------------------
>
> Key: NIFI-7073
> URL: https://issues.apache.org/jira/browse/NIFI-7073
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.11.0
> Reporter: Michael Hogue
> Priority: Major
>
> The PutHDFS processor can silently fail when closing an output stream when
> it's finished writing FlowFiles to a temporary space in HDFS.
> {{DFSOutputStream}} can throw an {{IOException}} when it's closed and that's
> ignored by the processor. This results in the FlowFile getting dropped (e.g.
> a loss of data) when it should be routed to the failure relationship.
> Source:
> [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
> Instead, the exception should be caught & thrown and the FlowFile should be
> routed to the failure relationship. There's probably an argument to avoid
> ignoring _any_ exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
