[
https://issues.apache.org/jira/browse/NIFI-7073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Hogue updated NIFI-7073:
--------------------------------
Description:
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Source:
[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship. There's probably an argument to avoid
ignoring _any_ exception.
was:
The PutHDFS processor can silently fail when closing an output stream when it's
finished writing FlowFiles to a temporary space in HDFS. {{DFSOutputStream}}
can throw an {{IOException}} when it's closed and that's ignored by the
processor. This results in the FlowFile getting dropped (e.g. a loss of data)
when it should be routed to the failure relationship.
Source:
[https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
Instead, the exception should be caught & thrown and the FlowFile should be
routed to the failure relationship.
> PutHDFS silently ignores exceptions when closing temporary file output stream
> -----------------------------------------------------------------------------
>
> Key: NIFI-7073
> URL: https://issues.apache.org/jira/browse/NIFI-7073
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.11.0
> Reporter: Michael Hogue
> Priority: Major
>
> The PutHDFS processor can silently fail when closing an output stream when
> it's finished writing FlowFiles to a temporary space in HDFS.
> {{DFSOutputStream}} can throw an {{IOException}} when it's closed and that's
> ignored by the processor. This results in the FlowFile getting dropped (e.g.
> a loss of data) when it should be routed to the failure relationship.
> Source:
> [https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-hadoop-bundle/nifi-hdfs-processors/src/main/java/org/apache/nifi/processors/hadoop/PutHDFS.java#L365]
> Instead, the exception should be caught & thrown and the FlowFile should be
> routed to the failure relationship. There's probably an argument to avoid
> ignoring _any_ exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
