[
https://issues.apache.org/jira/browse/HDFS-17978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18113055#comment-18113055
]
ASF GitHub Bot commented on HDFS-17978:
---------------------------------------
iamsanjaymalakar opened a new pull request, #8731:
URL: https://github.com/apache/hadoop/pull/8731
### Problem
`org.apache.hadoop.hdfs.server.datanode.DataNode.requestShortCircuitFdsForRead()`
opens the block data stream into `fis[0]` before opening the metadata stream
into `fis[1]`.
`org.apache.hadoop.hdfs.server.datanode.DatanodeUtil.getMetaDataInputStream()`
(and `checkStorageState` / `getBlockInputStream`) can throw
`java.io.IOException`. The catch clause handles only
`java.lang.ClassCastException`, so the `java.io.IOException` propagates with
`fis[0]` already open. The `fis` array is never returned on that path, so
`fis[0]` is leaked.
### Change
* Add a `java.io.IOException` catch that closes any stream already stored in
`fis` via `org.apache.hadoop.io.IOUtils.cleanupWithLogger` before rethrowing
the original exception.
* The successful path is unchanged.
### Testing
* `mvn -pl hadoop-hdfs-project/hadoop-hdfs -am
-Dtest=TestDataNodeShortCircuitFds test`
* New test
`TestDataNodeShortCircuitFds#testClosesBlockStreamWhenMetadataStreamFails`
fails against the previous code and passes with the change.
> Ensure requestShortCircuitFdsForRead() closes the block stream when
> metadata-stream setup fails
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-17978
> URL: https://issues.apache.org/jira/browse/HDFS-17978
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Affects Versions: 3.4.1
> Reporter: Sanjay Malakar
> Priority: Major
>
> {\{org.apache.hadoop.hdfs.server.datanode.DataNode.requestShortCircuitFdsForRead()}}
> opens the block data stream into \{{fis[0]}} before opening the metadata
> stream into \{{fis[1]}}. \{{DatanodeUtil.getMetaDataInputStream()}} (and
> \{{checkStorageState}} / \{{getBlockInputStream}}) can throw
> \{{{}java.io.IOException{}}}. The catch clause handles only
> \{{java.lang.ClassCastException}}, so the \{{java.io.IOException}} propagates
> with \{{fis[0]}} already open. The \{{fis}} array is never returned on that
> path, so \{{fis[0]}} is leaked.
> {\{requestShortCircuitFdsForRead()}} should close any stream already stored
> in \{{fis}} before propagating the \{{java.io.IOException}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
