[
https://issues.apache.org/jira/browse/HDFS-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingliang Liu updated HDFS-10383:
---------------------------------
Priority: Major (was: Minor)
The v1 patch revisited the {{DFSTestUtil}} class.
# {{urlGetBytes}} is still using the {{IOUtils.copyBytes(conn.getInputStream(),
out, 4096, true);}} as the {{URLConnection}} is not implementing
{{AutoCloseable}}. We need to close its input stream explicitly and the
{{IOUtils}} works just fine.
# {{waitCorruptReplicas}} swallows all the exception thrown by either
processing the resource or closing the resource. Ignored.
# {{runOperation}} runs a set of operations and generate all edit logs, and
thus explicit closing operation makes the edit log history clear for test and
debugging. Ignored.
Other than these, all other places have been switched to try-with-resource
statement.
> Safely close resources in DFSTestUtil
> -------------------------------------
>
> Key: HDFS-10383
> URL: https://issues.apache.org/jira/browse/HDFS-10383
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Reporter: Mingliang Liu
> Assignee: Mingliang Liu
> Attachments: HDFS-10383.000.patch
>
>
> There are a few of methods in {{DFSTestUtil}} that do not close the resource
> safely, or elegantly. We can use the try-with-resource statement to address
> this problem.
> Specially, as {{DFSTestUtil}} is popularly used in test, we need to preserve
> any exceptions thrown during the processing of the resource while still
> guaranteeing it's closed finally. Take for example,the current implementation
> of {{DFSTestUtil#createFile()}} closes the FSDataOutputStream in the
> {{finally}} block, and when closing if the internal
> {{DFSOutputStream#close()}} throws any exception, which it often does, the
> exception thrown during the processing will be lost. See this [test
> failure|https://builds.apache.org/job/PreCommit-HADOOP-Build/9320/testReport/org.apache.hadoop.hdfs/TestAsyncDFSRename/testAggressiveConcurrentAsyncRenameWithOverwrite/],
> and we have to guess what was the root cause.
> Using try-with-resource, we can close the resources safely, and the
> exceptions thrown both in processing and closing will be available (closing
> exception will be suppressed).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
