[
https://issues.apache.org/jira/browse/HDFS-7764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14325191#comment-14325191
]
Colin Patrick McCabe commented on HDFS-7764:
--------------------------------------------
I looked at this quickly, and it does look like the error handling is wrong
here. We shouldn't be aborting the whole directory scan because one
{{FileUtil#listFiles}} threw an exception. On the bright side, we do seem to
log the first problem we hit here:
{code}
try {
files = FileUtil.listFiles(dir);
} catch (IOException ioe) {
LOG.warn("Exception occured while compiling report: ", ioe);
// Ignore this directory and proceed.
return report;
}
{code}
If you want to improve this, then I would say:
* change it to use the jdk7 API that distinguishes between various different
types of I/O issues rather than just returning null. This is probably as
simple as using {{IOUtils#listDirectory}} instead of {{FileUtil.listFiles}}
* don't abort the scan on every directory just because one had an error. You
will need a unit test for this one.
> DirectoryScanner should cancel the future tasks when #compileReport throws
> exception
> ------------------------------------------------------------------------------------
>
> Key: HDFS-7764
> URL: https://issues.apache.org/jira/browse/HDFS-7764
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.7.0
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-7764.patch
>
>
> If there is an exception while preparing the ScanInfo for the blocks in the
> directory, DirectoryScanner is immediately throwing exception and coming out
> of the current scan cycle. It would be good if he can signal #cancel() to the
> other pending tasks .
> DirectoryScanner.java
> {code}
> for (Entry<Integer, Future<ScanInfoPerBlockPool>> report :
> compilersInProgress.entrySet()) {
> try {
> dirReports[report.getKey()] = report.getValue().get();
> } catch (Exception ex) {
> LOG.error("Error compiling report", ex);
> // Propagate ex to DataBlockScanner to deal with
> throw new RuntimeException(ex);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
