[
https://issues.apache.org/jira/browse/YARN-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928292#comment-15928292
]
Jason Lowe commented on YARN-6315:
----------------------------------
Thanks for updating the patch!
Catching Exception is too wide of a net here, IMHO. It masks serious issues
like SecurityException (which is not a normal I/O permission denied type of
error), NullPointerException, IllegalArgumentException,
UnsupportedOperationException, etc. If the operation really is unsupported
then this is going to think every resource is missing after it localizes it
which isn't good. It would be a dist cache that doesn't cache. We should just
catch NoSuchFileException and IOException. In the no such file case we can
simply log it isn't there, but in the IOException case since we don't really
know what happened we should log the full exception trace rather than just the
exception message to give proper context for debug.
Nit: The attributes variable declaration should be as close to the usage as
necessary. It only needs to be just before the the {{try}} block rather than
outside the {{if}} block.
> Improve LocalResourcesTrackerImpl#isResourcePresent to return false for
> corrupted files
> ---------------------------------------------------------------------------------------
>
> Key: YARN-6315
> URL: https://issues.apache.org/jira/browse/YARN-6315
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.3, 2.8.1
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: YARN-6315.001.patch, YARN-6315.002.patch,
> YARN-6315.003.patch
>
>
> We currently check if a resource is present by making sure that the file
> exists locally. There can be a case where the LocalizationTracker thinks that
> it has the resource if the file exists but with size 0 or less than the
> "expected" size of the LocalResource. This JIRA tracks the change to harden
> the isResourcePresent call to address that case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
