[
https://issues.apache.org/jira/browse/YARN-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925152#comment-15925152
]
Jason Lowe commented on YARN-6315:
----------------------------------
Thanks for the patch! Looks good overall, just some minor nits:
Extra semicolon on "import java.nio.file.Files;;"
The try/catch block covers more than necessary. Ideally it would not cover the
checkLocalResource call.
IOExceptions are treated like the resource is there. The prior exists call
that this is replacing would return false if an exception occurred.
I think it might be useful to emit at least an info message (if not warn)
indicating a resource we thought was there is no longer there, or if it's
corrupted what the size diffs are. That could help debugging cases where a
nodemanager keeps relocalizing when it shouldn't.
static Mockito.when import is not near the other static Mockito imports
> Improve LocalResourcesTrackerImpl#isResourcePresent to return false for
> corrupted files
> ---------------------------------------------------------------------------------------
>
> Key: YARN-6315
> URL: https://issues.apache.org/jira/browse/YARN-6315
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.3, 2.8.1
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Attachments: YARN-6315.001.patch, YARN-6315.002.patch
>
>
> We currently check if a resource is present by making sure that the file
> exists locally. There can be a case where the LocalizationTracker thinks that
> it has the resource if the file exists but with size 0 or less than the
> "expected" size of the LocalResource. This JIRA tracks the change to harden
> the isResourcePresent call to address that case.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
