[
https://issues.apache.org/jira/browse/YARN-3120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128253#comment-15128253
]
kumar ranganathan commented on YARN-3120:
-----------------------------------------
Hi,
I have facing the same problem in Hadoop-2.7.2 in Windows. I have found the
logs in nodemanager as below and it should be the actual root cause
{panel}
WARN localizer.ResourceLocalizationService: Failed to setup local dir
/tmp/hadoop-SYSTEM/nm-local-dir, which was marked as good.
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions incorrectly
set for dir /tmp/hadoop-SYSTEM/nm-local-dir/usercache, should be rwxr-xr-x,
actual value = rwxrwxr-x
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1443)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.getInitializedLocalDirs(ResourceLocalizationService.java:1410)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.access$800(ResourceLocalizationService.java:140)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1110)
16/02/02 18:50:45 INFO localizer.ResourceLocalizationService: Localizer failed
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local
dir /tmp/hadoop-SYSTEM/nm-local-dir, which was marked as good.
{panel}
I found another jira task related to this issue
[https://issues.apache.org/jira/browse/YARN-3962]
Even though i applied patch, the problem isn't resolved. So i have changed the
code manually by commenting the condition check as below,
{code:title=ResourceLocalizationService.java|borderStyle=solid}
private boolean checkLocalDir(String localDir) {
Map<Path, FsPermission> pathPermissionMap =
getLocalDirsPathPermissionsMap(localDir);
for (Map.Entry<Path, FsPermission> entry : pathPermissionMap.entrySet()) {
FileStatus status;
try {
status = lfs.getFileStatus(entry.getKey());
} catch (Exception e) {
String msg =
"Could not carry out resource dir checks for " + localDir
+ ", which was marked as good";
LOG.warn(msg, e);
throw new YarnRuntimeException(msg, e);
}
// if (!status.getPermission().equals(entry.getValue())) {
// String msg =
// "Permissions incorrectly set for dir " + entry.getKey()
// + ", should be " + entry.getValue() + ", actual value = "
// + status.getPermission();
// LOG.warn(msg);
// throw new YarnRuntimeException(msg);
// }
}
return true;
}
{code}
Now it works fine without any issue. It seems we can ignore this condition for
virtual account in windows.
> YarnException on windows +
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local
> dirnm-local-dir, which was marked as good.
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-3120
> URL: https://issues.apache.org/jira/browse/YARN-3120
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.6.0
> Environment: Windows 8 , Hadoop 2.6.0
> Reporter: vaidhyanathan
>
> Hi,
> I tried to follow the instructiosn in
> http://wiki.apache.org/hadoop/Hadoop2OnWindows and have setup
> hadoop-2.6.0.jar in my windows system.
> I was able to start everything properly but when i try to run the job
> wordcount as given in the above URL , the job fails with the below exception .
> 15/01/30 12:56:09 INFO localizer.ResourceLocalizationService: Localizer failed
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup local
> di
> r /tmp/hadoop-haremangala/nm-local-dir, which was marked as good.
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
> ResourceLocalizationService.getInitializedLocalDirs(ResourceLocalizationService.
> java:1372)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
> ResourceLocalizationService.access$900(ResourceLocalizationService.java:137)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.
> ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java
> :1085)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
