[ https://issues.apache.org/jira/browse/YARN-3962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638158#comment-14638158 ]
Hadoop QA commented on YARN-3962: --------------------------------- \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | patch | 0m 0s | The patch command could not apply the patch during dryrun. | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12746718/Yarn-3962.001.patch | | Optional Tests | | | git revision | trunk / ee98d63 | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/8633/console | This message was automatically generated. > If we change node manager identity to run as virtual account, then resource > localization service fails to start with incorrect permission > ----------------------------------------------------------------------------------------------------------------------------------------- > > Key: YARN-3962 > URL: https://issues.apache.org/jira/browse/YARN-3962 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager > Affects Versions: 2.6.0 > Reporter: madhumita chakraborty > Attachments: Yarn-3962.001.patch > > > For azure hdinsight we need to change node manager to run as virtual account > instead of use account. Else after azure reimage, it wont be able to access > the map output data of the running job in that node. But when we changed the > nodemanager to run as virtual account we got this error, > 2015-06-02 06:11:45,281 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Writing credentials to the nmPrivate file > c:/apps1/temp/hdfs/nm-local-dir/nmPrivate/container_1433128260970_0007_01_000001.tokens. > Credentials list: > 2015-06-02 06:11:45,313 WARN > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Permissions incorrectly set for dir > c:/apps1/temp/hdfs/nm-local-dir/usercache, should be rwxr-xr-x, actual value > = rwxrwxr-x > 2015-06-02 06:11:45,313 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Attempting to initialize c:/apps1/temp/hdfs/nm-local-dir > 2015-06-02 06:11:45,375 WARN > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Permissions incorrectly set for dir > c:/apps1/temp/hdfs/nm-local-dir/usercache, should be rwxr-xr-x, actual value > = rwxrwxr-x > 2015-06-02 06:11:45,375 WARN > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Failed to setup local dir c:/apps1/temp/hdfs/nm-local-dir, which was marked > as good. > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Permissions > incorrectly set for dir c:/apps1/temp/hdfs/nm-local-dir/usercache, should be > rwxr-xr-x, actual value = rwxrwxr-x > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1400) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.getInitializedLocalDirs(ResourceLocalizationService.java:1367) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.access$900(ResourceLocalizationService.java:137) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1085) > 2015-06-02 06:11:45,375 INFO > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService: > Localizer failed > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to setup > local dir c:/apps1/temp/hdfs/nm-local-dir, which was marked as good. > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.getInitializedLocalDirs(ResourceLocalizationService.java:1372) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.access$900(ResourceLocalizationService.java:137) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:1085) > Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > Permissions incorrectly set for dir > c:/apps1/temp/hdfs/nm-local-dir/usercache, should be rwxr-xr-x, actual value > = rwxrwxr-x > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.checkLocalDir(ResourceLocalizationService.java:1400) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.getInitializedLocalDirs(ResourceLocalizationService.java:1367) > Fix - When node manager runs as virtual account, the resourcelocalization > service fails to come. It checks for the permission of usercache and file > cache to be 755 and nmPrivate to be 700. But in windows, for virtual account, > the owner and group is same. So this pemrission check fails. So added a check > that is user is equal to group, then umask validation dont hold -- This message was sent by Atlassian JIRA (v6.3.4#6332)