[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268581#comment-13268581 ] xieguiming commented on HADOOP-8351: 1, NN had been implemented by other before. 2, I have provide 2 new testcasese. and modify 2 testcase already. 3,please review, thanks. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 0.23.1 Environment: suse Reporter: xieguiming Attachments: HADOOP-835-0.patch, HADOOP-835-1.patch, HADOOP-835.patch yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268588#comment-13268588 ] Hadoop QA commented on HADOOP-8351: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525654/HADOOP-835-1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/940//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/940//console This message is automatically generated. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 0.23.1 Environment: suse Reporter: xieguiming Attachments: HADOOP-835-0.patch, HADOOP-835-1.patch, HADOOP-835.patch yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267493#comment-13267493 ] Harsh J commented on HADOOP-8351: - He probably meant adding a new file for include/exclude at runtime (cause it was possibly forgotten before by the operator). I think thats a moderate request and if our current refresh-able methods allow for such a thing, we can grant it (though it does seem dangerous if one were to configure badly (bad perm, etc.) and cause NN to throw up? Tests needed!). Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Environment: suse Reporter: xieguiming yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267497#comment-13267497 ] Robert Joseph Evans commented on HADOOP-8351: - Oh I see that now, thanks Harsh for that explanation. My bad. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Environment: suse Reporter: xieguiming yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267512#comment-13267512 ] Harsh J commented on HADOOP-8351: - Np Robert! Was a little confusing indeed. @xieguiming - Are you willing to contribute a patch against trunk for this feature/improvement? We'd be happy to review it. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Environment: suse Reporter: xieguiming yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267989#comment-13267989 ] xieguiming commented on HADOOP-8351: OK, I will provide the patch. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Environment: suse Reporter: xieguiming yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268071#comment-13268071 ] xieguiming commented on HADOOP-8351: updateFileNames function already exists in the HostsFileReader. and HDFS has called this function. So, we only need to call updateFileNames in MR. and I provide two testcases. please review, thanks. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 0.23.1 Environment: suse Reporter: xieguiming Attachments: HADOOP-835-0.patch, HADOOP-835.patch yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268094#comment-13268094 ] Hadoop QA commented on HADOOP-8351: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525559/HADOOP-835-0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javadoc. The javadoc tool appears to have generated 2 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/937//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/937//console This message is automatically generated. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 0.23.1 Environment: suse Reporter: xieguiming Attachments: HADOOP-835-0.patch, HADOOP-835.patch yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8351) Add exclude/include file , need restart NN or RM.
[ https://issues.apache.org/jira/browse/HADOOP-8351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268120#comment-13268120 ] Devaraj K commented on HADOOP-8351: --- Thanks Xieguiming for the patch. There are few of my observations about the patch. * Changes have done in mapreduce code, can we move this jira to mapreduce project. As description says it is for NN also, raise a new jira for hdfs. * I don't think it will work, because if we want to get the updated values for 'yarn.resourcemanager.nodes.include-path' and 'yarn.resourcemanager.nodes.exclude-path', need to reload the configuration. * There are some chinese characters have come in the patch, can we avoid them. Add exclude/include file , need restart NN or RM. - Key: HADOOP-8351 URL: https://issues.apache.org/jira/browse/HADOOP-8351 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 0.23.1 Environment: suse Reporter: xieguiming Attachments: HADOOP-835-0.patch, HADOOP-835.patch yarn.resourcemanager.nodes.include-path default value is , if we need add one include file. and we must restart the RM. I suggest that adding one include or exclude file, no need restart the RM. only execute the refresh command. NN is the same. Modify the HostsFileReader class: public HostsFileReader(String inFile, String exFile) to public HostsFileReader(Configuration conf, String NODES_INCLUDE_FILE_PATH,String DEFAULT_NODES_INCLUDE_FILE_PATH, String NODES_EXCLUDE_FILE_PATH,String DEFAULT_NODES_EXCLUDE_FILE_PATH) and thus, we can read the config file dynamic. and no need to restart the NM/NN. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
