[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683454#comment-13683454 ] Tom White commented on YARN-803: Looks good to me. Is there a reason FairScheduler#validateConf is public? +1 otherwise. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13683457#comment-13683457 ] Alejandro Abdelnur commented on YARN-803: - that is a typo, i'll fix and upload patch, before committing. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[
https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682434#comment-13682434
]
Hadoop QA commented on YARN-803:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587663/YARN-803.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1217//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1217//console
This message is automatically generated.
factor out scheduler config validation from the ResourceManager to each
scheduler implementation
Key: YARN-803
URL: https://issues.apache.org/jira/browse/YARN-803
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager, scheduler
Affects Versions: 2.0.4-alpha
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Attachments: YARN-803.patch
Per discussion in YARN-789 we should factor out from the ResourceManager
class the scheduler config validations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682444#comment-13682444 ] Zhijie Shen commented on YARN-803: -- It sounds a good idea to support polymorphy of config validation. The patch looks almost fine. Here's two minor suggestions: 1. As all schedulers have implemented setConf, how about defining it in YarnScheduler as well? Therefore, the newly added scheduler in the future will also be forced to implement the method to validate its config (probably it have to do so). 2. setConf sounds a bit confusing, because the method doesn't set the config, but validate it. How about renaming it as validateConf? factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682495#comment-13682495 ] Alejandro Abdelnur commented on YARN-803: - Zhijie, On adding the setConf() method to the scheduler API, it is up to the scheduler impl to use or not the Yarn configuration. If a scheduler impl decides to do so, by implementing Configurable it will get it. I've initially thought about adding a validateConf() method to the scheduler API. I've decided to go the Configurable route for the reason above. Because of that, I think we should do things as in the current patch. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682503#comment-13682503 ] Sandy Ryza commented on YARN-803: - A Configuration already gets passed in when a scheduler's reinitialize method is called. Would it be possible to do it there? factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682564#comment-13682564 ] Alejandro Abdelnur commented on YARN-803: - mmh, so i guess we could do a mix of what [~zjshen] and [~sandyr] suggest, have a private validateConf() in the schedulers and call it from the reinitialize(). I'll upload a patch with that approach. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682619#comment-13682619 ] Sandy Ryza commented on YARN-803: - Would it make sense to put validateConf in SchedulerUtils, where we also have normalizeRequests? Otherwise, LGTM. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682636#comment-13682636 ] Sandy Ryza commented on YARN-803: - +1 factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682634#comment-13682634 ] Alejandro Abdelnur commented on YARN-803: - Sandhy, SchedulerUtils is for all schedulers, the validation is specific to a scheduler impl. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682635#comment-13682635 ] Sandy Ryza commented on YARN-803: - Ok, makes sense. factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[ https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682652#comment-13682652 ] Zhijie Shen commented on YARN-803: -- looks good. +1 factor out scheduler config validation from the ResourceManager to each scheduler implementation Key: YARN-803 URL: https://issues.apache.org/jira/browse/YARN-803 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager, scheduler Affects Versions: 2.0.4-alpha Reporter: Alejandro Abdelnur Assignee: Alejandro Abdelnur Attachments: YARN-803.patch, YARN-803.patch Per discussion in YARN-789 we should factor out from the ResourceManager class the scheduler config validations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-803) factor out scheduler config validation from the ResourceManager to each scheduler implementation
[
https://issues.apache.org/jira/browse/YARN-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13682709#comment-13682709
]
Hadoop QA commented on YARN-803:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12587688/YARN-803.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1222//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1222//console
This message is automatically generated.
factor out scheduler config validation from the ResourceManager to each
scheduler implementation
Key: YARN-803
URL: https://issues.apache.org/jira/browse/YARN-803
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager, scheduler
Affects Versions: 2.0.4-alpha
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Attachments: YARN-803.patch, YARN-803.patch
Per discussion in YARN-789 we should factor out from the ResourceManager
class the scheduler config validations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
