[jira] [Comment Edited] (YARN-7707) [GPG] Policy generator framework

2018-02-27 Thread Giovanni Matteo Fumarola (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16379546#comment-16379546
 ] 

Giovanni Matteo Fumarola edited comment on YARN-7707 at 2/28/18 12:25 AM:
--

Thanks [~youchen] for the patch. I took a quick look and I have few starting 
comments:

*YarnConfiguration*
* Can you add a new prefix FEDERATION_GPG_POLICY_PREFIX for 
FEDERATION_GPG_PREFIX + "policy.generator"?

*pom.xml*
 * Why do you need {{hadoop-yarn-server-timelineservice}}?

*GPGUtils*
 * The javadoc comment should be different for {{createUniformWeights}}.

*DefaultGlobalPolicy*
 * Can you improve the javadoc of it?
 * You add a log line and it says "Updating..." but you just return null. There 
will be some confusion in the logs. Please change to something else.

*package-info.java*
 * Add javadoc for it.

*UniformWeightedLocalityGlobalPolicy*
 * Line 50 and 52 are identical.

Please revisit all the javadoc in {{PolicyGenerator}} and {{GPGPolicyFacade}}.
 You use {{state store}} in few comments, can you change to 
{{FederationStateStore}} to avoid confusion with ZK StateStore.


was (Author: giovanni.fumarola):
Thanks [~youchen] for the patch. I took a quick look and I have few starting 
comments:

*pom.xml*
* Why do you need {{hadoop-yarn-server-timelineservice}}?

*GPGUtils*
* The javadoc comment should be different for {{createUniformWeights}}.

*DefaultGlobalPolicy*
* Can you improve the javadoc of it?
* You add a log line and it says "Updating..." but you just return null. There 
will be some confusion in the logs. Please change to something else.

*package-info.java*
* Add javadoc for it.

*UniformWeightedLocalityGlobalPolicy*
* Line 50 and 52 are identical.

Please revisit all the javadoc in {{PolicyGenerator}} and {{GPGPolicyFacade}}.
You use {{state store}} in few comments, can you change to 
{{FederationStateStore}} to avoid confusion with ZK StateStore.

> [GPG] Policy generator framework
> 
>
> Key: YARN-7707
> URL: https://issues.apache.org/jira/browse/YARN-7707
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Carlo Curino
>Assignee: Young Chen
>Priority: Major
>  Labels: federation, gpg
> Attachments: YARN-7707-YARN-7402.01.patch, 
> YARN-7707-YARN-7402.02.patch, YARN-7707-YARN-7402.03.patch, 
> YARN-7707-YARN-7402.04.patch, YARN-7707-YARN-7402.05.patch, 
> YARN-7707-YARN-7402.06.patch
>
>
> This JIRA tracks the development of a generic framework for querying 
> sub-clusters for metrics, running policies, and updating them in the 
> FederationStateStore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (YARN-7707) [GPG] Policy generator framework

2018-02-15 Thread Botong Huang (JIRA) 

[ 
https://issues.apache.org/jira/browse/YARN-7707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16366518#comment-16366518
 ] 

Botong Huang edited comment on YARN-7707 at 2/16/18 1:47 AM:
-

Thanks [~youchen] for the patch. A few comments: 
1. Please remove {{PolicyGeneratorService}} and schedule the runnable directly 
in {{PolicyGenerator}}, similar to {{SubClusterCleaner}}
2. It is better to be able to disable {{PolicyGenerator}} from config. In fact, 
let's make the default DEFAULT_GPG_POLICY_GENERATOR_INTERVAL_MS to be -1. When 
the configured value is not positive, do not schedule the service at all, again 
similar to {{SubClusterCleaner}}. 
3. Why do we need hadoop-yarn-server-timelineservice dependency in GPG's pom 
file? 
4. In {{PolicyGeneratorService.extractQueues()}}, we are extracting all (leaf 
and non-leaf) queues. Do we need to generate policy for non-leaf queues? My 
understanding is that no applications will be running in non-leaf queues. 


was (Author: botong):
Thanks [~youchen] for the patch. A few comments: 
1. Please remove {{PolicyGeneratorService}} and schedule the runnable directly 
in {{PolicyGenerator}}, similar to {{SubClusterCleaner}}
2. It is better to be able to disable {{PolicyGenerator}} from config. In fact, 
let's make the default DEFAULT_GPG_POLICY_GENERATOR_INTERVAL_MS to be -1. When 
the configured value is not positive, do not scheduler the service at all, 
again similar to {{SubClusterCleaner}}. 
3. Why do we need hadoop-yarn-server-timelineservice dependency in GPG's pom 
file? 
4. In {{PolicyGeneratorService.extractQueues()}}, we are extracting all (leaf 
and non-leaf) queues. Do we need to generate policy for non-leaf queues? My 
understanding is that no applications will be running in non-leaf queues. 

> [GPG] Policy generator framework
> 
>
> Key: YARN-7707
> URL: https://issues.apache.org/jira/browse/YARN-7707
> Project: Hadoop YARN
>  Issue Type: Sub-task
>Reporter: Carlo Curino
>Assignee: Young Chen
>Priority: Major
>  Labels: federation, gpg
> Attachments: YARN-7707-YARN-7402.01.patch, 
> YARN-7707-YARN-7402.02.patch, YARN-7707-YARN-7402.03.patch
>
>
> This JIRA tracks the development of a generic framework for querying 
> sub-clusters for metrics, running policies, and updating them in the 
> FederationStateStore.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org