[jira] [Commented] (YARN-10565) Refactor CS queue initialization to simplify weight mode calculation
[
https://issues.apache.org/jira/browse/YARN-10565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496897#comment-17496897
]
Hadoop QA commented on YARN-10565:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Logfile || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red}{color} | {color:red} YARN-10565 does not apply to trunk. Rebase
required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-10565 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13018832/YARN-10565.002.patch |
| Console output |
https://ci-hadoop.apache.org/job/PreCommit-YARN-Build/1272/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
> Refactor CS queue initialization to simplify weight mode calculation
>
>
> Key: YARN-10565
> URL: https://issues.apache.org/jira/browse/YARN-10565
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Labels: pull-request-available
> Attachments: YARN-10565.001.patch, YARN-10565.002.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In YARN-10504 weight mode support was introduced to CS. This jira is a
> followup to simplify and restructure the initialization, so that the weight
> calculation/absolute/percentage mode is easier to understand and modify.
> To be refactored:
> * In ParentQueue.java#1099 the error message should be more specific, instead
> of the {{LOG.error("Fatal issue found: e", e);}}
> * AutoCreatedLeafQueue.clearConfigurableFields should clear NORMALIZED_WEIGHT
> just to be on the safe side
> * Uncomment the commented assertions in
> TestCapacitySchedulerAutoCreatedQueueBase.validateEffectiveMinResource
> * Check whether the assertion modification in TestRMWebServices is absolutely
> necessary or could be hiding a bug.
> * Same for TestRMWebServicesForCSWithPartitions.java
> Additional information:
> The original flow was modified to allow the dynamic weight-capacity
> calculation.
> This resulted in a new flow, which is now harder to understand.
> With a cleanup it could be made simpler, the duplicate calculations could be
> avoided.
> The changed functionality should either be explained (if deemed correct) or
> fixed (see YARN-10590).
> Investigate how the CS reinit works, it could contain some possibly redundant
> initialization code fragments.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10565) Refactor CS queue initialization to simplify weight mode calculation
[
https://issues.apache.org/jira/browse/YARN-10565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17496893#comment-17496893
]
Benjamin Teke commented on YARN-10565:
--
[~gandras] yes, it's mostly obsolete now, except for the first two points, for
which I added a PR.
> Refactor CS queue initialization to simplify weight mode calculation
>
>
> Key: YARN-10565
> URL: https://issues.apache.org/jira/browse/YARN-10565
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Labels: pull-request-available
> Attachments: YARN-10565.001.patch, YARN-10565.002.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In YARN-10504 weight mode support was introduced to CS. This jira is a
> followup to simplify and restructure the initialization, so that the weight
> calculation/absolute/percentage mode is easier to understand and modify.
> To be refactored:
> * In ParentQueue.java#1099 the error message should be more specific, instead
> of the {{LOG.error("Fatal issue found: e", e);}}
> * AutoCreatedLeafQueue.clearConfigurableFields should clear NORMALIZED_WEIGHT
> just to be on the safe side
> * Uncomment the commented assertions in
> TestCapacitySchedulerAutoCreatedQueueBase.validateEffectiveMinResource
> * Check whether the assertion modification in TestRMWebServices is absolutely
> necessary or could be hiding a bug.
> * Same for TestRMWebServicesForCSWithPartitions.java
> Additional information:
> The original flow was modified to allow the dynamic weight-capacity
> calculation.
> This resulted in a new flow, which is now harder to understand.
> With a cleanup it could be made simpler, the duplicate calculations could be
> avoided.
> The changed functionality should either be explained (if deemed correct) or
> fixed (see YARN-10590).
> Investigate how the CS reinit works, it could contain some possibly redundant
> initialization code fragments.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10565) Refactor CS queue initialization to simplify weight mode calculation
[
https://issues.apache.org/jira/browse/YARN-10565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17468411#comment-17468411
]
Andras Gyori commented on YARN-10565:
-
[~bteke] Is it still a viable fix? I think its description is obsolete.
> Refactor CS queue initialization to simplify weight mode calculation
>
>
> Key: YARN-10565
> URL: https://issues.apache.org/jira/browse/YARN-10565
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Benjamin Teke
>Assignee: Benjamin Teke
>Priority: Major
> Attachments: YARN-10565.001.patch, YARN-10565.002.patch
>
>
> In YARN-10504 weight mode support was introduced to CS. This jira is a
> followup to simplify and restructure the initialization, so that the weight
> calculation/absolute/percentage mode is easier to understand and modify.
> To be refactored:
> * In ParentQueue.java#1099 the error message should be more specific, instead
> of the {{LOG.error("Fatal issue found: e", e);}}
> * AutoCreatedLeafQueue.clearConfigurableFields should clear NORMALIZED_WEIGHT
> just to be on the safe side
> * Uncomment the commented assertions in
> TestCapacitySchedulerAutoCreatedQueueBase.validateEffectiveMinResource
> * Check whether the assertion modification in TestRMWebServices is absolutely
> necessary or could be hiding a bug.
> * Same for TestRMWebServicesForCSWithPartitions.java
> Additional information:
> The original flow was modified to allow the dynamic weight-capacity
> calculation.
> This resulted in a new flow, which is now harder to understand.
> With a cleanup it could be made simpler, the duplicate calculations could be
> avoided.
> The changed functionality should either be explained (if deemed correct) or
> fixed (see YARN-10590).
> Investigate how the CS reinit works, it could contain some possibly redundant
> initialization code fragments.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10565) Refactor CS queue initialization to simplify weight mode calculation
[ https://issues.apache.org/jira/browse/YARN-10565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17264003#comment-17264003 ] Benjamin Teke commented on YARN-10565: -- Added most of the changes from YARN-10506.011 patch, as it was abandoned there. > Refactor CS queue initialization to simplify weight mode calculation > > > Key: YARN-10565 > URL: https://issues.apache.org/jira/browse/YARN-10565 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Benjamin Teke >Priority: Major > Attachments: YARN-10565.001.patch > > > In YARN-10504 weight mode support was introduced to CS. This jira is a > followup to simplify and restructure the initialization, so that the weight > calculation/absolute/percentage mode is easier to understand and modify. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10565) Refactor CS queue initialization to simplify weight mode calculation
[ https://issues.apache.org/jira/browse/YARN-10565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17262597#comment-17262597 ] Benjamin Teke commented on YARN-10565: -- [~sunilg] created this item based on your [comment|https://issues.apache.org/jira/browse/YARN-10504?focusedCommentId=17262580=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17262580] on YARN-10504 > Refactor CS queue initialization to simplify weight mode calculation > > > Key: YARN-10565 > URL: https://issues.apache.org/jira/browse/YARN-10565 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Benjamin Teke >Priority: Major > > In YARN-10504 weight mode support was introduced to CS. This jira is a > followup to simplify and restructure the initialization, so that the weight > calculation/absolute/percentage mode is easier to understand and modify. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
