[jira] [Commented] (YARN-10364) Absolute Resource [memory=0] is considered as Percentage config type
[
https://issues.apache.org/jira/browse/YARN-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165056#comment-17165056
]
Bilwa S T commented on YARN-10364:
--
Hi [~prabhujoseph]
This is because
CapacitySchedulerConfiguration#internalGetLabeledResourceRequirementForQueue
returns Resources.none() if memory is 0.
{code:java}
// Memory has to be configured always.
if (resource.getMemorySize() == 0L) {
return Resources.none();
}
{code}
When assigning config type it checks if minresource is not equal to
Resources.none() in AbstractCSQueue#updateConfigurableResourceRequirement
{code:java}
if (this.capacityConfigType.equals(CapacityConfigType.NONE)) {
this.capacityConfigType = (!minResource.equals(Resources.none())
&& queueCapacities.getAbsoluteCapacity(label) == 0f)
? CapacityConfigType.ABSOLUTE_RESOURCE
: CapacityConfigType.PERCENTAGE;
LOG.debug("capacityConfigType is updated as '{}' for queue {}",
capacityConfigType, getQueuePath());
}
{code}
> Absolute Resource [memory=0] is considered as Percentage config type
>
>
> Key: YARN-10364
> URL: https://issues.apache.org/jira/browse/YARN-10364
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Bilwa S T
>Priority: Major
>
> Absolute Resource [memory=0] is considered as Percentage config type. This
> causes failure while converting queues from Percentage to Absolute Resources
> automatically.
> *Repro:*
> 1. Queue A = 100% and child queues Queue A.B = 0%, A.C=100%
> 2. While converting above to absolute resource automatically, capacity of
> queue A = [memory=], A.B = [memory=0]
> This fails with below as A is considered as Absolute Resource whereas B is
> considered as Percentage config type.
> {code}
> 2020-07-23 09:36:40,499 WARN
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices:
> CapacityScheduler configuration validation failed:java.io.IOException: Failed
> to re-init queues : Parent queue 'root.A' and child queue 'root.A.B' should
> use either percentage based capacityconfiguration or absolute resource
> together for label:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10359) Log container report only if list is not empty
[
https://issues.apache.org/jira/browse/YARN-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17165053#comment-17165053
]
Hadoop QA commented on YARN-10359:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 28m
49s{color} | {color:red} Docker failed to build yetus/hadoop:cce5a6f6094.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-10359 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13008426/YARN-10359.002.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/26307/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> Log container report only if list is not empty
> --
>
> Key: YARN-10359
> URL: https://issues.apache.org/jira/browse/YARN-10359
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Minor
> Attachments: YARN-10359.001.patch, YARN-10359.002.patch
>
>
> In NodeStatusUpdaterImpl print log only if containerReports list is not empty
> {code:java}
> if (containerReports != null) {
> LOG.info("Registering with RM using containers :" + containerReports);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10359) Log container report only if list is not empty
[
https://issues.apache.org/jira/browse/YARN-10359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilwa S T updated YARN-10359:
-
Attachment: YARN-10359.002.patch
> Log container report only if list is not empty
> --
>
> Key: YARN-10359
> URL: https://issues.apache.org/jira/browse/YARN-10359
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Minor
> Attachments: YARN-10359.001.patch, YARN-10359.002.patch
>
>
> In NodeStatusUpdaterImpl print log only if containerReports list is not empty
> {code:java}
> if (containerReports != null) {
> LOG.info("Registering with RM using containers :" + containerReports);
> }
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10367) Failed to get nodejs 10.21.0 when building docker image
[
https://issues.apache.org/jira/browse/YARN-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka reassigned YARN-10367:
Assignee: Akira Ajisaka
> Failed to get nodejs 10.21.0 when building docker image
> ---
>
> Key: YARN-10367
> URL: https://issues.apache.org/jira/browse/YARN-10367
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, webapp
>Reporter: Akira Ajisaka
>Assignee: Akira Ajisaka
>Priority: Blocker
>
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/213/console
> {noformat}
> Version '10.21.0-1nodesource1' for 'nodejs' was not found
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10367) Failed to get nodejs 10.21.0 when building docker image
[
https://issues.apache.org/jira/browse/YARN-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17164923#comment-17164923
]
Akira Ajisaka commented on YARN-10367:
--
It seems that the old version has been removed from the nodejs repository. We
have to upgrade the nodejs version or change the repository to that hosts
previous versions.
> Failed to get nodejs 10.21.0 when building docker image
> ---
>
> Key: YARN-10367
> URL: https://issues.apache.org/jira/browse/YARN-10367
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, webapp
>Reporter: Akira Ajisaka
>Priority: Blocker
>
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/213/console
> {noformat}
> Version '10.21.0-1nodesource1' for 'nodejs' was not found
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-10367) Failed to get nodejs 10.21.0 when building docker image
[
https://issues.apache.org/jira/browse/YARN-10367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira Ajisaka moved HADOOP-17154 to YARN-10367:
---
Component/s: (was: build)
webapp
build
Key: YARN-10367 (was: HADOOP-17154)
Project: Hadoop YARN (was: Hadoop Common)
> Failed to get nodejs 10.21.0 when building docker image
> ---
>
> Key: YARN-10367
> URL: https://issues.apache.org/jira/browse/YARN-10367
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build, webapp
>Reporter: Akira Ajisaka
>Priority: Blocker
>
> https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/213/console
> {noformat}
> Version '10.21.0-1nodesource1' for 'nodejs' was not found
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10364) Absolute Resource [memory=0] is considered as Percentage config type
[
https://issues.apache.org/jira/browse/YARN-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilwa S T reassigned YARN-10364:
Assignee: Bilwa S T (was: Prabhu Joseph)
> Absolute Resource [memory=0] is considered as Percentage config type
>
>
> Key: YARN-10364
> URL: https://issues.apache.org/jira/browse/YARN-10364
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.4.0
>Reporter: Prabhu Joseph
>Assignee: Bilwa S T
>Priority: Major
>
> Absolute Resource [memory=0] is considered as Percentage config type. This
> causes failure while converting queues from Percentage to Absolute Resources
> automatically.
> *Repro:*
> 1. Queue A = 100% and child queues Queue A.B = 0%, A.C=100%
> 2. While converting above to absolute resource automatically, capacity of
> queue A = [memory=], A.B = [memory=0]
> This fails with below as A is considered as Absolute Resource whereas B is
> considered as Percentage config type.
> {code}
> 2020-07-23 09:36:40,499 WARN
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices:
> CapacityScheduler configuration validation failed:java.io.IOException: Failed
> to re-init queues : Parent queue 'root.A' and child queue 'root.A.B' should
> use either percentage based capacityconfiguration or absolute resource
> together for label:
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
