[
https://issues.apache.org/jira/browse/YARN-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780830#comment-16780830
]
Adam Antal commented on YARN-9326:
----------------------------------
Here you can find my patch v4, please tell me your thoughts.
I incorporated the changes you suggested [~wilfreds], but one point is not
clear to me: the config {{yarn.scheduler.maximum-allocation-mb}} is used in
*maxContainerAllocation*, and not in *maxChildResources*. Is your comment
concerning maxContainerAllocation? I think
yarn.scheduler.maximum-allocation-vcores can be added to it anyways. What is
the situation of resource types from container allocation point of view?
I am still unable to build the hadoop site documents. When I run the command
{{mvn package -Pdist -Pdocs -DskipTests}} (no native code compile), I got the
following error message in the "Apache Hadoop Client Packaging Invariants for
Test" component:
{noformat}
[INFO] -------< org.apache.hadoop:hadoop-client-check-test-invariants >--------
[INFO] Building Apache Hadoop Client Packaging Invariants for Test
3.3.0-SNAPSHOT [93/98]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @
hadoop-client-check-test-invariants ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies)
@ hadoop-client-check-test-invariants ---
[INFO] Adding ignorable dependency: org.apache.hadoop:hadoop-annotations:null
[INFO] Adding ignore: *
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed
with message:
Duplicate classes found:
Found in:
org.apache.hadoop:hadoop-client-runtime:jar:3.3.0-SNAPSHOT:compile
org.apache.hadoop:hadoop-client-minicluster:jar:3.3.0-SNAPSHOT:compile
Duplicate classes:
org/apache/hadoop/shaded/org/apache/xerces/impl/dv/dtd/NMTOKENDatatypeValidator.class
org/apache/hadoop/shaded/org/apache/xerces/dom/DOMImplementationListImpl.class
org/apache/hadoop/shaded/org/apache/xerces/dom/DOMMessageFormatter.class
org/apache/hadoop/shaded/org/apache/xml/serialize/XMLSerializer.class
org/apache/hadoop/shaded/org/apache/xerces/util/URI.class
org/apache/hadoop/shaded/org/apache/wml/dom/WMLOptionElementImpl.class
org/apache/hadoop/shaded/org/apache/xerces/impl/xs/identity/Selector$Matcher.class
...
{noformat}
Has anyone seen errors like this? I'm really not a maven expert. I was
following [https://github.com/apache/hadoop/blob/trunk/BUILDING.txt] for "site"
goal. It may be something with my machine. Could you please confirm the mvn
compile succeeds in your machine?
> Fair Scheduler configuration defaults are not documented in case of min and
> maxResources
> ----------------------------------------------------------------------------------------
>
> Key: YARN-9326
> URL: https://issues.apache.org/jira/browse/YARN-9326
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: docs, documentation, fairscheduler, yarn
> Affects Versions: 3.2.0
> Reporter: Adam Antal
> Assignee: Adam Antal
> Priority: Major
> Attachments: YARN-9326.001.patch, YARN-9326.002.patch,
> YARN-9326.003.patch, YARN-9326.004.patch
>
>
> The FairScheduler's configuration has the following defaults (from the code:
> javadoc):
> {noformat}
> In new style resources, any resource that is not specified will be set to
> missing or 0%, as appropriate. Also, in the new style resources, units are
> not allowed. Units are assumed from the resource manager's settings for the
> resources when the value isn't a percentage. The missing parameter is only
> used in the case of new style resources without percentages. With new style
> resources with percentages, any missing resources will be assumed to be 100%
> because percentages are only used with maximum resource limits.
> {noformat}
> This is not documented in the hadoop yarn site FairScheduler.html. It is
> quite intuitive, but still need to be documented though.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
