[ https://issues.apache.org/jira/browse/YARN-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780830#comment-16780830 ]
Adam Antal commented on YARN-9326: ---------------------------------- Here you can find my patch v4, please tell me your thoughts. I incorporated the changes you suggested [~wilfreds], but one point is not clear to me: the config {{yarn.scheduler.maximum-allocation-mb}} is used in *maxContainerAllocation*, and not in *maxChildResources*. Is your comment concerning maxContainerAllocation? I think yarn.scheduler.maximum-allocation-vcores can be added to it anyways. What is the situation of resource types from container allocation point of view? I am still unable to build the hadoop site documents. When I run the command {{mvn package -Pdist -Pdocs -DskipTests}} (no native code compile), I got the following error message in the "Apache Hadoop Client Packaging Invariants for Test" component: {noformat} [INFO] -------< org.apache.hadoop:hadoop-client-check-test-invariants >-------- [INFO] Building Apache Hadoop Client Packaging Invariants for Test 3.3.0-SNAPSHOT [93/98] [INFO] --------------------------------[ pom ]--------------------------------- [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-client-check-test-invariants --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-banned-dependencies) @ hadoop-client-check-test-invariants --- [INFO] Adding ignorable dependency: org.apache.hadoop:hadoop-annotations:null [INFO] Adding ignore: * [WARNING] Rule 1: org.apache.maven.plugins.enforcer.BanDuplicateClasses failed with message: Duplicate classes found: Found in: org.apache.hadoop:hadoop-client-runtime:jar:3.3.0-SNAPSHOT:compile org.apache.hadoop:hadoop-client-minicluster:jar:3.3.0-SNAPSHOT:compile Duplicate classes: org/apache/hadoop/shaded/org/apache/xerces/impl/dv/dtd/NMTOKENDatatypeValidator.class org/apache/hadoop/shaded/org/apache/xerces/dom/DOMImplementationListImpl.class org/apache/hadoop/shaded/org/apache/xerces/dom/DOMMessageFormatter.class org/apache/hadoop/shaded/org/apache/xml/serialize/XMLSerializer.class org/apache/hadoop/shaded/org/apache/xerces/util/URI.class org/apache/hadoop/shaded/org/apache/wml/dom/WMLOptionElementImpl.class org/apache/hadoop/shaded/org/apache/xerces/impl/xs/identity/Selector$Matcher.class ... {noformat} Has anyone seen errors like this? I'm really not a maven expert. I was following [https://github.com/apache/hadoop/blob/trunk/BUILDING.txt] for "site" goal. It may be something with my machine. Could you please confirm the mvn compile succeeds in your machine? > Fair Scheduler configuration defaults are not documented in case of min and > maxResources > ---------------------------------------------------------------------------------------- > > Key: YARN-9326 > URL: https://issues.apache.org/jira/browse/YARN-9326 > Project: Hadoop YARN > Issue Type: Improvement > Components: docs, documentation, fairscheduler, yarn > Affects Versions: 3.2.0 > Reporter: Adam Antal > Assignee: Adam Antal > Priority: Major > Attachments: YARN-9326.001.patch, YARN-9326.002.patch, > YARN-9326.003.patch, YARN-9326.004.patch > > > The FairScheduler's configuration has the following defaults (from the code: > javadoc): > {noformat} > In new style resources, any resource that is not specified will be set to > missing or 0%, as appropriate. Also, in the new style resources, units are > not allowed. Units are assumed from the resource manager's settings for the > resources when the value isn't a percentage. The missing parameter is only > used in the case of new style resources without percentages. With new style > resources with percentages, any missing resources will be assumed to be 100% > because percentages are only used with maximum resource limits. > {noformat} > This is not documented in the hadoop yarn site FairScheduler.html. It is > quite intuitive, but still need to be documented though. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org