Sushanta Sen created YARN-10634:
-----------------------------------
Summary: The config parameter name is confusing when requesting
Opportunistic containers in YARN job
Key: YARN-10634
URL: https://issues.apache.org/jira/browse/YARN-10634
Project: Hadoop YARN
Issue Type: Bug
Components: applications
Reporter: Sushanta Sen
Execute the below job by Passing this config
-Dmapreduce.job.num-opportunistic-maps-percent ,which actually represents the
number of containers to be launched as Opportunistic, not in % of the total
mappers requested , i think this configuration name should be modified
accordingly and also {color:#de350b}the same gets printed in AM logs{color}
Job Command: hadoop jar
HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1-hw-ei-310001-SNAPSHOT.jar
pi -{color:#de350b}Dmapreduce.job.num-opportunistic-maps-percent{color}="20"
20 99
In AM logs this message is displayed. it should be {color:#de350b}20 , not 20%
{color}?
“2021-02-10 20:23:23,023 | INFO | main | {color:#de350b}20% of the
mappers{color} will be scheduled using OPPORTUNISTIC containers |
RMContainerAllocator.java:257”
Job Command: hadoop jar
HDFS/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1-hw-ei-310001-SNAPSHOT.jar
pi {color:#de350b}-Dmapreduce.job.num-opportunistic-maps-percent{color}="100"
20 99
In AM logs this message is displayed. It should be {color:#de350b}100, not
100%{color} ?
2021-02-10 20:28:16,016 | INFO | main | {color:#de350b}100% of the
mapper{color}s will be scheduled using OPPORTUNISTIC containers |
RMContainerAllocator.java:257
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
