deniskuzZ commented on code in PR #4920:
URL: https://github.com/apache/hive/pull/4920#discussion_r1432779169
##########
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##########
@@ -2461,7 +2461,17 @@ public static enum ConfVars {
"The default input format for tez. Tez groups splits in the AM."),
HIVETEZCONTAINERSIZE("hive.tez.container.size", -1,
- "By default Tez will spawn containers of the size of a mapper. This
can be used to overwrite."),
+ "The memory in MB that's used by a Tez task container (TezChild) in
Tez container mode. Hive uses this \n"
+ + "property to create a Resource object which is accepted by Yarn (and
used in TezAM to ask for TezChild \n"
+ + "containers. This should be distinguished from the Tez AM's
(DAGAppMaster) memory, \n"
+ + "which is driven by tez.am.resource.memory.mb! \n"
+ + "Also, as Hive takes care of TezChild memory by setting this option,
there is no need \n "
+ + "to set tez.task.resource.memory.mb differently. \n"
+ + "One more thing to note that the final -Xmx arg for TezChild process
is not equal to this setting, \n "
+ + "because Tez considers a heap fraction (80%), so by default: \n"
+ + "Xmx = hive.tez.container.size *
tez.container.max.java.heap.fraction. \n"
+ + "In case of values <= 0, container size falls back to
mapreduce.map.memory.mb. \n"
+ + "LLAP note: this option has no effect in LLAP mode, because LLAP
deamons are sized in another way."),
Review Comment:
1. no closing bracket after ` (and use`
2. could we drop everything after `One more thing to note that` and maybe
add a link to
https://community.cloudera.com/t5/Community-Articles/Demystify-Apache-Tez-Memory-Tuning-Step-by-Step/ta-p/245279
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
