abstractdog commented on code in PR #4920:
URL: https://github.com/apache/hive/pull/4920#discussion_r1433801520
##########
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:
##########
@@ -2461,7 +2461,17 @@ public static enum ConfVars {
"The default input format for tez. Tez groups splits in the AM."),
HIVETEZCONTAINERSIZE("hive.tez.container.size", -1,
- "By default Tez will spawn containers of the size of a mapper. This
can be used to overwrite."),
+ "The memory in MB that's used by a Tez task container (TezChild) in
Tez container mode. Hive uses this \n"
+ + "property to create a Resource object which is accepted by Yarn (and
used in TezAM to ask for TezChild \n"
+ + "containers. This should be distinguished from the Tez AM's
(DAGAppMaster) memory, \n"
+ + "which is driven by tez.am.resource.memory.mb! \n"
+ + "Also, as Hive takes care of TezChild memory by setting this option,
there is no need \n "
+ + "to set tez.task.resource.memory.mb differently. \n"
+ + "One more thing to note that the final -Xmx arg for TezChild process
is not equal to this setting, \n "
+ + "because Tez considers a heap fraction (80%), so by default: \n"
+ + "Xmx = hive.tez.container.size *
tez.container.max.java.heap.fraction. \n"
+ + "In case of values <= 0, container size falls back to
mapreduce.map.memory.mb. \n"
+ + "LLAP note: this option has no effect in LLAP mode, because LLAP
deamons are sized in another way."),
Review Comment:
1. I'll fix
2. maybe, but linking a cloudera blog post is not the Apache way, so if we
want to do it, we need to have an Apache page I believe...regarding the content
here: having 2-3 extra clarifying sentences should not hurt, because it
clarifies the most common confusing stuff here immediately, I still think it's
useful
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
