[
https://issues.apache.org/jira/browse/HIVE-24707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275918#comment-17275918
]
David Mollitor commented on HIVE-24707:
---------------------------------------
And here:
https://github.com/apache/hive/blob/6deef2c6ca5561204e364bb6724cdcd546cdbec7/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L1918-L1927
> Apply Sane Default for Tez Containers as Last Resort
> ----------------------------------------------------
>
> Key: HIVE-24707
> URL: https://issues.apache.org/jira/browse/HIVE-24707
> Project: Hive
> Issue Type: Improvement
> Reporter: David Mollitor
> Priority: Trivial
>
> {code:java|title=DagUtils.java}
> public static Resource getContainerResource(Configuration conf) {
> int memory = HiveConf.getIntVar(conf,
> HiveConf.ConfVars.HIVETEZCONTAINERSIZE) > 0 ?
> HiveConf.getIntVar(conf, HiveConf.ConfVars.HIVETEZCONTAINERSIZE) :
> conf.getInt(MRJobConfig.MAP_MEMORY_MB,
> MRJobConfig.DEFAULT_MAP_MEMORY_MB);
> int cpus = HiveConf.getIntVar(conf, HiveConf.ConfVars.HIVETEZCPUVCORES) >
> 0 ?
> HiveConf.getIntVar(conf, HiveConf.ConfVars.HIVETEZCPUVCORES) :
> conf.getInt(MRJobConfig.MAP_CPU_VCORES,
> MRJobConfig.DEFAULT_MAP_CPU_VCORES);
> return Resource.newInstance(memory, cpus);
> }
> {code}
> If Tez Container Size or VCores is an invalid value ( <= 0 ) then it falls
> back onto the MapReduce configurations, but if the MapReduce configurations
> have invalid values ( <= 0 ), they are excepted regardless and this will
> cause failures down the road.
> This code should also check the MapReduce values and fall back to MapReduce
> default values if they are <= 0.
> Also, some logging would be nice here too, reporting about where the
> configuration values came from.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
