[
https://issues.apache.org/jira/browse/AMBARI-5455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Srimanth Gunturi resolved AMBARI-5455.
--------------------------------------
Resolution: Fixed
+1 received. Change committed to trunk and branch-1.5.1.
Changes tested out by installing on fresh cluster and all service checks
passing. Updated values showed up in configs during and post install.
Validation messages were also showing up.
Latest value logic is
{code}
var containerSize = configs['mapreduce.map.memory.mb'] > 2048 ?
configs['mapreduce.map.memory.mb'] : configs['mapreduce.reduce.memory.mb'];
containerSize = Math.min(configs['yarn.scheduler.maximum-allocation-mb'],
containerSize);
configs['hive.auto.convert.join.noconditionaltask.size'] =
Math.round(containerSize / 3) * 1048576; // MB to Bytes
configs['hive.tez.java.opts'] = "-server -Xmx"+containerSize+"m
-Djava.net.preferIPv4Stack=true";
configs['hive.tez.container.size'] = containerSize;
{code}
> Ambari configuration for map join conversion and tez container size seems
> wrong
> -------------------------------------------------------------------------------
>
> Key: AMBARI-5455
> URL: https://issues.apache.org/jira/browse/AMBARI-5455
> Project: Ambari
> Issue Type: Bug
> Components: client
> Affects Versions: 1.5.1
> Reporter: Srimanth Gunturi
> Assignee: Srimanth Gunturi
> Priority: Critical
> Fix For: 1.5.1
>
> Attachments: AMBARI-5455.2.patch, AMBARI-5455.patch
>
>
> For hive:
> hive.auto.convert.join.noconditionaltask.size is set to 1000000000
> - This should be a fraction (1/3) of the container size.
> hive.tez.java.opts has "-Xmx1024m"
> This is different from both map and reduce sizes. Desired values are:
> - map size if map size > 2g else reduce size
> map size is set on the same cluster to ~500mb
> The settings as the are will lead to many failed queries because the mapjoin
> conversion is to aggressive. If we don't change the container sizes based on
> cluster configs we will see wide spread problems with containers being killed
> or perf problems.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
