Laszlo Gaal (Cloudera) has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18630 )
Change subject: IMPALA-10316: Increase Yarn minimum container size for dataload ...................................................................... IMPALA-10316: Increase Yarn minimum container size for dataload This is an attempt to get rod of IMPALA-10669 and friends, crashing Tez containers during the loading of nested ORC data. The usual error message logged for these failures is: Container [pid=11530,containerID=container_1618776748992_0039_01_000003] is running 2785280B beyond the 'PHYSICAL' memory limit. Current usage: 1.0 GB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used. Killing container. https://stackoverflow.com/a/43827548/143681 explains that the tunable setting 'yarn.scheduler.minimum-allocation-mb' in yarn-site.xml sets both the minimum memory size and the memory size increment for Yarn containers This patch is an attempt to work around the failure by forcibly setting a minimum size for the Yarn containers used in dataload that is significantly larger than the 1 GB size reported in the failure messages. Tested by running the dataload phase successfully on the following platform combinations: - Ubuntu 16.04, m6i.8xlarge (128 GB RAM, Docker) - Ubuntu 16.04, m5.12xlarge (192 GB RAM, Docker) - Centos 7.4, m5.4xlarge (64 GB RAM) - Centos 7.4, r5.4xlarge (128 GB RAM) - Ubuntu 16.04, m6i.4xlarge (64 GB RAM) Change-Id: I77e7c9e9fa3491c6e5652351869d3a4410bbb7b8 Reviewed-on: http://gerrit.cloudera.org:8080/18630 Tested-by: Impala Public Jenkins <[email protected]> Reviewed-by: Michael Smith <[email protected]> Reviewed-by: Laszlo Gaal (Cloudera) <[email protected]> --- M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py 1 file changed, 4 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Verified Michael Smith: Looks good to me, but someone else must approve Laszlo Gaal (Cloudera): Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/18630 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I77e7c9e9fa3491c6e5652351869d3a4410bbb7b8 Gerrit-Change-Number: 18630 Gerrit-PatchSet: 5 Gerrit-Owner: Laszlo Gaal (Cloudera) <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal (Cloudera) <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]>
