Redistribute Flat Hive Table is done by hive sql. Please add related
configs to kylin_hive_conf.xml of kylin job swerver.
-Shaohui Liu
On 07/19/2018 01:47 PM, 赵希欣 wrote:
Hi Kylin users/developers
Kylin 2.3.0, hadoop 2.6.5 cdh 5.12
A error occurred when I built a cube at step 2 Redistribute Flat Hive
Table. I checked the MR log and the log shows that reducers containers
is running beyond physical memory limits
below is detail logs:
Diagnostic Messages for this Task:
*Container
[pid=50920,containerID=container_1518617687620_384770_01_000287] is
running beyond physical memory limits. Current usage: 1.0 GB of 1 GB
physical memory used; 2.6 GB of 2.1 GB virtual memory used. Killing
container.
*
then I try to overwrites the related config
kylin.engine.mr.config-override.mapreduce.reduce.memory.mb = 4096
kylin.engine.mr.config-override.mapreduce.reduce.java.opts = -Xmx3g
then built the cube again, the the same error came again.
I check the yarn config mapreduce.reduce.memory.mb = 2G. this time I'm
fully confused. where kylin get the 1GB physical memory config limit?
why the overwrites not work?
--
BR
Xixin
--
智能云.云技术.计算平台组 刘绍辉
主要负责计算平台资源调度服务(Yarn)和分布式并行计算框架（Spark,MR等）的开发
电话： 15210682070
邮箱： [email protected]
