Hi, Yes, I tried to overwrite the following: kylin.job.mr.config.override.mapreduce.map.java.opts -Xmx256000m kylin.job.mr.config.override.mapreduce.map.memory.mb 320000 kylin.job.mr.config.override.mapreduce.reduce.memory.mb 640000 kylin.job.mr.config.override.mapreduce.reduce.java.opts-Xmx512000m
and increase set env : export KYLIN_JVM_SETTINGS="-Xms8g -Xmx300g -XX:MaxPermSize=512m -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.%p -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap" still getting errors as if i have not changed the RAM. and again the machine has 1.5 TB RAM. Please advice, any help will be appreciated. Thanks Omid On Fri, Aug 14, 2020 at 4:50 AM Yaqian Zhang <[email protected]> wrote: > Hi: > Do you use this method to configure the yarn parameter in Kylin? > > > 在 2020年8月13日，19:55，Omid Vahdaty <[email protected]> 写道： > > HI, > > Just installed kyling using docker. > > I performed a job and got a reduce error > > i have tried increasing the amount of RAM configuration for yarn get same > error > > i twould seem that regardless of what i configure i keep getting yarn > total moery 6GB and total vcores 4, even though u have 40 cores and 1.5TB > of ram and i have not limited the container to any limit. > > what is the correct way to allocate RAM for a kylin job or generally > speaking? > > > >
