Thanks to Temple for the sharing! Best regards,
Shaofeng Shi 史少锋 Apache Kylin PMC Email: [email protected] Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: [email protected] Join Kylin dev mail group: [email protected] MrWell <[email protected]> 于2019年11月22日周五 下午5:03写道： > Hi Temple Zhou, > > Thanks!!! i have solved it. I think u r right. > > > ------------------ 原始邮件 ------------------ > *发件人:* "Temple Zhou"<[email protected]>; > *发送时间:* 2019年11月22日(星期五) 下午4:51 > *收件人:* "user"<[email protected]>; > *主题:* Re: metastore clean OutOfMemoryError > > Hi MrWell, > You should set the KYLIN_EXTRA_START_OPTS instead of KYLIN_JVM_SETTINGS > > exec hbase ${KYLIN_EXTRA_START_OPTS} >> -Dkylin.hive.dependency=${hive_dependency} >> -Dkylin.hbase.dependency=${hbase_dependency} >> -Dlog4j.configuration=file:${KYLIN_HOME}/conf/kylin-tools-log4j.properties >> "$@" > > > It is *HBase Client *that became OOM because of the default small Java > memory size. > > So, you can "export HBASE_OPTS="-Xmx??????"" before executing > "bin/metastore.sh clean --delete true" > If you are working with CDH HBase, you can increase the "-Xmx268435456" in > /etc/hbase/conf/hbase-env.sh directly. > > On Fri, Nov 22, 2019 at 4:30 PM MrWell <[email protected]> wrote: > >> Hi Shaofeng Shi, >> >> I have set KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g" in setenv.sh, but it >> fail. Can metadata be larger than 16g? >> >> Thanks for reply. >> >> ------------------ 原始邮件 ------------------ >> *发件人:* "ShaoFeng Shi"<[email protected]>; >> *发送时间:* 2019年11月22日(星期五) 下午3:18 >> *收件人:* "user"<[email protected]>; >> *主题:* Re: metastore clean OutOfMemoryError >> >> hi Huangpeng, >> >> I guess your JVM heap is small or your Kylin metadata is big. You can try >> to increase the java heap in "bin/setenv.sh" >> >> Best regards, >> >> Shaofeng Shi 史少锋 >> Apache Kylin PMC >> Email: [email protected] >> >> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html >> Join Kylin user mail group: [email protected] >> Join Kylin dev mail group: [email protected] >> >> >> >> >> MrWell <[email protected]> 于2019年11月22日周五 下午3:01写道： >> >>> Hi, Kylin Team. >>> >>> When I execute "bin/metastore.sh clean --delete true" , I get a >>> "OutOfMemoryError" like this >>> >>> >>> java.lang.OutOfMemoryError: Java heap space >>> Dumping heap to java_pid4839.hprof ... >>> Heap dump file created [317991670 bytes in 2.120 secs] >>> # >>> # java.lang.OutOfMemoryError: Java heap space >>> # -XX:OnOutOfMemoryError="kill -9 %p" >>> # Executing /bin/sh -c "kill -9 4839"... >>> bin/metastore.sh: line 109: 4839 Killed >>> ${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.MetadataCleanupJob "${@:2}" >>> >>> >>> I have set 'setenv.sh' file, like this >>> >>> export KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g -XX:MaxPermSize=512m >>> -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4 >>> -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled >>> -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode >>> -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly >>> -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc >>> -XX:+PrintGCDetails -XX:+PrintGCDateStamps >>> -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation >>> -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M" >>> >>> Dose it means heap memory is still small? >>> >>
