Hi MrWell,
You should set the KYLIN_EXTRA_START_OPTS instead of KYLIN_JVM_SETTINGS
exec hbase ${KYLIN_EXTRA_START_OPTS}
> -Dkylin.hive.dependency=${hive_dependency}
> -Dkylin.hbase.dependency=${hbase_dependency}
> -Dlog4j.configuration=file:${KYLIN_HOME}/conf/kylin-tools-log4j.properties
> "$@"
It is *HBase Client *that became OOM because of the default small Java
memory size.
So, you can "export HBASE_OPTS="-Xmx??????"" before executing
"bin/metastore.sh clean --delete true"
If you are working with CDH HBase, you can increase the "-Xmx268435456" in
/etc/hbase/conf/hbase-env.sh directly.
On Fri, Nov 22, 2019 at 4:30 PM MrWell <[email protected]> wrote:
> Hi Shaofeng Shi,
>
> I have set KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g" in setenv.sh, but it
> fail. Can metadata be larger than 16g?
>
> Thanks for reply.
>
> ------------------ 原始邮件 ------------------
> *发件人:* "ShaoFeng Shi"<[email protected]>;
> *发送时间:* 2019年11月22日(星期五) 下午3:18
> *收件人:* "user"<[email protected]>;
> *主题:* Re: metastore clean OutOfMemoryError
>
> hi Huangpeng,
>
> I guess your JVM heap is small or your Kylin metadata is big. You can try
> to increase the java heap in "bin/setenv.sh"
>
> Best regards,
>
> Shaofeng Shi 史少锋
> Apache Kylin PMC
> Email: [email protected]
>
> Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
> Join Kylin user mail group: [email protected]
> Join Kylin dev mail group: [email protected]
>
>
>
>
> MrWell <[email protected]> 于2019年11月22日周五 下午3:01写道：
>
>> Hi, Kylin Team.
>>
>> When I execute "bin/metastore.sh clean --delete true" , I get a
>> "OutOfMemoryError" like this
>>
>>
>> java.lang.OutOfMemoryError: Java heap space
>> Dumping heap to java_pid4839.hprof ...
>> Heap dump file created [317991670 bytes in 2.120 secs]
>> #
>> # java.lang.OutOfMemoryError: Java heap space
>> # -XX:OnOutOfMemoryError="kill -9 %p"
>> # Executing /bin/sh -c "kill -9 4839"...
>> bin/metastore.sh: line 109: 4839 Killed
>> ${KYLIN_HOME}/bin/kylin.sh org.apache.kylin.tool.MetadataCleanupJob "${@:2}"
>>
>>
>> I have set 'setenv.sh' file, like this
>>
>> export KYLIN_JVM_SETTINGS="-Xms16g -Xmx16g -XX:MaxPermSize=512m
>> -XX:NewSize=3g -XX:MaxNewSize=3g -XX:SurvivorRatio=4
>> -XX:+CMSClassUnloadingEnabled -XX:+CMSParallelRemarkEnabled
>> -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode
>> -XX:CMSInitiatingOccupancyFraction=70 -XX:+UseCMSInitiatingOccupancyOnly
>> -XX:+DisableExplicitGC -XX:+HeapDumpOnOutOfMemoryError -verbose:gc
>> -XX:+PrintGCDetails -XX:+PrintGCDateStamps
>> -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ -XX:+UseGCLogFileRotation
>> -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M"
>>
>> Dose it means heap memory is still small?
>>
>
