Re: java.lang.OutOfMemoryError: Java heap space
1.3G is too big as a lookup table. Consider make it part of fact table, by using a view to join it with the origin fact for example. On Tue, Oct 11, 2016 at 9:31 AM, Billy(Yiming) Liu wrote: > 50,000,000 is ultra high cardinality. What kind of Dictionary did you > choose? > > 2016-10-11 9:16 GMT+08:00 沙漠火狐 <278211...@qq.com>: > > > thanks xwhfcen...@gmail.com > > > > > > in /bin/setev.sh my setting is > > > > > > export KYLIN_JVM_SETTINGS="-Xms2048M -Xmx10240M -Xss1024K > > -XX:MaxPermSize=10240M -verbose:gc -XX:+PrintGCDetails > > -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ > > -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 > -XX:GCLogFileSize=64M" > > > > > > the problem is still. or i any config setting is wrong? > > > > > > > > > > -- 原始邮件 ------ > > 发件人: "Mario Copperfield";; > > 发送时间: 2016年10月10日(星期一) 下午5:26 > > 收件人: "dev"; > > > > 主题: Re: java.lang.OutOfMemoryError: Java heap space > > > > > > > > can try set KYLIN_JVM_SETTINGS in /bin/setenv.sh > > > > On Mon, Oct 10, 2016 at 4:55 PM, 沙漠火狐 <278211...@qq.com> wrote: > > > > > hi > > >I have a big lookuptable about 1.3G, when I cube it, in > > > /logs/kylin.out say : > > > > > > > > > Oct 10, 2016 4:20:22 PM org.apache.catalina.startup.Catalina start > > > INFO: Server startup in 24004 ms > > > Found segment ordrpath_goodsid[2016100800_2016100900] > > > # > > > # java.lang.OutOfMemoryError: Java heap space > > > # -XX:OnOutOfMemoryError="kill -9 %p" > > > # Executing /bin/sh -c "kill -9 23272"... > > > > > > > > > > > > > > > then the kylin is stop. > > > > > > > > > so I add JAVA_OPTS="-Xms256m -Xmx10240m -XX:PermSize=128m > > > -XX:MaxPermSize=4096m" config in catalin.sh > > > > > > > > > and change the config in kylin_job_conf_inmem.xml to > > > > > > mapreduce.map.memory.mb > > > 4096 > > > > > > > > > > > > > > > > > > mapreduce.map.java.opts > > > -Xmx10240m > > > > > > > > > > > > > > > I also change the propery in kylin.properies > > > kylin.dictionary.max.cardinality=5000 > > > kylin.table.snapshot.max_mb=1500 > > > > > > > > > > > > > > > > > > > > > > > > but the problem is still in.is there any config i should add to > solve > > > this problem. thanks ! > > > > > > > > > > -- > > Best regards, > > Amuro Copperfield > > > > > > -- > With Warm regards > > Yiming Liu (刘一鸣) >
Re: java.lang.OutOfMemoryError: Java heap space
50,000,000 is ultra high cardinality. What kind of Dictionary did you choose? 2016-10-11 9:16 GMT+08:00 沙漠火狐 <278211...@qq.com>: > thanks xwhfcen...@gmail.com > > > in /bin/setev.sh my setting is > > > export KYLIN_JVM_SETTINGS="-Xms2048M -Xmx10240M -Xss1024K > -XX:MaxPermSize=10240M -verbose:gc -XX:+PrintGCDetails > -XX:+PrintGCDateStamps -Xloggc:$KYLIN_HOME/logs/kylin.gc.$$ > -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=64M" > > > the problem is still. or i any config setting is wrong? > > > > > -- 原始邮件 -- > 发件人: "Mario Copperfield";; > 发送时间: 2016年10月10日(星期一) 下午5:26 > 收件人: "dev"; > > 主题: Re: java.lang.OutOfMemoryError: Java heap space > > > > can try set KYLIN_JVM_SETTINGS in /bin/setenv.sh > > On Mon, Oct 10, 2016 at 4:55 PM, 沙漠火狐 <278211...@qq.com> wrote: > > > hi > >I have a big lookuptable about 1.3G, when I cube it, in > > /logs/kylin.out say : > > > > > > Oct 10, 2016 4:20:22 PM org.apache.catalina.startup.Catalina start > > INFO: Server startup in 24004 ms > > Found segment ordrpath_goodsid[2016100800_2016100900] > > # > > # java.lang.OutOfMemoryError: Java heap space > > # -XX:OnOutOfMemoryError="kill -9 %p" > > # Executing /bin/sh -c "kill -9 23272"... > > > > > > > > > > then the kylin is stop. > > > > > > so I add JAVA_OPTS="-Xms256m -Xmx10240m -XX:PermSize=128m > > -XX:MaxPermSize=4096m" config in catalin.sh > > > > > > and change the config in kylin_job_conf_inmem.xml to > > > > mapreduce.map.memory.mb > > 4096 > > > > > > > > > > > > mapreduce.map.java.opts > > -Xmx10240m > > > > > > > > > > I also change the propery in kylin.properies > > kylin.dictionary.max.cardinality=5000 > > kylin.table.snapshot.max_mb=1500 > > > > > > > > > > > > > > > > but the problem is still in.is there any config i should add to solve > > this problem. thanks ! > > > > > -- > Best regards, > Amuro Copperfield > -- With Warm regards Yiming Liu (刘一鸣)
Re: java.lang.OutOfMemoryError: Java heap space
can try set KYLIN_JVM_SETTINGS in /bin/setenv.sh On Mon, Oct 10, 2016 at 4:55 PM, 沙漠火狐 <278211...@qq.com> wrote: > hi >I have a big lookuptable about 1.3G, when I cube it, in > /logs/kylin.out say : > > > Oct 10, 2016 4:20:22 PM org.apache.catalina.startup.Catalina start > INFO: Server startup in 24004 ms > Found segment ordrpath_goodsid[2016100800_2016100900] > # > # java.lang.OutOfMemoryError: Java heap space > # -XX:OnOutOfMemoryError="kill -9 %p" > # Executing /bin/sh -c "kill -9 23272"... > > > > > then the kylin is stop. > > > so I add JAVA_OPTS="-Xms256m -Xmx10240m -XX:PermSize=128m > -XX:MaxPermSize=4096m" config in catalin.sh > > > and change the config in kylin_job_conf_inmem.xml to > > mapreduce.map.memory.mb > 4096 > > > > > > mapreduce.map.java.opts > -Xmx10240m > > > > > I also change the propery in kylin.properies > kylin.dictionary.max.cardinality=5000 > kylin.table.snapshot.max_mb=1500 > > > > > > > > but the problem is still in.is there any config i should add to solve > this problem. thanks ! -- Best regards, Amuro Copperfield