[jira] [Created] (KYLIN-2316) Build Base Cuboid Data ERROR

[konglei (JIRA)]

konglei created KYLIN-2316:
--

 Summary:  Build Base Cuboid Data ERROR
 Key: KYLIN-2316
 URL: https://issues.apache.org/jira/browse/KYLIN-2316
 Project: Kylin
  Issue Type: Test
  Components: Web 
Affects Versions: v1.6.0
 Environment: hadoop:cdh 5.7.0
kylin:apache-kylin-1.6.0-cdh.5.7.0
system:centos 6.8
Reporter: konglei
Assignee: Zhong,Jason
 Fix For: v1.6.0


Dec 23, 2016 3:24:40 PM com.google.common.cache.LocalCache 
processPendingNotifications
WARNING: Exception thrown by removal listener
java.lang.RuntimeException: unexpected evict reason COLLECTED
at 
org.apache.kylin.dict.CachedTreeMap$1.onRemoval(CachedTreeMap.java:155)
at 
com.google.common.cache.LocalCache.processPendingNotifications(LocalCache.java:2004)
at 
com.google.common.cache.LocalCache$Segment.runUnlockedCleanup(LocalCache.java:3490)
at 
com.google.common.cache.LocalCache$Segment.postWriteCleanup(LocalCache.java:3466)
at 
com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2304)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
at 
com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:292)
at org.apache.kylin.dict.CachedTreeMap.get(CachedTreeMap.java:52)
at 
org.apache.kylin.dict.AppendTrieDictionary.getIdFromValueBytesImpl(AppendTrieDictionary.java:971)
at 
org.apache.kylin.common.util.Dictionary.getIdFromValueBytes(Dictionary.java:160)
at 
org.apache.kylin.dimension.DictionaryDimEnc.encode(DictionaryDimEnc.java:91)
at 
org.apache.kylin.cube.kv.RowKeyColumnIO.writeColumn(RowKeyColumnIO.java:55)
at 
org.apache.kylin.cube.kv.RowKeyEncoder.fillColumnValue(RowKeyEncoder.java:195)
at org.apache.kylin.cube.kv.RowKeyEncoder.encode(RowKeyEncoder.java:163)
at 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.buildKey(BaseCuboidMapperBase.java:151)
at 
org.apache.kylin.engine.mr.steps.BaseCuboidMapperBase.outputKV(BaseCuboidMapperBase.java:198)
at 
org.apache.kylin.engine.mr.steps.HiveToBaseCuboidMapper.map(HiveToBaseCuboidMapper.java:52)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-2315) Kylin run mode query not update metadata

[hoangle (JIRA)]

hoangle created KYLIN-2315:
--

 Summary: Kylin run mode query not update metadata
 Key: KYLIN-2315
 URL: https://issues.apache.org/jira/browse/KYLIN-2315
 Project: Kylin
  Issue Type: Bug
Affects Versions: v1.6.0
 Environment: CentOS
Reporter: hoangle


I'm running two kylin instance on two nodes in my cluster.
Two kylin point the same  kylin.rest.servers=mycluster:7070

Node1: I set kyin.server.mode=all ( for build cube and query)

Node2: I set kyin.server.mode=query ( for query only)

I face with problem when I finish build cube on Node 1, but then I move to Node 
2 I don't see any update about cube.

The cube data on Node 2 only update when I restart Kylin instance on Node 2




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-2314) Add Integration Test (IT) for snowflake

[liyang (JIRA)]

liyang created KYLIN-2314:
-

 Summary: Add Integration Test (IT) for snowflake
 Key: KYLIN-2314
 URL: https://issues.apache.org/jira/browse/KYLIN-2314
 Project: Kylin
  Issue Type: Improvement
Reporter: liyang






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (KYLIN-2313) Cannot find a cube in a subquery case with count distinct

[Dong Li (JIRA)]

Dong Li created KYLIN-2313:
--

 Summary: Cannot find a cube in a subquery case with count distinct
 Key: KYLIN-2313
 URL: https://issues.apache.org/jira/browse/KYLIN-2313
 Project: Kylin
  Issue Type: Bug
  Components: Query Engine
Affects Versions: v1.6.0
Reporter: Dong Li
Assignee: liyang
Priority: Minor


With sample cube, 
The first query can find a cube and give correct result:
select p.part_dt, p.lstg_site_id, p.grp, count(distinct user_id), sum(price) 
from (
select t.part_dt, t.lstg_site_id, t.user_id, t.price,
case t.lstg_format_name when 'ABIN' then 'AAA' when 'BBIN' then 'BBB' else 
'CCC' end as grp from kylin_sales t) p 
group by p.part_dt, p.lstg_site_id, p.grp

The second query will throw exception: cannot find any realization:
select p.part_dt, p.lstg_site_id, p.grp, count(distinct user_id), sum(price) 
from (
select t.part_dt, t.lstg_site_id, t.user_id, t.price,
case t.lstg_format_name when 'ABIN' then 'AAA'
when 'BBIN' then 'BBB'
else 'CCC' end as grp
from kylin_sales t
) p 
where p.part_dt='2013-01-01'
group by p.part_dt, p.lstg_site_id, p.grp

Error message:
Error while executing SQL "select p.part_dt, p.lstg_site_id, p.grp, 
count(distinct user_id), sum(price) from ( select t.part_dt, t.lstg_site_id, 
t.user_id, t.price, case t.lstg_format_name when 'ABIN' then 'AAA' when 'BBIN' 
then 'BBB' else 'CCC' end as grp from kylin_sales t ) p where 
p.part_dt='2013-01-01' group by p.part_dt, p.lstg_site_id, p.grp LIMIT 5": 
Can't find any realization. Please confirm with providers. SQL digest: fact 
table DEFAULT.KYLIN_SALES,group by [DEFAULT.KYLIN_SALES.PART_DT, 
DEFAULT.KYLIN_SALES.LSTG_SITE_ID, 
UNKNOWN_MODEL:DEFAULT._KYLIN_TABLE.GRP],filter on 
[DEFAULT.KYLIN_SALES.PART_DT],with aggregates[FunctionDesc 
[expression=COUNT_DISTINCT, parameter=ParameterDesc [type=column, 
value=USER_ID, nextParam=null], returnType=null], FunctionDesc [expression=SUM, 
parameter=ParameterDesc [type=column, value=PRICE, nextParam=null], 
returnType=null]].



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSS] Call the next release v2.0?

[Li Yang]

Thank you all for the support!  I'm renaming the v1.6.1 to v2.0.0.  :-)

On Thu, Dec 22, 2016 at 9:03 PM, Li Yang  wrote:

> > lots of users are still running a HBase 0.98.x cluster in production.
> Bump to HBase 1.x may affect the adoption of the new release.
>
> Well understood. The plan is not abandon HBase 0.98. There will still be
> packages for 0.98, 1.x etc..  The difference is master branch will use
> HBase 1.x, and 0.98 support will be on a secondary branch.
>
> On Wed, Dec 21, 2016 at 9:23 PM, Dayue  wrote:
>
>> I'm afraid lots of users are still running a HBase 0.98.x cluster in
>> production. Bump to HBase 1.x may affect the adoption of the new release.
>>
>>
>> > 在 2016年12月21日，下午6:20，Li Yang  写道：
>> >
>> > https://issues.apache.org/jira/browse/KYLIN-2307
>> >
>> > To track the HBase 1.x on master task.
>> >
>> > On Tue, Dec 20, 2016 at 1:14 AM, 杨朝中 
>> wrote:
>> >
>> >> +1
>> >>
>> >> 2016-12-19 22:26 GMT+08:00 ShaoFeng Shi :
>> >>
>> >>> +1 The snowflake support deserves a new start point.
>> >>>
>> >>>
>> >>> 2016-12-19 21:48 GMT+08:00 Luke Han :
>> >>>
>>  +1
>> 
>>  Those big changes will make Kylin evolute to next generation and
>>  bring better experience, support more cases which have been asked
>>  many times from different users.
>> 
>>  Also agreed to Billy, the default branch should using HBase 1.x from
>>  this version, more and more production environment already upgraded
>>  to 1.x.
>> 
>>  Looking forward for that.
>> 
>>  Thanks.
>>  Luke
>> 
>> 
>>  Best Regards!
>>  -
>> 
>>  Luke Han
>> 
>>  On Mon, Dec 19, 2016 at 2:47 PM, Li Yang  wrote:
>> 
>> >> KYLIN-2006: Make job engine distributed and HA may be also a big
>>  change?
>> > Yes, that counts!  :-)
>> >
>> > On Mon, Dec 19, 2016 at 2:34 PM, 康凯森  wrote:
>> >
>> >> +1. KYLIN-2006: Make job engine distributed and HA may be also a
>> >> big
>> >> change?
>> >>
>> >>
>> >>
>> >>
>> >> -- 原始邮件 --
>> >> 发件人: "Li Yang";;
>> >> 发送时间: 2016年12月19日(星期一) 下午2:23
>> >> 收件人: "dev";
>> >>
>> >> 主题: [DISCUSS] Call the next release v2.0?
>> >>
>> >>
>> >>
>> >> Guys,
>> >>
>> >> I'm thinking maybe it's time to call the next release v2.0. Like to
>>  hear
>> >> your thoughts.
>> >>
>> >> Actually the current v1.6, for the streaming cubing capability, is
>> > already
>> >> a good candidate of v2.0. However there were some other big changes
>> > ongoing
>> >> and we decided to let v2.0 wait a bit.
>> >>
>> >> These big changes are:
>> >>
>> >> - KYLIN-1726: Streaming cubing we know.
>> >> - KYLIN-1875: Snowflake support. Big metadata change which is not
>> >>> fully
>> >> backward compatible.
>> >> - KYLIN-2195: All Kylin properties renamed, to follow a convention.
>> >> - KYLIN-2255: The old HBase storage (called v1 internally) is
>> >>> dropped.
>> >> Cubes created by v1.3 and before are no longer supported.
>> >>
>> >> With all these changes on master, the next release deserves a plus
>> >> on
>>  the
>> >> major version.
>> >>
>> >> What do you think?
>> >>
>> >>
>> >> Cheers
>> >> Yang
>> >>
>> >
>> 
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Best regards,
>> >>>
>> >>> Shaofeng Shi 史少锋
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> 这个家伙很懒，什么也没留下..
>> >>
>>
>>
>>
>

Re: can not get hive table

[Li Yang]

这是环境问题，外人只能靠猜的。我猜可能多套hive配置在环境里搞混了。

2016-12-22 10:58 GMT+08:00 35925138 <35925...@qq.com>:

> 这个和是不是default数据库还有关系吗？
>
>
>
>
> -- 原始邮件 --
> 发件人: "35925138";<35925...@qq.com>;
> 发送时间: 2016年12月22日(星期四) 上午10:33
> 收件人: "dev";
>
> 主题: 回复： can not get hive table
>
>
>
> 是的，在default数据库
>
>
>
>
>
> -- 原始邮件 --
> 发件人: "Cheng Wang";;
> 发送时间: 2016年12月22日(星期四) 上午10:22
> 收件人: "dev@kylin.apache.org";
>
> 主题: Re: can not get hive table
>
>
>
> 在hive客户端看一下表是不是在default数据库，还是其他数据库。
>
> On 12/22/16, 10:04 AM, "35925138" <35925...@qq.com> wrote:
>
> 我用hive客户端可以看到main_stu_action表，数据也都正常，使用hcatalog，客户端运行命令也能够查找到这个表，
> 可在kylin中要添加表的时候，就是失败，查看日志，报如下错误，请帮忙看看，我是哪里错误了。
>
>
> Caused by: NoSuchObjectException(message:default.main_stu_action table
> not found)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$
> HMSHandler.get_table_core(HiveMetaStore.java:1808)
> at org.apache.hadoop.hive.metastore.HiveMetaStore$
> HMSHandler.get_table(HiveMetaStore.java:1778)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.hive.metastore.RetryingHMSHandler.
> invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy48.get_table(Unknown Source)
> at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.
> getTable(HiveMetaStoreClient.java:1208)
> at org.apache.kylin.source.hive.CLIHiveClient.getHiveTableMeta(
> CLIHiveClient.java:79)
> at org.apache.kylin.source.hive.HiveSourceTableLoader.
> extractHiveTables(HiveSourceTableLoader.java:93)
>

Re: Kylin Performance

[Li Yang]

Very good work!

Btw, we are also doing benchmarks on SSB and TPC-H data sets, based on
below work. Will share more info soon.

- http://www.cs.umb.edu/~poneil/StarSchemaB.PDF
- https://github.com/hortonworks/hive-testbench


Cheers
Yang

On Wed, Dec 21, 2016 at 8:45 PM, Alberto Ramón 
wrote:

> When Kylin 2149  will be
> solved the performance will be* improve even more*, because:
>
> you know that 2016-05-05 Belongs to May, Week 18, and friday , but kylin
> doesnt know it
> It will try to calulate the combination of 2016-05-05 with January February
> March, ... Monday Tuesday ..., W1 W2 ..., Q2 Q3 Q4 ==> There are a lot of
> combination wasted
>
> 2016-12-21 12:57 GMT+01:00 Luke_Selina :
>
> > Great and Agree! But I still have an question like Alberto, why in an AGG
> > one
> > dim can use only one regulation(mandatory, join, hierachy)?
> >
> > --
> > View this message in context: http://apache-kylin.74782.x6.
> > nabble.com/Kylin-Performance-tp6713p6728.html
> > Sent from the Apache Kylin mailing list archive at Nabble.com.
> >
>

Re: Re: kylin 1.6 supports SQL right join

[Li Yang]

The sql looks simply doing the below. Not sure why it is this complicated.

select  CATE1, count (distinct CATE2)  from kylinlabel.USER_TAG group
by CATE1

The issue with original SQL is it does not have a select clause at the
highest level. If hide out sub-queries, what is left is

t5 left join t4

Where is the SELECT?


On Wed, Dec 21, 2016 at 1:30 PM, wang...@snqu.com  wrote:

> excute sql:
> (select cate1 from kylinlabel.USER_TAG group by cate1) as t5
> left join
> (select t3.CATE1,count(*) from (select  CATE1 ,CATE2 from
> kylinlabel.USER_TAG  group by CATE1 ,CATE2) as t3 group by t3.CATE1) as t4
> on (t4.cate1=t5.cate1)
>
> get error:
> Encountered "as" at line 1, column 56. Was expecting one of:  "ORDER"
> ... "LIMIT" ... "OFFSET" ... "FETCH" ... "UNION" ... "INTERSECT" ...
> "EXCEPT" ... "NOT" ... "IN" ... "BETWEEN" ... "LIKE" ... "SIMILAR" ... "="
> ... ">" ... "<" ... "<=" ... ">=" ... "<>" ... "+" ... "-" ... "*" ... "/"
> ... "||" ... "AND" ... "OR" ... "IS" ... "MEMBER" ... "SUBMULTISET" ...
> "MULTISET" ... "[" ... "OVERLAPS" ... "YEAR" ... "MONTH" ... "DAY" ...
> "HOUR" ... "MINUTE" ... "SECOND" ...
>
>
> wang...@snqu.com
>
> From: Billy Liu
> Date: 2016-12-21 12:19
> To: dev
> Subject: Re: kylin 1.6 supports SQL right join
> Kylin supports left join and inner join. The right join could be rewritten
> into left join. Could you have a try?
>
> 2016-12-21 11:52 GMT+08:00 wang...@snqu.com :
>
> > Hi
> > when I excute the sql:
> > (select  t3.CATE1,count(*) from (select  CATE1 ,CATE2 from
> > kylinlabel.USER_TAG  group by CATE1 ,CATE2) as t3 group by t3.CATE1) as
> t4
> > right join
> > (select cate1 from kylinlabel.USER_TAG group by cate1) as t5 on
> > (t4.cate1=t5.cate1)
> >
> > I got the error:
> > Encountered "as" at line 1, column 129. Was expecting one of: 
> > "ORDER" ... "LIMIT" ... "OFFSET" ... "FETCH" ... "UNION" ... "INTERSECT"
> > ... "EXCEPT" ... "NOT" ... "IN" ... "BETWEEN" ... "LIKE" ... "SIMILAR"
> ...
> > "=" ... ">" ... "<" ... "<=" ... ">=" ... "<>" ... "+" ... "-" ... "*"
> ...
> > "/" ... "||" ... "AND" ... "OR" ... "IS" ... "MEMBER" ... "SUBMULTISET"
> ...
> > "MULTISET" ... "[" ... "OVERLAPS" ... "YEAR" ... "MONTH" ... "DAY" ...
> > "HOUR" ... "MINUTE" ... "SECOND" ...
> >
> > I think the sql is right but cann't get the result, what can I do?
> >
> >
> >
> >
>

[jira] [Created] (KYLIN-2312) Display Server Config/Environment in order

[Billy Liu (JIRA)]

Billy Liu created KYLIN-2312:


 Summary: Display Server Config/Environment in order 
 Key: KYLIN-2312
 URL: https://issues.apache.org/jira/browse/KYLIN-2312
 Project: Kylin
  Issue Type: Improvement
  Components: Web 
Affects Versions: v1.6.0
Reporter: Billy Liu
Assignee: Billy Liu
Priority: Minor


The system tab page shows Server Config and Environment, it's useful for 
debugging, but the item order is undetermined currently. The Config should show 
the same order as the properties file. The Environment should show the items 
order by name. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

build cube error

[35925138]

when I build the cube, the step is  Extract Fact Table Distinct Columns  ,and 
the kylin create the table 
kylin_intermediate_ttt_39f8cf5a_b873_4606_a4b3_f52e99d5c771,  and it realy in 
the hive,but it is can not readed by kylin .
I already config the hive.metastore.uris,it's value is 
thrift://172.16.1.90:9083,and I start the hcatalog with the commond  
/home/hadooper/hive/bin/hive --service metastore -p 9083


but the kylin give me the error on the above step


java.lang.RuntimeException: java.io.IOException: 
NoSuchObjectException(message:default.kylin_intermediate_ttt_39f8cf5a_b873_4606_a4b3_f52e99d5c771
 table not found) at 
org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:110)
 at 
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:119)
 at 
org.apache.kylin.engine.mr.steps.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:103)
 at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)   at 
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
   at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
   at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
 at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
   at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: 
NoSuchObjectException(message:default.kylin_intermediate_ttt_39f8cf5a_b873_4606_a4b3_f52e99d5c771
 table not found) at 
org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:97)
 at 
org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
 at 
org.apache.kylin.source.hive.HiveMRInput$HiveTableInputFormat.configureJob(HiveMRInput.java:105)
 ... 11 more Caused by: 
NoSuchObjectException(message:default.kylin_intermediate_ttt_39f8cf5a_b873_4606_a4b3_f52e99d5c771
 table not found)   at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table_core(HiveMetaStore.java:1808)
at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1778)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)   
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
  at com.sun.proxy.$Proxy49.get_table(Unknown Source) at 
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:1208)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)   
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at 
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:152)
at com.sun.proxy.$Proxy50.getTable(Unknown Source)  at 
org.apache.hive.hcatalog.common.HCatUtil.getTable(HCatUtil.java:180) at 
org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:105)
 at 
org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:88)
 at 
org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
 ... 13 more result code:2

[jira] [Created] (KYLIN-2311) Build Cube

[konglei (JIRA)]

konglei created KYLIN-2311:
--

 Summary: Build Cube
 Key: KYLIN-2311
 URL: https://issues.apache.org/jira/browse/KYLIN-2311
 Project: Kylin
  Issue Type: Test
  Components: Job Engine
Affects Versions: v1.6.0
 Environment: hadoop:cdh 5.7.0 
kylin:apache-kylin-1.6.0-cdh5.7-bin.tar.gz
system:centos 6.8
Reporter: konglei
Assignee: Dong Li
Priority: Critical
 Fix For: v1.6.0


Error: java.io.IOException: Failed to build cube in mapper 1 at 
org.apache.kylin.engine.mr.steps.InMemCuboidMapper.cleanup(InMemCuboidMapper.java:145)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:148) at 
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at 
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at 
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at 
java.security.AccessController.doPrivileged(Native Method) at 
javax.security.auth.Subject.doAs(Subject.java:415) at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: 
java.util.concurrent.ExecutionException: java.lang.RuntimeException: 
java.io.IOException: java.io.IOException: java.lang.IllegalStateException at 
java.util.concurrent.FutureTask.report(FutureTask.java:122) at 
java.util.concurrent.FutureTask.get(FutureTask.java:188) at 
org.apache.kylin.engine.mr.steps.InMemCuboidMapper.cleanup(InMemCuboidMapper.java:143)
 ... 8 more Caused by: java.lang.RuntimeException: java.io.IOException: 
java.io.IOException: java.lang.IllegalStateException at 
org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:84)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at 
java.util.concurrent.FutureTask.run(FutureTask.java:262) at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
at java.lang.Thread.run(Thread.java:744) Caused by: java.io.IOException: 
java.io.IOException: java.lang.IllegalStateException at 
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:128)
 at 
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder.build(DoggedCubeBuilder.java:75)
 at 
org.apache.kylin.cube.inmemcubing.AbstractInMemCubeBuilder$1.run(AbstractInMemCubeBuilder.java:82)
 ... 5 more Caused by: java.io.IOException: java.lang.IllegalStateException at 
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.abort(DoggedCubeBuilder.java:196)
 at 
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.checkException(DoggedCubeBuilder.java:169)
 at 
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$BuildOnce.build(DoggedCubeBuilder.java:101)
 ... 7 more Caused by: java.lang.IllegalStateException at 
com.google.common.base.Preconditions.checkState(Preconditions.java:129) at 
org.apache.kylin.common.util.MemoryBudgetController.(MemoryBudgetController.java:73)
 at 
org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.makeMemoryBudget(InMemCubeBuilder.java:324)
 at 
org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.build(InMemCubeBuilder.java:174)
 at 
org.apache.kylin.cube.inmemcubing.InMemCubeBuilder.build(InMemCubeBuilder.java:137)
 at 
org.apache.kylin.cube.inmemcubing.DoggedCubeBuilder$SplitThread.run(DoggedCubeBuilder.java:284)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)