Re: Cube build error on Step 7 Build Base Cuboid Data

[lufeng]

Hi ShaoFeng

I think it’s not an empty segment problem because Kylin has also consider this 
situation. Maybe is an none file issue that Kylin is not checked. 

I have open a issue on Kylin JIRA. Please check it 
https://issues.apache.org/jira/browse/KYLIN-2590 


> 在 2017年5月4日，上午9:28，ShaoFeng Shi  写道：
> 
> Lufeng,
> 
> Thanks for the update. I also received reporting about this error recently, 
> but no finding. Is it because the intermediate table be empty (no data in the 
> selected time range)? As Kylin allows empty segment, this should not block 
> the build, please feel free to open a JIRA to us. 

[jira] [Created] (KYLIN-2590) Cube build error on Step 7 Build Base Cuboid Data

[lufeng (JIRA)]

lufeng created KYLIN-2590:
-

 Summary: Cube build error on Step 7 Build Base Cuboid Data
 Key: KYLIN-2590
 URL: https://issues.apache.org/jira/browse/KYLIN-2590
 Project: Kylin
  Issue Type: Bug
Affects Versions: v1.6.0
Reporter: lufeng


The cube build failed on step 7 with following error message. Any help please 

Hadoop 2.7 HBase 1.1.2 Kylin 1.6.0 Tez 0.7.0

<<<
2017-05-01 01:10:38,430 INFO  [pool-9-thread-10] common.AbstractHadoopJob:499 : 
HDFS meta dir is: 
file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
2017-05-01 01:10:38,430 INFO  [pool-9-thread-10] common.AbstractHadoopJob:372 : 
Job 'tmpfiles' updated -- 
file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
2017-05-01 01:10:38,433 INFO  [pool-9-thread-10] mapred.FileInputFormat:249 : 
Total input paths to process : 0
2017-05-01 01:10:38,433 INFO  [pool-9-thread-10] common.AbstractHadoopJob:506 : 
tempMetaFileString is : 
file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
2017-05-01 01:10:38,440 ERROR [pool-9-thread-10] common.MapReduceExecutable:127 
: error execute MapReduceExecutable{id=37f887a1-1f2a-40d0-a4dc-9eca7be28ab7-06, 
name=Build Base Cuboid Data, state=RUNNING}
java.lang.IllegalArgumentException: Map input splits are 0 bytes, something is 
wrong!
at 
org.apache.kylin.engine.mr.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:555)
at 
org.apache.kylin.engine.mr.steps.CuboidJob.setReduceTaskNum(CuboidJob.java:175)
at org.apache.kylin.engine.mr.steps.CuboidJob.run(CuboidJob.java:138)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
at 
org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
at 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
at 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
>>>>

---
I sovled this issue by changing execution engine from Tez to MR。 Because Tez 
engine has zero file issue . https://issues.apache.org/jira/browse/HIVE-13988

I think Kylin should not blocked by empty segment. 




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Cube build error on Step 7 Build Base Cuboid Data

[ShaoFeng Shi]

Lufeng,

Thanks for the update. I also received reporting about this error recently,
but no finding. Is it because the intermediate table be empty (no data in
the selected time range)? As Kylin allows empty segment, this should not
block the build, please feel free to open a JIRA to us.