[
https://issues.apache.org/jira/browse/KYLIN-2590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16013644#comment-16013644
]
Shaofeng SHI commented on KYLIN-2590:
-------------------------------------
Good catch Lufeng! I remember if using MR, Hive will generate files which are
very small if no matched record, so that will not hit such exception. While
Tez's behavior is a little different, caused Kylin throws such exception. Kylin
need fix this. [~amuseme.lu], would you like to solve and then contribute a
patch?
> Cube build error on Step 7 Build Base Cuboid Data
> -------------------------------------------------
>
> Key: KYLIN-2590
> URL: https://issues.apache.org/jira/browse/KYLIN-2590
> Project: Kylin
> Issue Type: Bug
> Affects Versions: v1.6.0
> Reporter: lufeng
>
> The cube build failed on step 7 with following error message. Any help please
> Hadoop 2.7 HBase 1.1.2 Kylin 1.6.0 Tez 0.7.0
> <<<
> 2017-05-01 01:10:38,430 INFO [pool-9-thread-10] common.AbstractHadoopJob:499
> : HDFS meta dir is:
> file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
> 2017-05-01 01:10:38,430 INFO [pool-9-thread-10] common.AbstractHadoopJob:372
> : Job 'tmpfiles' updated --
> file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
> 2017-05-01 01:10:38,433 INFO [pool-9-thread-10] mapred.FileInputFormat:249 :
> Total input paths to process : 0
> 2017-05-01 01:10:38,433 INFO [pool-9-thread-10] common.AbstractHadoopJob:506
> : tempMetaFileString is :
> file:///opt/kylin/bin/../tomcat/temp/kylin_job_meta8686959176405233518/meta
> 2017-05-01 01:10:38,440 ERROR [pool-9-thread-10]
> common.MapReduceExecutable:127 : error execute
> MapReduceExecutable{id=37f887a1-1f2a-40d0-a4dc-9eca7be28ab7-06, name=Build
> Base Cuboid Data, state=RUNNING}
> java.lang.IllegalArgumentException: Map input splits are 0 bytes, something
> is wrong!
> at
> org.apache.kylin.engine.mr.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:555)
> at
> org.apache.kylin.engine.mr.steps.CuboidJob.setReduceTaskNum(CuboidJob.java:175)
> at org.apache.kylin.engine.mr.steps.CuboidJob.run(CuboidJob.java:138)
> at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:92)
> at
> org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:120)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
> at
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:57)
> at
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
> at
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:136)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> >>>>
> ---
> I sovled this issue by changing execution engine from Tez to MR。 Because Tez
> engine has zero file issue . https://issues.apache.org/jira/browse/HIVE-13988
> I think Kylin should not blocked by empty segment.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
