Hi Team, We are using Kylin 2.3.1, And in the merge Job(which gets automatically triggered), we are getting this error in Merge Cuboid Data Step;
java.io.IOException: No input paths specified in job at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:239) at org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387) at org.apache.kylin.engine.mr.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:622) at org.apache.kylin.storage.hbase.steps.HBaseMROutput2Transition$HBaseMergeMROutputFormat.configureJobOutput(HBaseMROutput2Transition.java:166) at org.apache.kylin.engine.mr.steps.MergeCuboidJob.run(MergeCuboidJob.java:82) at org.apache.kylin.engine.mr.common.MapReduceExecutable.doWork(MapReduceExecutable.java:130) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162) at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67) at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162) at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) result code:2 As it is a merge job, whenever I discard this job,, it again starts automatically. So we need a way(if there is one), of permanently discarding this job somehow. Also in case this can be done by changing the metadata for the cube segment, do let me know how it can be done(even if I have to delete/remove the segments for these below days) We are open for all options (except dropping the cube as it has more than 1 year data, and is crucial for organisation). We are stuck on this for some days now, help would really be appreciated. Merge Thresholds: 8 (Hours) 1 (Days) 7 (Days) 15 (Days) Merge Job, (start and end): MERGE CUBE - XXXX - 20181005080000_20181012170000 Here are list of segments; Segment: 20181005080000_20181006080000 Start Time: 2018-10-05 08:00:00 End Time: 2018-10-06 08:00:00 Source Count: 14899048 HBase Table: KYLIN_R1MUK56K71 Region Count: 1 Size: 860 MB Segment: 20181005080000_20181012170000 //**This segment Table was deleted from Hbase (somehow, we don’t have the reason)**// Start Time: 2018-10-05 08:00:00 End Time: 2018-10-12 17:00:00 Source Count: 0 HBase Table: KYLIN_CFLY2CKMCU Region Count: 3 Size: less than 1 MB Segment: 20181006080000_20181008000000 Start Time: 2018-10-06 08:00:00 End Time: 2018-10-08 00:00:00 Source Count: 24455686 HBase Table: KYLIN_0KH6PHTEM2 Region Count: 1 Size: 1.0498 GB Segment: 20181008000000_20181009000000 Start Time: 2018-10-08 00:00:00 End Time: 2018-10-09 00:00:00 Source Count: 14882090 HBase Table: KYLIN_V1CC4LDSIR Region Count: 1 Size: 598 MB Segment: 20181009000000_20181010000000 Start Time: 2018-10-09 00:00:00 End Time: 2018-10-10 00:00:00 Source Count: 16245847 HBase Table: KYLIN_4A44K2VJEU Region Count: 1 Size: 628 MB Segment: 20181010000000_20181010080000 Start Time: 2018-10-10 00:00:00 End Time: 2018-10-10 08:00:00 Source Count: 5213022 HBase Table: KYLIN_EHO316VC7M Region Count: 1 Size: 397 MB Segment: 20181010080000_20181010090000 Start Time: 2018-10-10 08:00:00 End Time: 2018-10-10 09:00:00 Source Count: 865722 HBase Table: KYLIN_I9LEJ2JDZ8 Region Count: 1 Size: 181 MB Segment: 20181010090000_20181010100000 Start Time: 2018-10-10 09:00:00 End Time: 2018-10-10 10:00:00 Source Count: 859127 HBase Table: KYLIN_9IBX3W4UNL Region Count: 1 Size: 180 MB Segment: 20181010100000_20181010110000 Start Time: 2018-10-10 10:00:00 End Time: 2018-10-10 11:00:00 Source Count: 855752 HBase Table: KYLIN_HRDJ16B3O8 Region Count: 1 Size: 179 MB Segment: 20181010110000_20181010120000 Start Time: 2018-10-10 11:00:00 End Time: 2018-10-10 12:00:00 Source Count: 849363 HBase Table: KYLIN_6BFHFA5LU1 Region Count: 1 Size: 178 MB Segment: 20181010120000_20181010130000 Start Time: 2018-10-10 12:00:00 End Time: 2018-10-10 13:00:00 Source Count: 851162 HBase Table: KYLIN_H41KZXUIRN Region Count: 1 Size: 177 MB Segment: 20181010130000_20181010140000 Start Time: 2018-10-10 13:00:00 End Time: 2018-10-10 14:00:00 Source Count: 836481 HBase Table: KYLIN_8RXPI7T0PA Region Count: 1 Size: 173 MB Segment: 20181010140000_20181010150000 Start Time: 2018-10-10 14:00:00 End Time: 2018-10-10 15:00:00 Source Count: 780337 HBase Table: KYLIN_7L3WHR3ZQY Region Count: 1 Size: 164 MB Segment: 20181010150000_20181010160000 Start Time: 2018-10-10 15:00:00 End Time: 2018-10-10 16:00:00 Source Count: 723669 HBase Table: KYLIN_RM0ICHV5EP Region Count: 1 Size: 155 MB Segment: 20181010160000_20181011170000 Start Time: 2018-10-10 16:00:00 End Time: 2018-10-11 17:00:00 Source Count: 17476745 HBase Table: KYLIN_Y8ZOSIWNJP Region Count: 1 Size: 941 MB Segment: 20181011170000_20181012170000 Start Time: 2018-10-11 17:00:00 End Time: 2018-10-12 17:00:00 Source Count: 15485276 HBase Table: KYLIN_RWTQZFY6J4 Region Count: 1 Size: 887 MB Thanks, Ketan@Exponential