Thanks guys,, really appreciate the prompt response.@ShaoFeng,Yes we have the 
data in hive(or we can load that if needed). And we will be rebuilding those 
segments. Do we have any mail thread or document or blog to refer to for 
manully editing and restoring metadata. That would actually be great help, as 
we often get into situations like this as we don't want to take risk for 3 plus 
years (120+TB) data in Hbase for Kylin.Again,, really appreciate the help 
provided.Thanks,Ketan Sent from my Samsung Galaxy smartphone.
-------- Original message --------From: ShaoFeng Shi <shaofeng...@apache.org> 
Date: 29/10/2018  12:36 pm  (GMT+05:30) To: dev <dev@kylin.apache.org> Subject: 
Re: Merge Job in inconsistent state It is a known issue; the auto-merge was 
triggered on each segment change.Maybe, Kylin should not trigger the auto-merge 
on canceling/deleting ajob/segment?But you can keep that error job/segment, it 
won't impact on the query. Theonly thing is an error job there.Do you know the 
root cause of  "No input paths specified in job"? Did youdelete some folders 
from HDFS?If you have the source data in Hive, you can rebuild those segments; 
Youcan take a backup of the metadata, and then dump metadata in local disk,copy 
that cube's json to a clean folder, edit it to delete these segment,and then 
restore metadata from the clean folder (same structure, 
e.g/cube/yourcube.json). After restored, build the segments for the missingdate 
range.Chao Long <wayn...@qq.com> 于2018年10月29日周一 下午2:12写道:> Hi Ketan,>    As 
this merge job is an automatically triggered job, so it start again> when you 
discard it. If you don't want this job to be triggered again, you> can remove 
the "Auto Merge" related configuration on the cube design page> until the 
problem is resolved or fixed(if it's a bug).>>>    This is the merging 
Segment[20181005080000_20181012170000]. And the> exception occurred during the 
merging job, so it's in an incorrect state.>    Segment: 
20181005080000_20181012170000 //**This segment Table was> deleted from Hbase 
(somehow, we don’t have the reason)**//>    Start Time: 2018-10-05 08:00:00>    
End Time: 2018-10-12 17:00:00>    Source Count: 0>    HBase Table: 
KYLIN_CFLY2CKMCU>    Region Count: 3>    Size: less than 1 MB>>>    To identify 
the root cause, you may provide more log around the error> message.>>>>>> 
------------------ 原始邮件 ------------------> 发件人: "ketan 
dikshit"<kdcool6...@yahoo.com.INVALID>;> 发送时间: 2018年10月28日(星期天) 凌晨0:25> 收件人: 
"dev"<dev@kylin.apache.org>;>> 主题: Merge Job in inconsistent state>>>> Hi 
Team,> We are using Kylin 2.3.1, And in the merge Job(which gets automatically> 
triggered), we are getting this error in Merge Cuboid Data Step;>> 
java.io.IOException: No input paths specified in job>         at> 
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:239)>
         at> 
org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)>
         at> 
org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)>
         at org.apache.kylin.engine.mr> 
.common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:622)>       
  at> 
org.apache.kylin.storage.hbase.steps.HBaseMROutput2Transition$HBaseMergeMROutputFormat.configureJobOutput(HBaseMROutput2Transition.java:166)>
         at org.apache.kylin.engine.mr> 
.steps.MergeCuboidJob.run(MergeCuboidJob.java:82)>         at 
org.apache.kylin.engine.mr> 
.common.MapReduceExecutable.doWork(MapReduceExecutable.java:130)>         at> 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)>
         at> 
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)>
         at> 
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)>
         at> 
org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)>
         at> 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)>
         at> 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)>
         at java.lang.Thread.run(Thread.java:745)>> result code:2>> As it is a 
merge job, whenever I discard this job,, it again starts> automatically.> So we 
need a way(if there is one), of permanently discarding this job> somehow.>> 
Also in case this can be done by changing the metadata for the cube> segment, 
do let me know how it can be done(even if I have to delete/remove> the segments 
for these below days)> We are open for all options (except dropping the cube as 
it has more than> 1 year data, and is crucial for organisation). We are stuck 
on this for> some days now, help would really be appreciated.>> Merge 
Thresholds:> 8 (Hours)> 1 (Days)> 7 (Days)> 15 (Days)>> Merge Job, (start and 
end):> MERGE CUBE - XXXX - 20181005080000_20181012170000>> Here are list of 
segments;> Segment: 20181005080000_20181006080000> Start Time: 2018-10-05 
08:00:00> End Time: 2018-10-06 08:00:00> Source Count: 14899048> HBase Table: 
KYLIN_R1MUK56K71> Region Count: 1> Size: 860 MB>> Segment: 
20181005080000_20181012170000 //**This segment Table was deleted> from Hbase 
(somehow, we don’t have the reason)**//> Start Time: 2018-10-05 08:00:00> End 
Time: 2018-10-12 17:00:00> Source Count: 0> HBase Table: KYLIN_CFLY2CKMCU> 
Region Count: 3> Size: less than 1 MB>> Segment: 20181006080000_20181008000000> 
Start Time: 2018-10-06 08:00:00> End Time: 2018-10-08 00:00:00> Source Count: 
24455686> HBase Table: KYLIN_0KH6PHTEM2> Region Count: 1> Size: 1.0498 GB>> 
Segment: 20181008000000_20181009000000> Start Time: 2018-10-08 00:00:00> End 
Time: 2018-10-09 00:00:00> Source Count: 14882090> HBase Table: 
KYLIN_V1CC4LDSIR> Region Count: 1> Size: 598 MB>> Segment: 
20181009000000_20181010000000> Start Time: 2018-10-09 00:00:00> End Time: 
2018-10-10 00:00:00> Source Count: 16245847> HBase Table: KYLIN_4A44K2VJEU> 
Region Count: 1> Size: 628 MB>> Segment: 20181010000000_20181010080000> Start 
Time: 2018-10-10 00:00:00> End Time: 2018-10-10 08:00:00> Source Count: 
5213022> HBase Table: KYLIN_EHO316VC7M> Region Count: 1> Size: 397 MB>> 
Segment: 20181010080000_20181010090000> Start Time: 2018-10-10 08:00:00> End 
Time: 2018-10-10 09:00:00> Source Count: 865722> HBase Table: KYLIN_I9LEJ2JDZ8> 
Region Count: 1> Size: 181 MB>> Segment: 20181010090000_20181010100000> Start 
Time: 2018-10-10 09:00:00> End Time: 2018-10-10 10:00:00> Source Count: 859127> 
HBase Table: KYLIN_9IBX3W4UNL> Region Count: 1> Size: 180 MB>> Segment: 
20181010100000_20181010110000> Start Time: 2018-10-10 10:00:00> End Time: 
2018-10-10 11:00:00> Source Count: 855752> HBase Table: KYLIN_HRDJ16B3O8> 
Region Count: 1> Size: 179 MB>> Segment: 20181010110000_20181010120000> Start 
Time: 2018-10-10 11:00:00> End Time: 2018-10-10 12:00:00> Source Count: 849363> 
HBase Table: KYLIN_6BFHFA5LU1> Region Count: 1> Size: 178 MB>> Segment: 
20181010120000_20181010130000> Start Time: 2018-10-10 12:00:00> End Time: 
2018-10-10 13:00:00> Source Count: 851162> HBase Table: KYLIN_H41KZXUIRN> 
Region Count: 1> Size: 177 MB>> Segment: 20181010130000_20181010140000> Start 
Time: 2018-10-10 13:00:00> End Time: 2018-10-10 14:00:00> Source Count: 836481> 
HBase Table: KYLIN_8RXPI7T0PA> Region Count: 1> Size: 173 MB>> Segment: 
20181010140000_20181010150000> Start Time: 2018-10-10 14:00:00> End Time: 
2018-10-10 15:00:00> Source Count: 780337> HBase Table: KYLIN_7L3WHR3ZQY> 
Region Count: 1> Size: 164 MB>> Segment: 20181010150000_20181010160000> Start 
Time: 2018-10-10 15:00:00> End Time: 2018-10-10 16:00:00> Source Count: 723669> 
HBase Table: KYLIN_RM0ICHV5EP> Region Count: 1> Size: 155 MB>> Segment: 
20181010160000_20181011170000> Start Time: 2018-10-10 16:00:00> End Time: 
2018-10-11 17:00:00> Source Count: 17476745> HBase Table: KYLIN_Y8ZOSIWNJP> 
Region Count: 1> Size: 941 MB>> Segment: 20181011170000_20181012170000> Start 
Time: 2018-10-11 17:00:00> End Time: 2018-10-12 17:00:00> Source Count: 
15485276> HBase Table: KYLIN_RWTQZFY6J4> Region Count: 1> Size: 887 MB>> 
Thanks,> Ketan@Exponential-- Best regards,Shaofeng Shi 史少锋

Reply via email to