Hi Ketan,

You're welcome. JIRA is created for this issue:
https://issues.apache.org/jira/browse/KYLIN-3652

kdcool6932 <kdcool6...@yahoo.com.invalid> 于2018年10月30日周二 上午10:49写道:

> Thanks, ShaoFeng,That's just what we need.I will use 'to-add' as
> reference, along with the online 'howto', that explains it
> all.Thanks,ke...@exponential.com Sent from my Samsung Galaxy smartphone.
> -------- Original message --------From: ShaoFeng Shi <
> shaofeng...@apache.org> Date: 30/10/2018  7:49 am  (GMT+05:30) To: dev <
> dev@kylin.apache.org> Subject: Re: Merge Job in inconsistent state I'm
> updating the document for modifying metadata in a selective way, butthe
> jenkins has a problem today. Here is the to-add part inhttps://
> kylin.apache.org/docs/howto/howto_backup_metadata.html:##
> <http://kylin.apache.org/docs/howto/howto_backup_metadata.html:#%23>
> Restore metadata selectively (Recommended)If only changes a couple of
> metadata files, the administrator can just pickthese files to restore,
> without having to cover all the metadata. Comparedto the full recovery,
> this approach is more efficient, safer, so it isrecommended.Create a new
> empty directory, and then create subdirectories in itaccording to the
> location of the metadata files to restore; for example, torestore a Cube
> instance, you should create a "cube" subdirectory:{% highlight Groff markup
> %}mkdir /path/to/restore_newmkdir /path/to/restore_new/cube{% endhighlight
> %}Copy the metadata file to be restored to this new directory:{% highlight
> Groff markup %}cp
> meta_backups/meta_2016_06_10_20_24_50/cube/kylin_sales_cube.json/path/to/restore_new/cube/{%
> endhighlight %}At this point, you can modify/fix the metadata
> manually.Restore from this directory:{% highlight Groff markup %}cd
> $KYLIN_HOME./bin/metastore.sh restore /path/to/restore_new{% endhighlight
> %}Only the files in the folder will be uploaded to Kylin
> metastore.Similarly, after the recovery is finished, click Reload Metadata
> button onthe Web UI to flush the cache.kdcool6932
> <kdcool6...@yahoo.com.invalid> 于2018年10月29日周一 下午7:54写道:> Thanks guys,,
> really appreciate the prompt response.@ShaoFeng,Yes we have> the data in
> hive(or we can load that if needed). And we will be rebuilding> those
> segments. Do we have any mail thread or document or blog to refer to> for
> manully editing and restoring metadata. That would actually be great> help,
> as we often get into situations like this as we don't want to take> risk
> for 3 plus years (120+TB) data in Hbase for Kylin.Again,, really>
> appreciate the help provided.Thanks,Ketan Sent from my Samsung Galaxy>
> smartphone.> -------- Original message --------From: ShaoFeng Shi <>
> shaofeng...@apache.org> Date: 29/10/2018  12:36 pm  (GMT+05:30) To: dev
> <> dev@kylin.apache.org> Subject: Re: Merge Job in inconsistent state It
> is> a known issue; the auto-merge was triggered on each segment
> change.Maybe,> Kylin should not trigger the auto-merge on
> canceling/deleting> ajob/segment?But you can keep that error job/segment,
> it won't impact on> the query. Theonly thing is an error job there.Do you
> know the root cause> of  "No input paths specified in job"? Did youdelete
> some folders from> HDFS?If you have the source data in Hive, you can
> rebuild those segments;> Youcan take a backup of the metadata, and then
> dump metadata in local> disk,copy that cube's json to a clean folder, edit
> it to delete these> segment,and then restore metadata from the clean folder
> (same structure,> e.g/cube/yourcube.json). After restored, build the
> segments for the> missingdate range.Chao Long <wayn...@qq.com>
> 于2018年10月29日周一 下午2:12写道:> Hi> Ketan,>    As this merge job is an
> automatically triggered job, so it start> again> when you discard it. If
> you don't want this job to be triggered> again, you> can remove the "Auto
> Merge" related configuration on the cube> design page> until the problem is
> resolved or fixed(if it's a bug).>>>> This is the merging
> Segment[20181005080000_20181012170000]. And the>> exception occurred during
> the merging job, so it's in an incorrect> state.>    Segment:
> 20181005080000_20181012170000 //**This segment Table> was> deleted from
> Hbase (somehow, we don’t have the reason)**//>    Start> Time: 2018-10-05
> 08:00:00>    End Time: 2018-10-12 17:00:00>    Source> Count: 0>    HBase
> Table: KYLIN_CFLY2CKMCU>    Region Count: 3>    Size:> less than 1 MB>>>
> To identify the root cause, you may provide more log> around the error>
> message.>>>>>> ------------------ 原始邮件> ------------------> 发件人: "ketan
> dikshit"<kdcool6...@yahoo.com.INVALID>;>> 发送时间: 2018年10月28日(星期天) 凌晨0:25>
> 收件人: "dev"<dev@kylin.apache.org>;>> 主题:> Merge Job in inconsistent
> state>>>> Hi Team,> We are using Kylin 2.3.1, And> in the merge Job(which
> gets automatically> triggered), we are getting this> error in Merge Cuboid
> Data Step;>> java.io.IOException: No input paths> specified in job>
> at>>
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:239)>>
> at>>
> org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat.listStatus(SequenceFileInputFormat.java:59)>>
> at>>
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)>>
> at org.apache.kylin.engine.mr>>
> .common.AbstractHadoopJob.getTotalMapInputMB(AbstractHadoopJob.java:622)>>
> at>>
> org.apache.kylin.storage.hbase.steps.HBaseMROutput2Transition$HBaseMergeMROutputFormat.configureJobOutput(HBaseMROutput2Transition.java:166)>>
> at org.apache.kylin.engine.mr>>
> .steps.MergeCuboidJob.run(MergeCuboidJob.java:82)>         at>
> org.apache.kylin.engine.mr>>
> .common.MapReduceExecutable.doWork(MapReduceExecutable.java:130)>> at>>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)>>
> at>>
> org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:67)>>
> at>>
> org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:162)>>
> at>>
> org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:300)>>
> at>>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)>>
> at>>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)>>
> at java.lang.Thread.run(Thread.java:745)>> result code:2>> As it is a
> merge> job, whenever I discard this job,, it again starts> automatically.>
> So we> need a way(if there is one), of permanently discarding this job>
> somehow.>>> Also in case this can be done by changing the metadata for the
> cube>> segment, do let me know how it can be done(even if I have to
> delete/remove>> the segments for these below days)> We are open for all
> options (except> dropping the cube as it has more than> 1 year data, and is
> crucial for> organisation). We are stuck on this for> some days now, help
> would really> be appreciated.>> Merge Thresholds:> 8 (Hours)> 1 (Days)> 7
> (Days)> 15> (Days)>> Merge Job, (start and end):> MERGE CUBE - XXXX ->
> 20181005080000_20181012170000>> Here are list of segments;> Segment:>
> 20181005080000_20181006080000> Start Time: 2018-10-05 08:00:00> End Time:>
> 2018-10-06 08:00:00> Source Count: 14899048> HBase Table:
> KYLIN_R1MUK56K71>> Region Count: 1> Size: 860 MB>> Segment:
> 20181005080000_20181012170000> //**This segment Table was deleted> from
> Hbase (somehow, we don’t have the> reason)**//> Start Time: 2018-10-05
> 08:00:00> End Time: 2018-10-12> 17:00:00> Source Count: 0> HBase Table:
> KYLIN_CFLY2CKMCU> Region Count: 3>> Size: less than 1 MB>> Segment:
> 20181006080000_20181008000000> Start Time:> 2018-10-06 08:00:00> End Time:
> 2018-10-08 00:00:00> Source Count: 24455686>> HBase Table:
> KYLIN_0KH6PHTEM2> Region Count: 1> Size: 1.0498 GB>> Segment:>
> 20181008000000_20181009000000> Start Time: 2018-10-08 00:00:00> End Time:>
> 2018-10-09 00:00:00> Source Count: 14882090> HBase Table:
> KYLIN_V1CC4LDSIR>> Region Count: 1> Size: 598 MB>> Segment:
> 20181009000000_20181010000000>> Start Time: 2018-10-09 00:00:00> End Time:
> 2018-10-10 00:00:00> Source> Count: 16245847> HBase Table:
> KYLIN_4A44K2VJEU> Region Count: 1> Size: 628> MB>> Segment:
> 20181010000000_20181010080000> Start Time: 2018-10-10> 00:00:00> End Time:
> 2018-10-10 08:00:00> Source Count: 5213022> HBase> Table: KYLIN_EHO316VC7M>
> Region Count: 1> Size: 397 MB>> Segment:> 20181010080000_20181010090000>
> Start Time: 2018-10-10 08:00:00> End Time:> 2018-10-10 09:00:00> Source
> Count: 865722> HBase Table: KYLIN_I9LEJ2JDZ8>> Region Count: 1> Size: 181
> MB>> Segment: 20181010090000_20181010100000>> Start Time: 2018-10-10
> 09:00:00> End Time: 2018-10-10 10:00:00> Source> Count: 859127> HBase
> Table: KYLIN_9IBX3W4UNL> Region Count: 1> Size: 180> MB>> Segment:
> 20181010100000_20181010110000> Start Time: 2018-10-10> 10:00:00> End Time:
> 2018-10-10 11:00:00> Source Count: 855752> HBase Table:> KYLIN_HRDJ16B3O8>
> Region Count: 1> Size: 179 MB>> Segment:> 20181010110000_20181010120000>
> Start Time: 2018-10-10 11:00:00> End Time:> 2018-10-10 12:00:00> Source
> Count: 849363> HBase Table: KYLIN_6BFHFA5LU1>> Region Count: 1> Size: 178
> MB>> Segment: 20181010120000_20181010130000>> Start Time: 2018-10-10
> 12:00:00> End Time: 2018-10-10 13:00:00> Source> Count: 851162> HBase
> Table: KYLIN_H41KZXUIRN> Region Count: 1> Size: 177> MB>> Segment:
> 20181010130000_20181010140000> Start Time: 2018-10-10> 13:00:00> End Time:
> 2018-10-10 14:00:00> Source Count: 836481> HBase Table:> KYLIN_8RXPI7T0PA>
> Region Count: 1> Size: 173 MB>> Segment:> 20181010140000_20181010150000>
> Start Time: 2018-10-10 14:00:00> End Time:> 2018-10-10 15:00:00> Source
> Count: 780337> HBase Table: KYLIN_7L3WHR3ZQY>> Region Count: 1> Size: 164
> MB>> Segment: 20181010150000_20181010160000>> Start Time: 2018-10-10
> 15:00:00> End Time: 2018-10-10 16:00:00> Source> Count: 723669> HBase
> Table: KYLIN_RM0ICHV5EP> Region Count: 1> Size: 155> MB>> Segment:
> 20181010160000_20181011170000> Start Time: 2018-10-10> 16:00:00> End Time:
> 2018-10-11 17:00:00> Source Count: 17476745> HBase> Table:
> KYLIN_Y8ZOSIWNJP> Region Count: 1> Size: 941 MB>> Segment:>
> 20181011170000_20181012170000> Start Time: 2018-10-11 17:00:00> End Time:>
> 2018-10-12 17:00:00> Source Count: 15485276> HBase Table:
> KYLIN_RWTQZFY6J4>> Region Count: 1> Size: 887 MB>> Thanks,>
> Ketan@Exponential-- Best> regards,Shaofeng Shi 史少锋-- Best
> regards,Shaofeng Shi 史少锋



-- 
Best regards,

Shaofeng Shi 史少锋

Reply via email to