Hi Xiaoxiang,
Thanks for your advices, I will follow them.
Thank you for your help again.
Best Regards,
Shylin Zhang
On 02/18/2020 23:35，Xiaoxiang Yu<[email protected]> wrote：
Hi shylinzhang,
Thank you for your update. I am glad to hear you have fixed your problem
finally.
I wish you can attach the some detailed description or some error log if you
face some problem in the future; using the simple/misunderstanding sentence "it
failed' is not good enough in my side.
Here is some advice provided by apache website:
1. http://www.catb.org/~esr/faqs/smart-questions.html,
2. http://www.apache.org/dev/contrib-email-tips .
--
Best wishes to you !
From ：Xiaoxiang Yu
在 2020-02-18 19:43:15，"shylinzhang" <[email protected]> 写道：
Hi Xiaoxiang,
I’m sorry that I misunderstood the meaning of “delete the remaining
marker”.I followed what you said and solved one system cube issue;
But I fix anther same issued system cube with same way, it failed，And then I
purge and rebuild the cube and it works well.
Thank you for your help.
Best Wishes!
Shylin Zhang
| |
shylinzhang
|
|
[email protected]
|
签名由网易邮箱大师定制
On 02/17/2020 19:45，Xiaoxiang Yu<[email protected]> wrote：
Hi,
I don't understand what problem did you face in "I tried to delete the
remaining marker, it failed"?
As far as I see, you can open the hbase shell. Use the "get
/dict/KYLIN.HIVE_METRICS_QUERY_CUBE_QA/CUBE_NAME/xxx.dict" to get the value of
that entry,
and use "delete" to delete that empty entry. Before doing this, I think you
should back up your metadata.
--
Best wishes to you !
From ：Xiaoxiang Yu
在 2020-02-17 10:44:23，"shylinzhang" <[email protected]> 写道：
Hi George,
Thank you for your reply, I tried to delete the remaining marker, it
failed. So I had to update.
Thank you!
Best Wishes!
Shylin Zhang
| |
shylinzhang
|
|
[email protected]
|
签名由网易邮箱大师定制
On 02/16/2020 21:05，nichunen<[email protected]> wrote：
Hi Shylin,
I think the description part in
https://issues.apache.org/jira/browse/KYLIN-4153 is clear enough, you can find
the root cause and workaround there, it happened due to the lack of atomicity
during the rollback for the "big resource".
As a workaround, you may delete the remaining marker in your resource store,
but it’s more advised to do an upgrade for your Kylin.
Best regards,
Ni Chunen / George
On 02/16/2020 16:22，shylinzhang<[email protected]> wrote：
Dear all,
The cube is ok when 2:00 am,but after 4:30 am it always shows error ,it shows
``` #4 Step Name: Build Dimension Dictionary
org.apache.kylin.engine.mr.exception.HadoopShellException:
java.lang.RuntimeException: java.io.IOException: Failed to read big resource
/dict/KYLIN.HIVE_METRICS_QUERY_CUBE_QA/CUBE_NAME/b9bb85ec-b72e-4df1-9ee7-e8c971ec5c72.dict
at
org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:108)
at
org.apache.kylin.dict.DictionaryManager.checkDupByContent(DictionaryManager.java:173)
at
org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.java:151)
at
org.apache.kylin.dict.DictionaryManager.saveDictionary(DictionaryManager.java:320)
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.saveDictionary(CubeManager.java:1117)
at org.apache.kylin.cube.CubeManager.saveDictionary(CubeManager.java:1079)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:74)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:55)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
What I do:
1. I disable the cube and purge the cube and rebulid, it also show the same
error;
2. Because the cube at 2:00 am is ok, so I use the meta_backs in 2:00 am and
restore and cube; I try short period to build the cube, 2:00
am~3:00am~4:00am~4:00am~6:00am are ok, but in 6:am~7:00 am it also shows above
error;
I found the solution in kylin jira, it tells me to update to release 2.6.4; our
kylin clusters are used in 2019-06; just today shows the error.
Here’s the jira url:
https://issues.apache.org/jira/browse/KYLIN-4153?jql=summary%20~%20%22Failed%20to%20read%20big%20resource*%22%20OR%20description%20~%20%22Failed%20to%20read%20big%20resource*%22%20ORDER%20BY%20lastViewed%20DESC
Therefore I want to know why it occured and if there is a way to fix it except
updating.
I am looking forward for your replay, thank you.
Kylin version: 2.6.1
Hadoop version: 3.0.0+cdh6.0.0
Best Wishes!
Shylin Zhang
| |
shylinzhang
|
|
[email protected]
|
签名由网易邮箱大师定制
