Dear all,
The cube is ok when 2:00 am,but after 4:30 am it always shows error ,it
shows ``` #4 Step Name: Build Dimension Dictionary
org.apache.kylin.engine.mr.exception.HadoopShellException:
java.lang.RuntimeException: java.io.IOException: Failed to read big resource
/dict/KYLIN.HIVE_METRICS_QUERY_CUBE_QA/CUBE_NAME/b9bb85ec-b72e-4df1-9ee7-e8c971ec5c72.dict
at
org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(DictionaryManager.java:108)
at
org.apache.kylin.dict.DictionaryManager.checkDupByContent(DictionaryManager.java:173)
at
org.apache.kylin.dict.DictionaryManager.trySaveNewDict(DictionaryManager.java:151)
at
org.apache.kylin.dict.DictionaryManager.saveDictionary(DictionaryManager.java:320)
at
org.apache.kylin.cube.CubeManager$DictionaryAssist.saveDictionary(CubeManager.java:1117)
at
org.apache.kylin.cube.CubeManager.saveDictionary(CubeManager.java:1079)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:74)
at
org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(DictionaryGeneratorCLI.java:55)
at
org.apache.kylin.engine.mr.steps.CreateDictionaryJob.run(CreateDictionaryJob.java:73)
at org.apache.kylin.engine.mr.MRUtil.runMRJob(MRUtil.java:93)
at
org.apache.kylin.engine.mr.common.HadoopShellExecutable.doWork(HadoopShellExecutable.java:63)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:71)
at
org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:166)
at
org.apache.kylin.job.impl.threadpool.DistributedScheduler$JobRunner.run(DistributedScheduler.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
```
What I do:
1. I disable the cube and purge the cube and rebulid, it also show the same
error;
2. Because the cube at 2:00 am is ok, so I use the meta_backs in 2:00 am and
restore and cube; I try short period to build the cube, 2:00
am~3:00am~4:00am~4:00am~6:00am are ok, but in 6:am~7:00 am it also shows above
error;
I found the solution in kylin jira, it tells me to update to release 2.6.4; our
kylin clusters are used in 2019-06; just today shows the error.
Here’s the jira url:
https://issues.apache.org/jira/browse/KYLIN-4153?jql=summary%20~%20%22Failed%20to%20read%20big%20resource*%22%20OR%20description%20~%20%22Failed%20to%20read%20big%20resource*%22%20ORDER%20BY%20lastViewed%20DESC
Therefore I want to know why it occured and if there is a way to fix it except
updating.
I am looking forward for your replay, thank you.
Kylin version: 2.6.1
Hadoop version: 3.0.0+cdh6.0.0
Best Wishes!
Shylin Zhang
| |
shylinzhang
|
|
[email protected]
|
签名由网易邮箱大师定制
