[
https://issues.apache.org/jira/browse/KYLIN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17434068#comment-17434068
]
mzz commented on KYLIN-5107:
----------------------------
[~zhangyaqian]
The version of hbase back up from 2.3.4 to 1.4.9, it's not reporting a similar
error.
> when excuting the cube job at "Convert Cuboid Data to HFile" step：Error
> running child
> --------------------------------------------------------------------------------------
>
> Key: KYLIN-5107
> URL: https://issues.apache.org/jira/browse/KYLIN-5107
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v3.1.2
> Environment: hadoop3.2.1
> habse2.3.4
> Reporter: mzz
> Priority: Blocker
>
> As title, when excuting the cube job at "Convert Cuboid Data to HFile"
> step,throw an exception:
> {code:java}
> 2021-10-21 10:15:38,928 ERROR [main] org.apache.kylin.engine.mr.KylinReducer:
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer:
> Do cleanup, available memory: 2773m
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer:
> Total rows: 1
> 2021-10-21 10:15:38,929 ERROR [main] org.apache.hadoop.mapred.YarnChild:
> Error running child : java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {code}
> I try move all of hbase libs to {KYLIN_HOME}/lib ,but this problem not
> solved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
