[
https://issues.apache.org/jira/browse/KYLIN-5107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17433535#comment-17433535
]
Yaqian Zhang commented on KYLIN-5107:
-------------------------------------
Hi:
Is the kylin binary package you downloaded for Hadoop 3?
It seems to be caused by some kind of version conflict,I think it may be that
HBase 2.3.4 is too high for kylin to be compatible. You can try using a lower
version of HBase.
Or you can use kylin4.0 to avoid HBase dependency.
> when excuting the cube job at "Convert Cuboid Data to HFile" step：Error
> running child
> --------------------------------------------------------------------------------------
>
> Key: KYLIN-5107
> URL: https://issues.apache.org/jira/browse/KYLIN-5107
> Project: Kylin
> Issue Type: Bug
> Components: Job Engine
> Affects Versions: v3.1.2
> Environment: hadoop3.2.1
> habse2.3.4
> Reporter: mzz
> Priority: Blocker
>
> As title, when excuting the cube job at "Convert Cuboid Data to HFile"
> step,throw an exception:
> {code:java}
> 2021-10-21 10:15:38,928 ERROR [main] org.apache.kylin.engine.mr.KylinReducer:
> java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer:
> Do cleanup, available memory: 2773m
> 2021-10-21 10:15:38,929 INFO [main] org.apache.kylin.engine.mr.KylinReducer:
> Total rows: 1
> 2021-10-21 10:15:38,929 ERROR [main] org.apache.hadoop.mapred.YarnChild:
> Error running child : java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.util.FSUtils.setStoragePolicy(Lorg/apache/hadoop/fs/FileSystem;Lorg/apache/hadoop/fs/Path;Ljava/lang/String;)V
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3.configureStoragePolicy(HFileOutputFormat3.java:468)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:287)
> at
> org.apache.kylin.storage.hbase.steps.HFileOutputFormat3$1.write(HFileOutputFormat3.java:243)
> at
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:559)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:141)
> at
> org.apache.kylin.storage.hbase.steps.CubeHFileJob$KeyValueReducer.doReduce(CubeHFileJob.java:135)
> at org.apache.kylin.engine.mr.KylinReducer.reduce(KylinReducer.java:77)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
> at
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1732)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {code}
> I try move all of hbase libs to {KYLIN_HOME}/lib ,but this problem not
> solved.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
