[
https://issues.apache.org/jira/browse/HIVE-24706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422021#comment-17422021
]
cadl commented on HIVE-24706:
-----------------------------
I meet the issue too. After tracking the stacktrace, there are two problems
that cause the issue.
# The `org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat` not implements
`org.apache.hadoop.mapreduce.InputFormat` completely. As [~Lysak] said,
`HiveHBaseTableInputFormat` doesn't overwrite `getSplits(JobContext context)`
and `createRecordReader`, and doesn't initialize the table.
# Because of extending the
`org.apache.hadoop.hbase.mapreduce.TableInputFormatBase extends
InputFormat<ImmutableBytesWritable, Result>`, `HiveHBaseTableInputFormat` can't
cast to `InputFormat<Writable, Writable>` at [spark
createNewHadoopRDD|https://github.com/apache/spark/blob/35848385ae6518b4b72c2f5c1e9ca5a83a190723/sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala#L373]
Is it safe to change `HiveHBaseTableInputFormat` from
`InputFormat<ImmutableBytesWritable, Result>` to
`InputFormat<ImmutableBytesWritable, ResultWritable>`?
I'd like to submit a pull request about it.
> Spark SQL access hive on HBase table access exception
> -----------------------------------------------------
>
> Key: HIVE-24706
> URL: https://issues.apache.org/jira/browse/HIVE-24706
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Reporter: zhangzhanchang
> Priority: Major
> Attachments: image-2021-01-30-15-51-58-665.png
>
>
> Hivehbasetableinputformat relies on two versions of inputformat，one is
> org.apache.hadoop.mapred.InputFormat, the other is
> org.apache.hadoop.mapreduce.InputFormat,Causes
> spark 3.0(https://github.com/apache/spark/pull/31302) both conditions to be
> true:
> # classOf[oldInputClass[_, _]].isAssignableFrom(inputFormatClazz) is true
> # classOf[newInputClass[_, _]].isAssignableFrom(inputFormatClazz) is true
> !image-2021-01-30-15-51-58-665.png|width=430,height=137!
> Hivehbasetableinputformat relies on inputformat to be changed to
> org.apache.hadoop.mapreduce or org.apache.hadoop.mapred?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
