[
https://issues.apache.org/jira/browse/HIVE-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16369718#comment-16369718
]
Dave Latham edited comment on HIVE-8808 at 2/20/18 5:02 AM:
------------------------------------------------------------
Bumped into this problem with another input format that wrapped an hbase input
format. The wrapper did not have hbase in its name.
I've gotta ask, though: What's the purpose of caching a stateless object?
If M/R supports calling setConf as part of the instantiation system, that seems
intended to initialize state.
Other examples of stateful input formats broken by the caching:
[https://github.com/cloudera/kudu/blob/master/java/kudu-mapreduce/src/main/java/org/apache/kudu/mapreduce/KuduTableInputFormat.java]
[https://github.com/twitter/elephant-bird/blob/master/core/src/main/java/com/twitter/elephantbird/mapred/input/DeprecatedInputFormatWrapper.java]
[https://github.com/hanborq/hadoop/blob/master/src/mapred/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.java]
was (Author: davelatham):
Bumped into this problem with another input format that wrapped an hbase input
format. The wrapper did not have hbase in its name.
I've gotta ask, though: What's the purpose of caching a stateless object?
If M/R supports calling setConf as part of the instantiation system, that seems
intended to initialize state.
> HiveInputFormat caching cannot work with all input formats
> ----------------------------------------------------------
>
> Key: HIVE-8808
> URL: https://issues.apache.org/jira/browse/HIVE-8808
> Project: Hive
> Issue Type: Bug
> Reporter: Brock Noland
> Assignee: Brock Noland
> Priority: Major
> Fix For: 1.1.0
>
> Attachments: HIVE-8808.patch, HIVE-8808.patch, HIVE-8808.patch
>
>
> In {{HiveInputFormat}} we implement instance caching (see
> {{getInputFormatFromCache}}). In HS2, this assumes that InputFormats are
> stateless but I don't think this assumption is true, especially with regards
> to HBase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
