[
https://issues.apache.org/jira/browse/HIVE-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150686#comment-15150686
]
Yongzhi Chen commented on HIVE-13065:
-------------------------------------
[~Yibing], the reading part works fine, in my patch, there is a select query in
the test which list the insert results. The null works fine. The "skip null
value and its separator" is a way to tell hbase the value is null.
This jira is only handle null values.
Without any fix, the null key case does not throw exception. But the return
value looks strange. For example the map {null, 'value'} becomes {} . I think
hive and hbase do not support null key yet.
> Hive throws NPE when writing map type data to a HBase backed table
> ------------------------------------------------------------------
>
> Key: HIVE-13065
> URL: https://issues.apache.org/jira/browse/HIVE-13065
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
> Affects Versions: 1.1.0, 2.0.0
> Reporter: Yongzhi Chen
> Assignee: Yongzhi Chen
> Attachments: HIVE-13065.1.patch
>
>
> Hive throws NPE when writing data to a HBase backed table with below
> conditions:
> # There is a map type column
> # The map type column has NULL in its values
> Below are the reproduce steps:
> *1) Create a HBase backed Hive table*
> {code:sql}
> create table hbase_test (id bigint, data map<string, string>)
> stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> with serdeproperties ("hbase.columns.mapping" = ":key,cf:map_col")
> tblproperties ("hbase.table.name" = "hive_test");
> {code}
> *2) insert data into above table*
> {code:sql}
> insert overwrite table hbase_test select 1 as id, map('abcd', null) as data
> from src limit 1;
> {code}
> The mapreduce job for insert query fails. Error messages are as below:
> {noformat}
> 2016-02-15 02:26:33,225 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row (tag=0) {"key":{},"value":{"_col0":1,"_col1":{"abcd":null}}}
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:265)
> at
> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row (tag=0)
> {"key":{},"value":{"_col0":1,"_col1":{"abcd":null}}}
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:253)
> ... 7 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.serde2.SerDeException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:731)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.LimitOperator.processOp(LimitOperator.java:51)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:244)
> ... 7 more
> Caused by: org.apache.hadoop.hive.serde2.SerDeException:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:286)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:666)
> ... 14 more
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:221)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:236)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:275)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:222)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serializeField(HBaseRowSerializer.java:194)
> at
> org.apache.hadoop.hive.hbase.HBaseRowSerializer.serialize(HBaseRowSerializer.java:118)
> at
> org.apache.hadoop.hive.hbase.HBaseSerDe.serialize(HBaseSerDe.java:282)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
