[
https://issues.apache.org/jira/browse/IMPALA-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongjun Zhang resolved IMPALA-7929.
-----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.2.0
Many thanks to [~joemcdonnell], [~tgar] and [~paul-rogers] for the review and
commit!
> Impala query on HBASE table failing with InternalException: Required field*
> ---------------------------------------------------------------------------
>
> Key: IMPALA-7929
> URL: https://issues.apache.org/jira/browse/IMPALA-7929
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.2.0
> Reporter: Yongjun Zhang
> Assignee: Yongjun Zhang
> Priority: Major
> Fix For: Impala 3.2.0
>
>
> This looks a corner case bug demonstrated at impala-hbase boundary.
> The way to reproduce:
> Create a table in hive shell,
> {code}
> create database abc;
> CREATE TABLE abc.test_hbase1 (k STRING, c STRING) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('
> hbase.columns.mapping'=':key,cf:c', 'serialization.format'='1') TBLPROPERTIES
> ('hbase.table.name'='test_hbase1', 'storage_handler'='o
> rg.apache.hadoop.hive.hbase.HBaseStorageHandler');
> {code}
> Then issue query at impala shell:
> {code}
> select * from abc.test_hbase1 where k != "row1";
> {code}
> Observe:
> {code}
> Query: select * from abc.test_hbase1 where k != "row1"
>
> Query submitted at: 2018-12-04 17:02:42 (Coordinator: http://xyz:25000)
> ERROR: InternalException: Required field 'qualifier' was not present! Struct:
> THBaseFilter(family::key, qualifier:null, op_ordinal:3, filter_constant:row1)
> {code}
> More observations:
> # Replacing {{k != "row1"}} with {{k <> "row1"}} fails the same way. However,
> replacing it with other operators, such as ">", "<", "=", all works.
> # Replacing {{k != "row1}} with {{c != "row1"}}, it succeeded without the
> error reported above.
> The above example uses a two-column table, creating a similar table with
> three columns fails the same way: adding inequality predicate on the first
> column fails, adding inequility predicate doesn't fail.
> The code that issues the error message is in HBase, it seems Impala did not
> pass the needed info to HBase in this special case. Also wonder if it's
> because the first column of the table is the key in hbase table that could
> reveal the bug.
> {code}
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnIncrement.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required field
> 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnValue.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required field
> 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required
> field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required
> field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required
> field 'qualifier' was not present! Struct: " + toString());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
