Yongjun Zhang created IMPALA-7929:
-------------------------------------
Summary: Impala query on HBASE table failing with
InternalException: Required field*
Key: IMPALA-7929
URL: https://issues.apache.org/jira/browse/IMPALA-7929
Project: IMPALA
Issue Type: Bug
Reporter: Yongjun Zhang
This looks a corner case bug demonstrated at impala-hbase boundary.
The way to reproduce:
Create a table in hive shell,
{code}
create database abc;
CREATE TABLE abc.test_hbase1 (k STRING, c STRING) STORED BY
'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('
hbase.columns.mapping'=':key,cf:c', 'serialization.format'='1') TBLPROPERTIES
('hbase.table.name'='test_hbase1', 'storage_handler'='o
rg.apache.hadoop.hive.hbase.HBaseStorageHandler');
{code}
Then issue query at impala shell:
{code}
select * from abc.test_hbase1 where k != "row1";
{code}
Observe:
{code}
Query: select * from abc.test_hbase1 where k != "row1"
Query submitted at: 2018-12-04 17:02:42 (Coordinator: http://xyz:25000)
ERROR: InternalException: Required field 'qualifier' was not present! Struct:
THBaseFilter(family::key, qualifier:null, op_ordinal:3, filter_constant:row1)
{code}
More observations:
# Replacing {{k != "row1"}} with {{k <> "row1"}} fails the same way. However,
replacing it with other operators, such as ">", "<", "=", all works.
# Replacing {{k != "row1}} with {{c != "row1"}}, it succeeded without the error
reported above.
The above example uses a two-column table, creating a similar table with three
columns fails the same way: adding inequality predicate on the first column
fails, adding inequility predicate doesn't fail.
The code that issues the error message is in HBase, it seems Impala did not
pass the needed info to HBase in this special case. Also wonder if it's because
the first column of the table is the key in hbase table that could reveal the
bug.
{code}
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnIncrement.java:
throw new org.apache.thrift.protocol.TProtocolException("Required field
'qualifier' was not present! Struct: " + toString());
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnValue.java:
throw new org.apache.thrift.protocol.TProtocolException("Required field
'qualifier' was not present! Struct: " + toString());
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
throw new org.apache.thrift.protocol.TProtocolException("Required field
'qualifier' was not present! Struct: " + toString());
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
throw new org.apache.thrift.protocol.TProtocolException("Required field
'qualifier' was not present! Struct: " + toString());
hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
throw new org.apache.thrift.protocol.TProtocolException("Required field
'qualifier' was not present! Struct: " + toString());
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
