[
https://issues.apache.org/jira/browse/IMPALA-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16763161#comment-16763161
]
ASF subversion and git services commented on IMPALA-7929:
-
Commit 06d078e76bad6944ad487fd432ff6d0aa9174960 in impala's branch
refs/heads/master from Yongjun Zhang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=06d078e ]
IMPALA-7929: Allow null qualifier in THBaseFilter
Impala query failed with "IntenalException: Required field 'qualifier'
was not present!" on table created via hive and mapped to HBase, because
the qualifier of the HBase key column is null in the mapped table, and
Impala required non-null qualifier. The fix here is to relax this
requirement.
Test:
Added unit test.
Tested in real cluster.
Change-Id: I378c2249604481067b5b1c3a3bbb28c30ad4d751
Reviewed-on: http://gerrit.cloudera.org:8080/12213
Reviewed-by: Joe McDonnell
Tested-by: Tim Armstrong
> Impala query on HBASE table failing with InternalException: Required field*
> ---
>
> Key: IMPALA-7929
> URL: https://issues.apache.org/jira/browse/IMPALA-7929
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
>Affects Versions: Impala 3.2.0
>Reporter: Yongjun Zhang
>Assignee: Yongjun Zhang
>Priority: Major
>
> This looks a corner case bug demonstrated at impala-hbase boundary.
> The way to reproduce:
> Create a table in hive shell,
> {code}
> create database abc;
> CREATE TABLE abc.test_hbase1 (k STRING, c STRING) STORED BY
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ('
> hbase.columns.mapping'=':key,cf:c', 'serialization.format'='1') TBLPROPERTIES
> ('hbase.table.name'='test_hbase1', 'storage_handler'='o
> rg.apache.hadoop.hive.hbase.HBaseStorageHandler');
> {code}
> Then issue query at impala shell:
> {code}
> select * from abc.test_hbase1 where k != "row1";
> {code}
> Observe:
> {code}
> Query: select * from abc.test_hbase1 where k != "row1"
>
> Query submitted at: 2018-12-04 17:02:42 (Coordinator: http://xyz:25000)
> ERROR: InternalException: Required field 'qualifier' was not present! Struct:
> THBaseFilter(family::key, qualifier:null, op_ordinal:3, filter_constant:row1)
> {code}
> More observations:
> # Replacing {{k != "row1"}} with {{k <> "row1"}} fails the same way. However,
> replacing it with other operators, such as ">", "<", "=", all works.
> # Replacing {{k != "row1}} with {{c != "row1"}}, it succeeded without the
> error reported above.
> The above example uses a two-column table, creating a similar table with
> three columns fails the same way: adding inequality predicate on the first
> column fails, adding inequility predicate doesn't fail.
> The code that issues the error message is in HBase, it seems Impala did not
> pass the needed info to HBase in this special case. Also wonder if it's
> because the first column of the table is the key in hbase table that could
> reveal the bug.
> {code}
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnIncrement.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required field
> 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/TColumnValue.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required field
> 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required
> field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required
> field 'qualifier' was not present! Struct: " + toString());
> hbase-thrift/src/main/java/org/apache/hadoop/hbase/thrift2/generated/THBaseService.java:
> throw new org.apache.thrift.protocol.TProtocolException("Required
> field 'qualifier' was not present! Struct: " + toString());
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
