[
https://issues.apache.org/jira/browse/IMPALA-12489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837958#comment-17837958
]
Alexey Serbin edited comment on IMPALA-12489 at 4/17/24 4:49 AM:
-----------------------------------------------------------------
[~MadBeeDo],
It turned out the root cause was a bug in Kudu. [[email protected]] and
I were able to reproduce it with {{kudu table scan}}, so the issue has been
localized.
The fix is available in Kudu's
[master|https://github.com/apache/kudu/commit/946acb711d722b1e6fe27af2c7de92960d724980]
and
[1.17.x|https://github.com/apache/kudu/commit/0de168f7e0abcf0c29facefcc9c0c9e12b284140]
branches. Follow-up patches are going to introduce test scenario(s) to
reproduce the issue with minimum amount of data to catch future regressions, if
any.
You could find more details in
[KUDU-3518|https://issues.apache.org/jira/browse/KUDU-3518].
Thank you for reporting the issue!
was (Author: aserbin):
[~MadBeeDo],
It turned out the root cause was a bug in Kudu. [[email protected]] and
I were able to reproduce it with {{kudu table scan}}, so the issue has been
localized.
The fix is available in Kudu's main and 1.17.x branches. Follow-up patches are
going to introduce test scenario(s) to reproduce the issue with minimum amount
of data to catch future regressions, if any.
You could find more details in
[KUDU-3518|https://issues.apache.org/jira/browse/KUDU-3518].
Thank you for reporting the issue!
> Error when scan kudu-1.17.0
> ---------------------------
>
> Key: IMPALA-12489
> URL: https://issues.apache.org/jira/browse/IMPALA-12489
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
> Affects Versions: Impala 4.3.0
> Environment: centos7.9
> Reporter: Pain Sun
> Priority: Major
> Labels: scankudu
>
> Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an
> empty string in primary key field.
> sql:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
>
> error:ERROR: Unable to open scanner for node with id '1' for Kudu table
> 'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
> column: shopnick
>
> If update sql like this:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shopnick not in ('')
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
> no error.
>
> this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
>
> There is 1000000 items in this table ,280000 items where empty string.
>
> create sql like this :
> CREATE TABLE member.qyexternaluserdetailinfo_new (
> mainshopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> shopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> ownercorpid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> shoptype STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> clientid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> thirdnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> id BIGINT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
> receivermobile STRING NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> thirdrealname STRING NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> remark STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
> createtime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> updatetime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> isdelete INT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION
> DEFAULT 0,
> buyernick STRING NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> PRIMARY KEY (
> mainshopnick,
> shopnick,
> ownercorpid,
> shoptype,
> clientid,
> thirdnick,
> id
> )
> ) PARTITION BY HASH (
> mainshopnick,
> shopnick,
> ownercorpid,
> shoptype,
> clientid,
> thirdnick,
> id
> ) PARTITIONS 10 STORED AS KUDU TBLPROPERTIES (
> 'kudu.master_addresses' = '192.168.134.132,192.168.134.133,192.168.134.134',
> 'kudu.num_tablet_replicas' = '1'
> );
> table schema like this:
> {+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
> |name |type
> |comment|primary_key|key_unique|nullable|default_value|encoding
> |compression |block_size|
> {+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
> |mainshopnick |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |shopnick |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |ownercorpid |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |shoptype |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |clientid |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |thirdnick |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |id |bigint | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |receivermobile|string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |thirdrealname |string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |remark |string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |createtime |timestamp| |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |updatetime |timestamp| |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |isdelete |int | |false | |true |0
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |buyernick |string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> {+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
