[
https://issues.apache.org/jira/browse/IMPALA-12489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pain Sun updated IMPALA-12489:
------------------------------
Description:
Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an empty
string in primary key field.
sql:
select
count(distinct thirdnick)
from
member.qyexternaluserdetailinfo_new
where
(
mainshopnick = "xxx"
and ownercorpid in ("xxx", "")
and shoptype not in ("35", "56")
and isDelete = 0
and thirdnick != ""
and thirdnick is not null
);
error:ERROR: Unable to open scanner for node with id '1' for Kudu table
'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
column: shopnick
If update sql like this:
select
count(distinct thirdnick)
from
member.qyexternaluserdetailinfo_new
where
(
mainshopnick = "xxx"
and ownercorpid in ("xxx", "")
and shopnick not in ('')
and shoptype not in ("35", "56")
and isDelete = 0
and thirdnick != ""
and thirdnick is not null
);
no error.
this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
There is 1000000 items in this table ,280000 items where empty string.
create sql like this :
CREATE TABLE member.qyexternaluserdetailinfo_new (
mainshopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
shopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
ownercorpid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
shoptype STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
clientid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
thirdnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
id BIGINT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
receivermobile STRING NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
thirdrealname STRING NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
remark STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
createtime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
updatetime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION
DEFAULT_COMPRESSION,
isdelete INT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION
DEFAULT 0,
buyernick STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
PRIMARY KEY (
mainshopnick,
shopnick,
ownercorpid,
shoptype,
clientid,
thirdnick,
id
)
) PARTITION BY HASH (
mainshopnick,
shopnick,
ownercorpid,
shoptype,
clientid,
thirdnick,
id
) PARTITIONS 10 STORED AS KUDU TBLPROPERTIES (
'kudu.master_addresses' = '192.168.134.132,192.168.134.133,192.168.134.134',
'kudu.num_tablet_replicas' = '1'
);
table schema like this:
{+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
|name |type
|comment|primary_key|key_unique|nullable|default_value|encoding
|compression |block_size|
{+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
|mainshopnick |string | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|shopnick |string | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|ownercorpid |string | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|shoptype |string | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|clientid |string | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|thirdnick |string | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|id |bigint | |true |true |false |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|receivermobile|string | |false | |true |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|thirdrealname |string | |false | |true |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|remark |string | |false | |true |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|createtime |timestamp| |false | |true |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|updatetime |timestamp| |false | |true |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|isdelete |int | |false | |true |0
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
|buyernick |string | |false | |true |
|AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
{+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
was:
Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an empty
string in primary key field.
sql:
select
count(distinct thirdnick)
from
member.qyexternaluserdetailinfo_new
where
(
mainshopnick = "xxx"
and ownercorpid in ("xxx", "")
and shoptype not in ("35", "56")
and isDelete = 0
and thirdnick != ""
and thirdnick is not null
);
error:ERROR: Unable to open scanner for node with id '1' for Kudu table
'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
column: shopnick
If update sql like this:
select
count(distinct thirdnick)
from
member.qyexternaluserdetailinfo_new
where
(
mainshopnick = "xxx"
and ownercorpid in ("xxx", "")
and shopnick not in ('')
and shoptype not in ("35", "56")
and isDelete = 0
and thirdnick != ""
and thirdnick is not null
);
no error.
this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
There is 1000000 items in this table ,280000 items where empty string.
table schema like this:
+----------------+-----------+---------+-------------+------------+----------+---------------+---------------+---------------------+------------+
| name | type | comment | primary_key | key_unique | nullable |
default_value | encoding | compression | block_size |
+----------------+-----------+---------+-------------+------------+----------+---------------+---------------+---------------------+------------+
| mainshopnick | string | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| shopnick | string | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| ownercorpid | string | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| shoptype | string | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| clientid | string | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| thirdnick | string | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| id | bigint | | true | true | false |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| receivermobile | string | | false | | true |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| thirdrealname | string | | false | | true |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| remark | string | | false | | true |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| createtime | timestamp | | false | | true |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| updatetime | timestamp | | false | | true |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| isdelete | int | | false | | true |
0 | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
| buyernick | string | | false | | true |
| AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
+----------------+-----------+---------+-------------+------------+----------+---------------+---------------+---------------------+------------+
> Error when scan kudu-1.17.0
> ---------------------------
>
> Key: IMPALA-12489
> URL: https://issues.apache.org/jira/browse/IMPALA-12489
> Project: IMPALA
> Issue Type: Bug
> Components: Backend, be
> Affects Versions: Impala 4.3.0
> Environment: centos7.9
> Reporter: Pain Sun
> Priority: Major
> Labels: scankudu
>
> Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an
> empty string in primary key field.
> sql:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
>
> error:ERROR: Unable to open scanner for node with id '1' for Kudu table
> 'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
> column: shopnick
>
> If update sql like this:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shopnick not in ('')
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
> no error.
>
> this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
>
> There is 1000000 items in this table ,280000 items where empty string.
>
> create sql like this :
> CREATE TABLE member.qyexternaluserdetailinfo_new (
> mainshopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> shopnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> ownercorpid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> shoptype STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> clientid STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> thirdnick STRING NOT NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> id BIGINT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
> receivermobile STRING NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> thirdrealname STRING NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> remark STRING NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
> createtime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> updatetime TIMESTAMP NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> isdelete INT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION
> DEFAULT 0,
> buyernick STRING NULL ENCODING AUTO_ENCODING COMPRESSION
> DEFAULT_COMPRESSION,
> PRIMARY KEY (
> mainshopnick,
> shopnick,
> ownercorpid,
> shoptype,
> clientid,
> thirdnick,
> id
> )
> ) PARTITION BY HASH (
> mainshopnick,
> shopnick,
> ownercorpid,
> shoptype,
> clientid,
> thirdnick,
> id
> ) PARTITIONS 10 STORED AS KUDU TBLPROPERTIES (
> 'kudu.master_addresses' = '192.168.134.132,192.168.134.133,192.168.134.134',
> 'kudu.num_tablet_replicas' = '1'
> );
> table schema like this:
> {+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
> |name |type
> |comment|primary_key|key_unique|nullable|default_value|encoding
> |compression |block_size|
> {+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
> |mainshopnick |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |shopnick |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |ownercorpid |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |shoptype |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |clientid |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |thirdnick |string | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |id |bigint | |true |true |false |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |receivermobile|string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |thirdrealname |string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |remark |string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |createtime |timestamp| |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |updatetime |timestamp| |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |isdelete |int | |false | |true |0
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> |buyernick |string | |false | |true |
> |AUTO_ENCODING|DEFAULT_COMPRESSION|0 |
> {+}---------------{-}{-}{+}---------{-}++{-}-------{-}{-}-----------{-}++{-}----------{-}{-}--------{-}++{-}-------------{-}{-}-------------{-}++{-}-------------------{-}{-}-----------+
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
