[
https://issues.apache.org/jira/browse/KUDU-3518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17778944#comment-17778944
]
YifanZhang edited comment on KUDU-3518 at 10/24/23 6:17 AM:
------------------------------------------------------------
I see in profile_error_1.17.txt:
{code:java}
00:SCAN KUDU [member.qyexternaluserdetailinfo_new]
predicates: shoptype NOT IN ('35', '56'), thirdnick !=
''
kudu predicates: thirdnick IS NOT NULL, isDelete = CAST(0 AS INT),
ownercorpid IN ('xxxxx', ''), mainshopnick =
'xxxxx'
mem-estimate=3.00MB mem-reservation=0B thread-reservation=1
tuple-ids=0 row-size=32B cardinality=0
in pipelines: 00(GETNEXT) {code}
IIUC, kudu predicates mean the predicates that should be pushed down to the
kudu scanner, and the column 'shopnick' is not in kudu predicates. Since the
column 'shopnick' is neither a predicate column nor a projection column, it
shouldn't be scanned and shouldn't be used in the query execution. So it's
quite weird that the error is related to this column "Invalid argument: No such
column: shopnick".
It's more like an impala issue instead of kudu issue. I think you can try to
create an empty table to check if the error is related to the table schema:
{code:java}
create table new_empty_table like member.qyexternaluserdetailinfo_new;
// query the new_empty_table to see if the error happens{code}
was (Author: zhangyifan27):
I see in profile_error_1.17.txt:
{code:java}
00:SCAN KUDU [member.qyexternaluserdetailinfo_new]
predicates: shoptype NOT IN ('35', '56'), thirdnick !=
''
kudu predicates: thirdnick IS NOT NULL, isDelete = CAST(0 AS INT),
ownercorpid IN ('xxxxx', ''), mainshopnick =
'xxxxx'
mem-estimate=3.00MB mem-reservation=0B thread-reservation=1
tuple-ids=0 row-size=32B cardinality=0
in pipelines: 00(GETNEXT) {code}
IIUC, kudu predicates mean the predicates that should be pushed down to the
kudu scanner, and the column 'shopnick' is not in kudu predicates. Since the
column 'shopnick' is neither a predicate column nor a projection column, it
shouldn't be scanned and shouldn't be used in the query execution. So it's
quite weird that the error is related to this column "Invalid argument: No such
column: shopnick".
> node error when impala query
> ----------------------------
>
> Key: KUDU-3518
> URL: https://issues.apache.org/jira/browse/KUDU-3518
> Project: Kudu
> Issue Type: Bug
> Components: tserver
> Affects Versions: 1.17.0
> Environment: centos7.9
> Reporter: Pain Sun
> Priority: Major
> Attachments: profile_error_1.17.txt, profile_success_1.16.txt,
> profile_success_1.17.txt
>
>
> Scan kudu with impala-4.3.0 ,there is a bug when reading a table with an
> empty string in primary key field.
> sql:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
>
> error:ERROR: Unable to open scanner for node with id '1' for Kudu table
> 'impala::member.qyexternaluserdetailinfo_new': Invalid argument: No such
> column: shopnick
>
> If update sql like this:
> select
> count(distinct thirdnick)
> from
> member.qyexternaluserdetailinfo_new
> where
> (
> mainshopnick = "xxx"
> and ownercorpid in ("xxx", "")
> and shopnick not in ('')
> and shoptype not in ("35", "56")
> and isDelete = 0
> and thirdnick != ""
> and thirdnick is not null
> );
> no error.
>
> this error appears in kudu-1.17.0 ,but kudu-1.16.0 is good.
>
> There is 1000000 items in this table ,280000 items where empty string.
> table schema like this:
> +----------------+-----------+---------+-------------+------------+----------+---------------+---------------+---------------------+------------+
> | name | type | comment | primary_key | key_unique | nullable
> | default_value | encoding | compression | block_size |
> +----------------+-----------+---------+-------------+------------+----------+---------------+---------------+---------------------+------------+
> | mainshopnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | shopnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | ownercorpid | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | shoptype | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | clientid | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | thirdnick | string | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | id | bigint | | true | true | false
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | receivermobile | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | thirdrealname | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | remark | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | createtime | timestamp | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | updatetime | timestamp | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | isdelete | int | | false | | true
> | 0 | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> | buyernick | string | | false | | true
> | | AUTO_ENCODING | DEFAULT_COMPRESSION | 0 |
> +----------------+-----------+---------+-------------+------------+----------+---------------+---------------+---------------------+------------+
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
