[
https://issues.apache.org/jira/browse/FLINK-32188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729665#comment-17729665
]
Xin Chen edited comment on FLINK-32188 at 6/6/23 8:57 AM:
----------------------------------------------------------
To make a summary, when submitting an SQL task with Flink to test a customized
data source connector, I specified to query an array-type field of a temporary
table with a fixed-value array. For example, "select * from image-source where
URL=ARRAY ['/flink. jpg', '/flink_1. jpg']", but it couldn't obtain the
corresponding predicate filters at all in the connector's
DynamicTableSource.applyFilters method. The change related to
"RexNodeExtractor.scala" can fix it.
By the way, linked to https://issues.apache.org/jira/browse/CALCITE-5733.
Simplification seem to not take into account that the specified field is of
array type. In other words，it can simplify "a = 1 AND a = 2" to "false"，but can
not simplify “a = [1,2] AND a = [2,3]” to "false". For example, "select * from
image-source where URL=ARRAY ['/flink. jpg', '/flink_1. jpg'] AND URL=ARRAY
['/f. jpg', '/f_1. jpg']" can obtain two predicate conditions, this is
illogical. Generally speaking, simplifying this SQL condition should not result
in any predicates. Changes related to “RexSimplify.java” can fix it.
Maybe it is not standard. Thank you for your reply if anybody has better
suggestions.
was (Author: JIRAUSER298666):
To make a summary, when submitting an SQL task with Flink to test a customized
data source connector, I specified to query an array-type field of a temporary
table with a fixed-value array. For example, "select * from image-source where
URL=ARRAY ['/flink. jpg', '/flink_1. jpg']", but it couldn't obtain the
corresponding predicate filters at all in the connector's
DynamicTableSource.applyFilters method. The change related to
"RexNodeExtractor.scala" can fix it.
By the way, linked to https://issues.apache.org/jira/browse/CALCITE-5733.
Simplification seem to not take into account that the specified field is of
array type. In other words，it can simplify "a = 1 AND a = 2" to "false"，but can
not simplify “a = [1,2] AND a = [2,3]” to "false". For example, "select * from
image-source where URL=ARRAY ['/flink. jpg', '/flink_1. jpg'] AND URL=ARRAY
['/f. jpg', '/f_1. jpg']" can obtain two predicate conditions, this is
illogical. Generally speaking, simplifying this SQL condition should not result
in any predicates. Changes related to “RexSimplify.java” can fix it.
> Does the custom connector not support pushing down "where" query predicates
> to query fields of array type？
> ----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-32188
> URL: https://issues.apache.org/jira/browse/FLINK-32188
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 1.12.2, 1.17.0, 1.16.1
> Reporter: Xin Chen
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2023-05-25-17-16-02-288.png,
> image-2023-05-25-20-44-08-834.png, image-2023-05-25-20-44-47-581.png,
> image-2023-06-06-16-50-10-805.png, image-2023-06-06-16-50-54-467.png,
> screenshot-1.png, screenshot-10.png, screenshot-11.png, screenshot-12.png,
> screenshot-2.png, screenshot-3.png, screenshot-4.png, screenshot-5.png,
> screenshot-6.png, screenshot-7.png, screenshot-8.png, screenshot-9.png
>
>
> When I customized a data source connector which assumed as image-connector, I
> met issues while creating a table with ddl to specify a field "URL" as an
> array type. When submitting an SQL task with Flink, I specified query of this
> field with a fixed array. For example, "select * from image source where
> URL=ARRAY ['/flink. jpg', '/flink_1. jpg']", but it couldn't obtain the
> corresponding predicate filters at all.
> Does the custom connector not support to query fields of array type with
> "where"？
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
